perf(cluster): cache the worked-index and bound the spot-status cache (RBN firehose)

A user on a slow PC with RBN saw OpsLog at 94% CPU and 8.5 GB RAM (Logger32: 3%
/ 34 MB on the same feeds). Two runaway costs under the spot firehose:

- ClusterSpotStatuses re-scanned the ENTIRE logbook (5-6 full-table maps) on
  every 50 ms spot batch — ~20×/second — its "one scan regardless of batch" doc
  was untrue. On a big log that's millions of row-scans/second → the pegged CPU.
  Now cached in clusterStatusCache (an immutable snapshot), rebuilt only when the
  logbook changes (noteWorked on a single log, invalidateAwardStats on bulk), so
  it's one scan per logged QSO instead of per batch.

- The frontend spotStatus map had no cap: one entry per call|band|mode ever seen,
  and RBN produces thousands of unique calls/hour → unbounded growth to GBs, plus
  a full {...prev} copy 20×/second. Now pruned back to the live (SPOTS_CAP=1000)
  spots once it drifts past 2×, with a cheap same-reference bail-out otherwise.
This commit is contained in:
2026-08-07 00:08:24 +02:00
parent e2bfe73bdb
commit 9a72afd467
3 changed files with 115 additions and 36 deletions
+4 -2
View File
@@ -4,11 +4,13 @@
"date": "",
"en": [
"E-mail: the QSO recording e-mail (subject and body) is now editable in Settings → E-mail, like the QSL card e-mail — with the {CALL} {DATE} {BAND} {MODE} {MYCALL} variables.",
"Backup fix: the backup now saves your CONTACTS (the logbook), not just the settings. Once the logbook was split into its own file, the backup kept snapshotting the settings database and silently missed the QSOs. It now writes the log to opslog-*.db and the configuration separately to opslogcfg-*.db (MySQL logs still export to ADIF)."
"Backup fix: the backup now saves your CONTACTS (the logbook), not just the settings. Once the logbook was split into its own file, the backup kept snapshotting the settings database and silently missed the QSOs. It now writes the log to opslog-*.db and the configuration separately to opslogcfg-*.db (MySQL logs still export to ADIF).",
"Performance: much lower CPU and memory on a busy cluster (RBN and other high-volume feeds). The Cluster tab was re-scanning the entire logbook for every batch of spots (~20×/second) and its spot-status cache grew without limit — on a firehose that could climb to gigabytes of RAM and peg a CPU. The worked-index is now cached (rebuilt only when you log a QSO) and the status cache is bounded to the spots actually shown."
],
"fr": [
"E-mail : le texte de l'e-mail d'enregistrement QSO (objet et corps) est désormais modifiable dans Réglages → E-mail, comme l'e-mail de carte QSL — avec les variables {CALL} {DATE} {BAND} {MODE} {MYCALL}.",
"Correction sauvegarde : la sauvegarde enregistre désormais tes CONTACTS (le journal), et plus seulement les réglages. Depuis que le journal a été séparé dans son propre fichier, la sauvegarde continuait à copier la base des réglages et oubliait les QSO. Elle écrit maintenant le log dans opslog-*.db et la configuration à part dans opslogcfg-*.db (les logs MySQL restent exportés en ADIF)."
"Correction sauvegarde : la sauvegarde enregistre désormais tes CONTACTS (le journal), et plus seulement les réglages. Depuis que le journal a été séparé dans son propre fichier, la sauvegarde continuait à copier la base des réglages et oubliait les QSO. Elle écrit maintenant le log dans opslog-*.db et la configuration à part dans opslogcfg-*.db (les logs MySQL restent exportés en ADIF).",
"Performances : CPU et mémoire nettement réduits sur un cluster chargé (RBN et autres flux à fort volume). L'onglet Cluster rescannait tout le journal à chaque lot de spots (~20×/seconde) et son cache de statuts grossissait sans limite — sur un flux intense cela pouvait atteindre des gigaoctets de RAM et saturer un cœur. L'index des contacts est désormais mis en cache (reconstruit seulement quand tu logues un QSO) et le cache de statuts est borné aux spots réellement affichés."
]
},
{