Compare commits

..
12 Commits
Author SHA1 Message Date
rouggy 1348dc2177 chore: release v0.24.8 2026-08-13 01:26:40 +02:00
rouggy 57ac382f91 fix(appearance): colour rows queued for LoTW upload, not just paper cards
The "to send" rule only looked at qsl_sent, so a contact waiting to go to LoTW
— marked R, and the commonest not-gone-out-yet state in a digital log — matched
nothing and stayed uncoloured. It now covers any route still queued, and the
label says so.
2026-08-13 01:14:44 +02:00
rouggy 858c04d267 feat(appearance): colour log rows by QSL status, and fix UDP QSO numbering
New Settings → Appearance section: whole-row colouring in the log grid driven
by QSL / LoTW state, each rule with its own colour from a palette or a free
picker. What Logger32 does, with one difference that matters — the colour is
applied as a 24% tint, not a fill. Logger32's grid is white; this one is dark,
and a saturated user-picked colour behind white text is unreadable at exactly
the moment the operator is scanning for what still needs sending.

Rules are ORDERED and the first match wins, because a contact is usually
several of these at once: one confirmed on LoTW and by card is confirmed, not
"sent, awaiting reply". The order lives in the data so the panel can show the
rules in the order they actually apply, numbered.

The colour is interpolated into a CSS color-mix(), so anything that is not
plainly #rrggbb is refused on the way in and falls back to the default.

Also fixes the QSO number column, which was empty for contacts logged from
WSJT-X: that path inserts through the repo directly and never reached AddQSO.
Rather than chase each of the remaining bulk-insert paths — the POTA hunter
import and the LoTW/QRZ "add what I was missing" passes, which insert OLD
dates and so shift every number after them — the index now checks its length
against a COUNT and rebuilds when they disagree. One indexed count beats
remembering to invalidate in a place that does not exist yet.
2026-08-13 01:06:21 +02:00
rouggy 2484cd2515 fix(log): the QSO number column was always empty
OrderedIDs scanned qso_date straight into a time.Time. The column holds a
formatted STRING — the repo writes it with Format(isoMillis) and reads it back
through parseTimeLoose everywhere else — so the scan failed on every call, the
index was never built, and omitempty then dropped the zero from the JSON. A
column that shipped blank.

Scans the string and parses only the newest row; the ordering already came from
SQL, so 30 000 parses were never needed.

Tested against a real SQLite file with the date stored exactly as the repo
writes it, and with the ids running opposite to the dates — the imported-ADIF
case that is the whole reason this is not the id. It fails without the fix with
the scan error itself.
2026-08-13 00:37:53 +02:00
rouggy 62256942a1 feat(log): QSO number column, oldest contact = 1
Not the id: the primary key follows insertion order, so importing an old ADIF
gives the oldest contacts the highest ids. This is a rank over qso_date.

Computed over the WHOLE log, not the query result — ranking inside the result
would renumber every contact the moment a filter is applied, and QSO #1 would
change identity as the operator typed.

Held as one id-to-rank map, built from a single ordered id query and dropped
with the other derived indexes when the log changes. A contact logged from the
entry form is the newest, so it takes the next number without rereading the
log: AddQSO deliberately avoids full invalidation because a contest run would
pay for it once per QSO. A contact entered with an OLDER date belongs in the
middle of the order, so there the map is dropped and rebuilt rather than
mis-numbered.
2026-08-13 00:31:38 +02:00
rouggy 3c8aadf41f chore(gridcache): log what each batch writes
A batch that never reaches the disk looks exactly like one that does: the
locators are in memory either way until the next restart, which is the one
moment the difference shows. The shutdown flush was the least observable of
all — it runs last, and nothing said whether it had written anything.
2026-08-12 17:39:16 +02:00
rouggy 2f5fd63afd fix(lookup): read the fields the providers were already sending
Audit of every field a lookup returns against what OpsLog can store, prompted
by a report that HamQTH was not fetching the email.

The email was never broken: pinned now against a captured HamQTH answer, it
parses and reaches the QSO. The reported callsign simply has no public address
on HamQTH, which the empty field could not distinguish from a fault — so the
"[email protected]" placeholder is gone. A greyed-out sample address in the one
field an operator checks to see whether the lookup found one reads as a value.

Real gaps found and closed:

- web: the qso table has had the column since migration 0003 and no provider
  mapping ever read it. HamQTH sends <web>.
- picture: Result.ImageURL was documented "QRZ only" because HamQTH's element
  is <picture>, not <image>. It was there all along.
- zip: sent by both providers, read by neither.
- adr_name: used only as a fallback when a record carries neither nick nor
  name. Deliberately NOT preferred over <nick> — a log wants the name the
  operator goes by on the air, "Igor", not "Igor Vladimirovich Getman".

The parse tests use a real captured payload, including the stray <div> advert
the server injects into its own XML.
2026-08-12 17:34:45 +02:00
rouggy 4f3c3edaee fix(bandopen): honour the operator's band selection
The chips in Settings only ever shaped the PSK Reporter SUBSCRIPTION. Neither
feed path checked them: both asked bandopen.Watched, which says which bands the
detector is capable of and knows nothing about the selection. So the cluster
path announced every watched band regardless, and widening the subscription to
"+" for grid chasing let PSK Reporter do the same — a station with only 6 m
ticked got 10 m and 2 m badges.

The selection is now cached beside the on/off flag and checked on both paths,
and unticking a band puts its badge out: badges fade on a timer fed by spots
the detector no longer looks at, so it would otherwise stay lit until a
restart.
2026-08-12 17:25:16 +02:00
rouggy 7d33379fe1 feat(cluster): feed the locator store from PSK Reporter, filtered at the broker
The store shipped without its main source. Locators came only from this
station's own WSJT-X decodes, which is what the whole MQTT discussion was
about.

PSK Reporter now feeds it through a new OnGrid callback, fired before any
geographic filtering: what the store wants is "which square is this callsign
in", and that is true whoever happened to hear the report.

The subscription filters on the RECEIVER's square, a level the v2 topic
exposes. Measured on the live feed: the four opening bands unfiltered are 83
messages a second, of which roughly one in a hundred survived the NearKm test
that already existed here — the rest was received, TLS-decrypted, JSON-parsed
and discarded. One ring of squares is 0.2 to 1.2 a second.

By square rather than by DXCC, which was the obvious alternative: one country
measured 1.2 messages a second (OH) against 72.5 (K) on a single band, because
a DXCC can be a continent. By square the same measurement is 0.2 to 1.2, so the
load follows distance — what the feed is actually about — and is the same for
every operator.

Grid chasing subscribes with the "+" band wildcard, so one subscription per
square covers every band instead of one per band per square.

The store gains a source column (decode | mqtt), migrated in place on an
existing file.
2026-08-12 17:16:47 +02:00
rouggy 82a49150d2 feat(cluster): persist learnt locators behind a "Chase new grids" option
The callsign->grid map died with the process. Every restart began with an empty
Locator column that took an hour of listening to refill, and everything learnt
the day before was thrown away.

internal/gridcache is its own SQLite file in the data directory, not a table in
the settings database: that one sits wherever the operator chose to put it,
often a synchronised folder, and a store that rewrites itself every minute has
no business there. Deleting the file costs a few days of listening and nothing
else.

Callsign is the primary key and the newest report wins — operators move, go
portable, go on expedition, and a stale locator is worse than none for grid
chasing because it reads as a square already worked. Entries not seen in two
years are pruned at open: that is the one way this cache can be actively wrong
rather than merely empty, and it is what bounds a store that would otherwise
only grow.

Only CHANGES are queued. The feeds repeat themselves, so writing every report
would put the whole stream in the batch instead of the news in it. Batches
flush on a timer and on shutdown — a restart is exactly when the cache is worth
the most.

Rotation is switched off while the store is attached: the cap would discard
callsigns the database still holds and the lookup would then miss what we know.
Age in the store is the bound instead.
2026-08-12 17:01:05 +02:00
rouggy 1ef9e553f7 perf(cluster): rotate the grid cache instead of emptying it, cap 100k
The cache dropped EVERY entry once it passed 20 000. That was survivable while
the only feed was this station's own WSJT-X decodes, which never reached the
ceiling — it is a cliff for anything larger, and every locator in the cluster
list would disappear at once, periodically, for no reason the operator could
see.

Two generations: when the current map fills it becomes the previous one and a
fresh map takes over; lookups consult both. A rotation therefore costs the
older half and nothing more. It needs no insertion order, no per-entry
timestamp and no bookkeeping on the write path — all of which "evict the
oldest thousand" would require, on a path that runs once per decode.

The cap is 100 000, measured at 82 bytes an entry: 8 MB a generation, 16 MB
for both. 20 000 was chosen when the ceiling was unreachable anyway.

Both maps now go through rememberDecodeGrid / lookupDecodeGrid. Rotation swaps
the map headers, so the read had to be guarded; routing every access through
one accessor is what makes that checkable rather than remembered.
2026-08-12 16:48:23 +02:00
rouggy 39fab68bd4 chore: open 0.24.8 2026-08-12 14:46:05 +02:00
33 changed files with 2002 additions and 58 deletions
+293 -20
View File
@@ -41,6 +41,7 @@ import (
"hamlog/internal/email" "hamlog/internal/email"
"hamlog/internal/extsvc" "hamlog/internal/extsvc"
"hamlog/internal/geo" "hamlog/internal/geo"
"hamlog/internal/gridcache"
"hamlog/internal/integrations/udp" "hamlog/internal/integrations/udp"
"hamlog/internal/lookup" "hamlog/internal/lookup"
"hamlog/internal/lotwusers" "hamlog/internal/lotwusers"
@@ -235,6 +236,8 @@ const (
keyUltrabeamStep = "ultrabeam.step_khz" // re-tune hysteresis: 25 | 50 | 100 kHz keyUltrabeamStep = "ultrabeam.step_khz" // re-tune hysteresis: 25 | 50 | 100 kHz
keyMotorTrackMode = "motor.track_mode" // "always" | "step" | "band" keyMotorTrackMode = "motor.track_mode" // "always" | "step" | "band"
keyMotorBandFreqs = "motor.band_freqs" // per-band tune frequency: "40m=7100,20m=14150" keyMotorBandFreqs = "motor.band_freqs" // per-band tune frequency: "40m=7100,20m=14150"
keyChaseNewGrids = "cluster.chase_grids" // "1" → persist learnt locators across restarts
keyRowColors = "appearance.row_colors"
keyMotorType = "ultrabeam.type" // "ultrabeam" | "steppir" (default ultrabeam) keyMotorType = "ultrabeam.type" // "ultrabeam" | "steppir" (default ultrabeam)
keyMotorTransport = "ultrabeam.transport" // "tcp" | "serial" (default tcp) keyMotorTransport = "ultrabeam.transport" // "tcp" | "serial" (default tcp)
keyMotorCOM = "ultrabeam.com" // serial device name (COM3, /dev/ttyUSB0) keyMotorCOM = "ultrabeam.com" // serial device name (COM3, /dev/ttyUSB0)
@@ -601,6 +604,14 @@ type App struct {
// or when a setting that shapes the maps flips. // or when a setting that shapes the maps flips.
clusterStatusIdx *clusterStatusCache clusterStatusIdx *clusterStatusCache
clusterStatusMu sync.Mutex clusterStatusMu sync.Mutex
// qsoNumbers maps a QSO id to its chronological position, oldest = 1. Built
// on demand from one ordered id query and dropped whenever the log changes,
// alongside the other derived indexes.
qsoNumbers map[int64]int
// qsoNumMax is the date of the newest contact the map has numbered, so a QSO
// logged now can be appended instead of forcing a rebuild.
qsoNumMax time.Time
qsoNumMu sync.Mutex
// decodeGrids maps a callsign to the 4-character grid it announced in a CQ // decodeGrids maps a callsign to the 4-character grid it announced in a CQ
// heard over the WSJT-X UDP link. It is the ONLY source of grids we have for // heard over the WSJT-X UDP link. It is the ONLY source of grids we have for
// a spot: a DX-cluster line carries the spotter's grid at best, never the // a spot: a DX-cluster line carries the spotter's grid at best, never the
@@ -611,8 +622,23 @@ type App struct {
// //
// In memory only, and bounded: it is a session-local view of who is on the // In memory only, and bounded: it is a session-local view of who is on the
// air now, not a database. // air now, not a database.
decodeGrids map[string]string //
decodeGridsMu sync.RWMutex // Bounded in TWO generations. The cap used to drop the whole map, which was
// survivable while the only source was this station's own decodes — it never
// reached the ceiling. It is a cliff for any larger feed: every locator in the
// list would vanish at once, periodically. Keeping the previous generation
// means a rotation costs the older half and nothing more, and it needs no
// insertion order, no per-entry timestamp and no bookkeeping on the write
// path — which "evict the oldest thousand" would all require.
decodeGrids map[string]string // current generation, written to
decodeGridsOld map[string]string // previous generation, still readable
decodeGridsMu sync.RWMutex
// gridStore persists the map across restarts when grid chasing is on. With
// it, rotation is switched OFF: rotating would drop callsigns the database
// still holds, and a lookup would then miss something we know. The store
// bounds itself by age instead, so memory follows how many distinct stations
// have actually been heard in two years rather than a made-up ceiling.
gridStore *gridcache.Store
// pskr is the PSK Reporter MQTT feed, up only while the opening watch is on. // pskr is the PSK Reporter MQTT feed, up only while the opening watch is on.
// It is the source that makes VHF detection work at all: the cluster and RBN // It is the source that makes VHF detection work at all: the cluster and RBN
// carry a handful of 6 m spots where PSK Reporter carries hundreds. // carry a handful of 6 m spots where PSK Reporter carries hundreds.
@@ -1375,8 +1401,11 @@ func (a *App) startup(ctx context.Context) {
go a.sendTelemetryHeartbeat() go a.sendTelemetryHeartbeat()
go a.liveStatusLoop() // multi-op: heartbeat current activity to shared MySQL go a.liveStatusLoop() // multi-op: heartbeat current activity to shared MySQL
go a.chatLoop() // multi-op: poll the shared chat + heartbeat presence go a.chatLoop() // multi-op: poll the shared chat + heartbeat presence
// PSK Reporter, when the opening watch is on. After the operator's grid is // Locator store BEFORE the feed: the feed asks whether it exists to decide
// known: without it there is no distance to measure and the feed stays down. // which bands to subscribe to.
a.startGridCache()
// PSK Reporter. After the operator's grid is known: without it there is no
// distance to measure and no receiver squares to filter on, so it stays down.
a.startBandOpenFeed() a.startBandOpenFeed()
// One-time tidy-up of a field nothing used to record. Background, once. // One-time tidy-up of a field nothing used to record. Background, once.
a.backfillDistancesOnce() a.backfillDistancesOnce()
@@ -1625,6 +1654,14 @@ func (a *App) shutdown(ctx context.Context) {
if a.qsoRec != nil { if a.qsoRec != nil {
a.qsoRec.Stop() a.qsoRec.Stop()
} }
// Before the databases: Close flushes what the last minute learnt, and a
// restart is exactly when the grid cache is worth the most.
if a.gridStore != nil {
if err := a.gridStore.Close(); err != nil {
applog.Printf("gridcache: close: %v", err)
}
a.gridStore = nil
}
if a.logDb != nil && a.logDb != a.db { if a.logDb != nil && a.logDb != a.db {
_ = a.logDb.Close() // shared MySQL logbook (separate from the local config DB) _ = a.logDb.Close() // shared MySQL logbook (separate from the local config DB)
} }
@@ -2682,6 +2719,9 @@ func (a *App) AddQSO(q qso.QSO) (id int64, err error) {
a.clusterStatusMu.Lock() a.clusterStatusMu.Lock()
a.clusterStatusIdx = nil a.clusterStatusIdx = nil
a.clusterStatusMu.Unlock() a.clusterStatusMu.Unlock()
// Give the contact its number without rereading the log — same reason as
// above, a contest run must not pay a full scan per QSO.
a.noteQSONumbered(id, q.QSODate)
// Announce the log RIGHT AWAY so the grid/UI refresh at once and the entry // Announce the log RIGHT AWAY so the grid/UI refresh at once and the entry
// form clears immediately — the operator is not made to wait on the DB. // form clears immediately — the operator is not made to wait on the DB.
wruntime.EventsEmit(a.ctx, "qso:logged", id) wruntime.EventsEmit(a.ctx, "qso:logged", id)
@@ -4440,6 +4480,11 @@ func (a *App) invalidateAwardStats() {
a.clusterStatusMu.Lock() a.clusterStatusMu.Lock()
a.clusterStatusIdx = nil a.clusterStatusIdx = nil
a.clusterStatusMu.Unlock() a.clusterStatusMu.Unlock()
// The numbering shifts whenever a contact is added or removed — and an
// imported ADIF inserts into the MIDDLE of the order, not at the end.
a.qsoNumMu.Lock()
a.qsoNumbers = nil
a.qsoNumMu.Unlock()
// Bulk QSO changes (import, delete, bulk edit) also land here — refresh the // Bulk QSO changes (import, delete, bulk edit) also land here — refresh the
// worked-index so alert "needed" checks stay accurate. Async: never block the // worked-index so alert "needed" checks stay accurate. Async: never block the
// mutation, and it's a single lightweight query. // mutation, and it's a single lightweight query.
@@ -5763,7 +5808,96 @@ func (a *App) ListQSOFiltered(f qso.QueryFilter) ([]qso.QSO, error) {
if a.qso == nil { if a.qso == nil {
return nil, fmt.Errorf("db not initialized") return nil, fmt.Errorf("db not initialized")
} }
return a.qso.ListFiltered(a.ctx, f) list, err := a.qso.ListFiltered(a.ctx, f)
if err != nil {
return nil, err
}
a.stampQSONumbers(list)
return list, nil
}
// stampQSONumbers fills each QSO's position in the log, oldest = 1.
//
// Stamped here rather than computed in the query because the number has to be
// the rank in the WHOLE log: ranking inside the result would renumber every
// contact the moment a filter is applied, and the operator would see QSO #1
// change identity as they typed.
func (a *App) stampQSONumbers(list []qso.QSO) {
idx := a.qsoNumberIndex()
if idx == nil {
return
}
for i := range list {
list[i].Number = idx[list[i].ID]
}
}
// countQSOForNumbering is Count with the nil-repo guard the numbering needs —
// it runs before the database is up on a fresh start.
func (a *App) countQSOForNumbering() (int64, error) {
if a.qso == nil {
return 0, fmt.Errorf("db not initialized")
}
return a.qso.Count(a.ctx)
}
// qsoNumberIndex returns id → chronological position, building it once and
// keeping it until the log changes (invalidateAwardStats drops it).
func (a *App) qsoNumberIndex() map[int64]int {
a.qsoNumMu.Lock()
defer a.qsoNumMu.Unlock()
if a.qsoNumbers != nil {
// Cheap consistency check rather than trusting every insert path to have
// remembered to invalidate. Several bulk paths — the POTA hunter import,
// the LoTW and QRZ "add contacts I was missing" passes — insert straight
// through the repo, and they add contacts with OLD dates, which lands them
// in the MIDDLE of the order and shifts every number after them. A stale
// map there is not merely incomplete, it is wrong.
//
// One indexed COUNT against a map length, versus rereading every id.
if n, err := a.countQSOForNumbering(); err == nil && int(n) != len(a.qsoNumbers) {
a.qsoNumbers = nil
} else {
return a.qsoNumbers
}
}
if a.qso == nil {
return nil
}
ids, newest, err := a.qso.OrderedIDs(a.ctx)
if err != nil {
applog.Printf("qso numbering: %v — the column will be empty", err)
return nil
}
m := make(map[int64]int, len(ids))
for i, id := range ids {
m[id] = i + 1
}
a.qsoNumbers = m
a.qsoNumMax = newest
return m
}
// noteQSONumbered keeps the numbering current for a contact just logged.
//
// A QSO logged from the entry form is the newest there is, so it simply takes
// the next number — a full rebuild would read every id in the log, which during
// a contest run would cost more than the three queries the grid refresh already
// makes. A contact entered with an OLDER date belongs in the middle of the
// order, so there the map is dropped and rebuilt correctly rather than
// mis-numbered.
func (a *App) noteQSONumbered(id int64, at time.Time) {
a.qsoNumMu.Lock()
defer a.qsoNumMu.Unlock()
if a.qsoNumbers == nil {
return // nothing built yet; the lazy build will see this QSO anyway
}
if at.Before(a.qsoNumMax) {
a.qsoNumbers = nil
return
}
a.qsoNumbers[id] = len(a.qsoNumbers) + 1
a.qsoNumMax = at
} }
// CountQSOFiltered returns how many QSOs match the filter (ignoring the row // CountQSOFiltered returns how many QSOs match the filter (ignoring the row
@@ -11879,6 +12013,10 @@ func (a *App) LogUDPLoggedADIF(adifText string) (int64, error) {
return 0, fmt.Errorf("insert qso: %w", err) return 0, fmt.Errorf("insert qso: %w", err)
} }
q.ID = id q.ID = id
// Same as the manual path: give the contact its number without rereading the
// log. This insert bypasses AddQSO entirely, which is why UDP-logged contacts
// came out unnumbered while hand-logged ones did not.
a.noteQSONumbered(id, q.QSODate)
a.noteLiveQSO() // multi-op: flip this operator back "online" (publishes async) a.noteLiveQSO() // multi-op: flip this operator back "online" (publishes async)
// Announce the log AT ONCE so the grid / ON-AIR badge / stations-on-air widget // Announce the log AT ONCE so the grid / ON-AIR badge / stations-on-air widget
// refresh immediately, then run the DB-heavy enrichment off the critical path // refresh immediately, then run the DB-heavy enrichment off the critical path
@@ -11937,15 +12075,7 @@ func (a *App) consumeUDPEvents() {
// Remember the grid before anything else: a CQ is the one message that // Remember the grid before anything else: a CQ is the one message that
// carries it, and the station may never send another. // carries it, and the station may never send another.
if ev.DecodeGrid != "" { if ev.DecodeGrid != "" {
a.decodeGridsMu.Lock() a.rememberDecodeGrid(ev.DecodeCall, ev.DecodeGrid, gridcache.SourceDecode)
if a.decodeGrids == nil {
a.decodeGrids = make(map[string]string, 512)
}
if len(a.decodeGrids) > 20000 {
a.decodeGrids = make(map[string]string, 512) // bound a long session
}
a.decodeGrids[strings.ToUpper(ev.DecodeCall)] = ev.DecodeGrid
a.decodeGridsMu.Unlock()
} }
// A WSJT-X decode (heard station). Render it on the FlexRadio // A WSJT-X decode (heard station). Render it on the FlexRadio
// panadapter when the option is on; green + SNR comment, auto-expiring // panadapter when the option is on; green + SNR comment, auto-expiring
@@ -17014,6 +17144,151 @@ func (a *App) clusterStatusMaps() *clusterStatusCache {
// was ambiguous and the frontend couldn't infer) we degrade gracefully // was ambiguous and the frontend couldn't infer) we degrade gracefully
// to band-only — saying "worked" rather than wrongly flagging "new-slot" // to band-only — saying "worked" rather than wrongly flagging "new-slot"
// just because we don't know the mode. // just because we don't know the mode.
// decodeGridsCap is how many callsigns one generation holds before it rotates.
//
// 100 000 measured at 82 bytes an entry — 8 MB a generation, 16 MB for both,
// which is nothing next to what it buys: a locator on a spot instead of an empty
// column. The old 20 000 was chosen when the only feed was this station's own
// decodes and the ceiling was never reached anyway.
const decodeGridsCap = 100000
// GetChaseNewGrids reports whether learnt locators are kept across restarts.
func (a *App) GetChaseNewGrids() bool { return a.settingOr(keyChaseNewGrids, "") == "1" }
// GridCacheStatus is the live count under the option. A store that is on but
// has learnt nothing looks exactly like a broken one until a number moves.
type GridCacheStatus struct {
Enabled bool `json:"enabled"`
Known int `json:"known"` // locators in memory, stored + learnt this session
Pending int `json:"pending"` // waiting for the next batch write
}
// GetGridCacheStatus reports what the locator store holds.
func (a *App) GetGridCacheStatus() GridCacheStatus {
out := GridCacheStatus{Enabled: a.gridStore != nil}
a.decodeGridsMu.RLock()
out.Known = len(a.decodeGrids) + len(a.decodeGridsOld)
a.decodeGridsMu.RUnlock()
if a.gridStore != nil {
out.Pending = a.gridStore.Pending()
}
return out
}
// SetChaseNewGrids turns grid chasing on or off and applies it immediately.
func (a *App) SetChaseNewGrids(on bool) error {
a.setSetting(keyChaseNewGrids, boolStr(on))
a.startGridCache()
// Resubscribe: with grid chasing the feed covers every band, without it only
// the opening bands — and with neither consumer it comes down entirely.
a.startBandOpenFeed()
return nil
}
// startGridCache brings the locator store up or down to match the setting.
//
// Switching it ON seeds the in-memory map from the database, which is the whole
// point: the cluster's locator column is populated in the first second instead
// of after an hour of listening. Switching it OFF closes the file and leaves the
// map alone — what has already been learnt this session stays usable, it simply
// stops being remembered. Nothing is deleted: turning the option back on picks
// up where it left off.
func (a *App) startGridCache() {
if a.gridStore != nil {
if err := a.gridStore.Close(); err != nil {
applog.Printf("gridcache: close: %v", err)
}
a.gridStore = nil
}
if !a.GetChaseNewGrids() {
return
}
path := filepath.Join(a.dataDir, "grids.db")
st, err := gridcache.Open(path, applog.Printf)
if err != nil {
applog.Printf("gridcache: disabled — %v", err)
return
}
seed, err := st.LoadAll()
if err != nil {
applog.Printf("gridcache: cannot read %s (%v) — starting empty", path, err)
seed = map[string]string{}
}
a.decodeGridsMu.Lock()
if a.decodeGrids == nil {
a.decodeGrids = make(map[string]string, len(seed)+512)
}
// Seed UNDER what this session already heard: a locator decoded a minute ago
// is newer than one stored days back, and the newest report is the one that
// counts when a station has moved.
for call, grid := range seed {
if _, live := a.decodeGrids[call]; !live {
a.decodeGrids[call] = grid
}
}
n := len(a.decodeGrids)
a.decodeGridsMu.Unlock()
a.gridStore = st
st.Start(a.ctx)
applog.Printf("gridcache: grid chasing on — %d locators loaded from %s (%d known in total)", len(seed), path, n)
}
// rememberDecodeGrid records the grid a station announced.
//
// Rotation, not eviction: when the current generation fills it becomes the
// previous one and a fresh map takes over. Nothing is scanned, nothing is
// timestamped, and the write stays a single map assignment — which matters
// because this runs once per decode.
func (a *App) rememberDecodeGrid(call, grid, source string) {
call = strings.ToUpper(strings.TrimSpace(call))
grid = strings.TrimSpace(grid)
if call == "" || grid == "" {
return
}
store := a.gridStore
a.decodeGridsMu.Lock()
if a.decodeGrids == nil {
a.decodeGrids = make(map[string]string, 512)
}
// Rotate only while nothing is persisting the map. With a store the cap would
// discard callsigns the database still holds, so the lookup would miss what
// we know; age in the store is the bound instead.
if store == nil && len(a.decodeGrids) >= decodeGridsCap {
a.decodeGridsOld = a.decodeGrids
a.decodeGrids = make(map[string]string, 512)
}
// Only a CHANGE is worth writing. The feeds repeat themselves — the same
// station is reported by dozens of receivers a minute — so queueing every
// report would put the whole stream in the batch instead of the news in it.
changed := a.decodeGrids[call] != grid
if changed && a.decodeGridsOld != nil && a.decodeGridsOld[call] == grid {
// Known already, just in the older generation: promote it without calling
// it news.
changed = false
}
a.decodeGrids[call] = grid
a.decodeGridsMu.Unlock()
if changed && store != nil {
store.Put(call, grid, source)
}
}
// lookupDecodeGrid returns the last grid heard for a callsign, "" if unknown.
// The previous generation is consulted second, so a station that has gone quiet
// keeps its locator across one rotation instead of losing it at the cliff.
func (a *App) lookupDecodeGrid(call string) string {
call = strings.ToUpper(call)
a.decodeGridsMu.RLock()
defer a.decodeGridsMu.RUnlock()
if g := a.decodeGrids[call]; g != "" {
return g
}
return a.decodeGridsOld[call]
}
func (a *App) ClusterSpotStatuses(spots []SpotQuery) []SpotStatus { func (a *App) ClusterSpotStatuses(spots []SpotQuery) []SpotStatus {
out := make([]SpotStatus, len(spots)) out := make([]SpotStatus, len(spots))
if a.qso == nil { if a.qso == nil {
@@ -17102,12 +17377,10 @@ func (a *App) ClusterSpotStatuses(spots []SpotQuery) []SpotStatus {
// is part of the key, so the "group digital modes" option decides whether a // is part of the key, so the "group digital modes" option decides whether a
// grid worked on FT8 still counts as new on FT4 — one rule, no branch here. // grid worked on FT8 still counts as new on FT4 — one rule, no branch here.
{ {
// The length check has to be INSIDE the lock: the decode goroutine // Read through the accessor: the decode goroutine swaps these maps on
// replaces this map wholesale when it grows too large, so reading len() // rotation, so touching them unguarded is a race on the map header,
// unguarded is a race on the map header, not a cheap fast path. // not a cheap fast path.
a.decodeGridsMu.RLock() g := a.lookupDecodeGrid(q.Call)
g := a.decodeGrids[strings.ToUpper(q.Call)]
a.decodeGridsMu.RUnlock()
if g != "" { if g != "" {
out[i].Grid = g out[i].Grid = g
cm := out[i].Mode cm := out[i].Mode
+89
View File
@@ -0,0 +1,89 @@
package main
// Row colouring for the log grid, by QSL / LoTW status — the thing Logger32 does
// and the reason an operator can tell at a glance what still needs sending.
//
// Rules are ORDERED and the first match wins, because a contact is usually
// several things at once: one confirmed on LoTW and by card is confirmed, not
// "sent, awaiting reply". Putting the order in the data rather than in a chain
// of ifs is what lets the settings panel show it in the same order it applies.
import (
"encoding/json"
"regexp"
"strings"
)
// RowColorRule is one status and the colour it paints.
type RowColorRule struct {
ID string `json:"id"`
Color string `json:"color"`
Enabled bool `json:"enabled"`
}
// RowColorSettings is the whole appearance block.
type RowColorSettings struct {
Enabled bool `json:"enabled"`
Rules []RowColorRule `json:"rules"`
}
// The rule ids, in priority order. The frontend matches on these and holds the
// labels, so a translated name never has to travel through the settings.
var rowColorOrder = []string{
"confirmed_lotw", // LoTW confirmation received
"confirmed_paper", // card or eQSL received
"sent_waiting", // sent by some route, nothing back yet
"to_send", // a card is requested / queued and has not gone out
}
// Defaults: green for done, amber for waiting, blue for owed. Deliberately
// muted — they are composited at low opacity over a dark grid, and a saturated
// value there reads as an error state rather than a status.
var rowColorDefaults = map[string]string{
"confirmed_lotw": "#16a34a",
"confirmed_paper": "#0ea5e9",
"sent_waiting": "#f59e0b",
"to_send": "#a855f7",
}
// hexColor guards what reaches the stylesheet. The value is interpolated into a
// CSS color-mix() by the grid, so anything that is not plainly a hex colour is
// refused rather than passed through.
var hexColor = regexp.MustCompile(`^#[0-9a-fA-F]{6}$`)
func normRowColors(s RowColorSettings) RowColorSettings {
byID := map[string]RowColorRule{}
for _, r := range s.Rules {
byID[r.ID] = r
}
out := RowColorSettings{Enabled: s.Enabled}
for _, id := range rowColorOrder {
r := byID[id]
r.ID = id
if !hexColor.MatchString(strings.TrimSpace(r.Color)) {
r.Color = rowColorDefaults[id]
}
out.Rules = append(out.Rules, r)
}
return out
}
// GetRowColors returns the row-colouring configuration, defaults included so the
// panel never has to invent one.
func (a *App) GetRowColors() RowColorSettings {
var s RowColorSettings
if raw := a.settingOr(keyRowColors, ""); raw != "" {
_ = json.Unmarshal([]byte(raw), &s)
}
return normRowColors(s)
}
// SaveRowColors persists it.
func (a *App) SaveRowColors(s RowColorSettings) error {
b, err := json.Marshal(normRowColors(s))
if err != nil {
return err
}
a.setSetting(keyRowColors, string(b))
return nil
}
+51
View File
@@ -0,0 +1,51 @@
package main
import "testing"
// The colour is interpolated into a CSS color-mix() by the grid, so anything
// that is not plainly a hex colour has to be refused rather than passed on.
func TestRowColorsRefuseAnythingButHex(t *testing.T) {
in := RowColorSettings{Enabled: true, Rules: []RowColorRule{
{ID: "confirmed_lotw", Color: "#123abc", Enabled: true},
{ID: "sent_waiting", Color: "red; background:url(x)", Enabled: true},
{ID: "to_send", Color: "", Enabled: true},
}}
got := normRowColors(in)
byID := map[string]RowColorRule{}
for _, r := range got.Rules {
byID[r.ID] = r
}
if byID["confirmed_lotw"].Color != "#123abc" {
t.Errorf("a valid colour was rewritten: %q", byID["confirmed_lotw"].Color)
}
if byID["sent_waiting"].Color != rowColorDefaults["sent_waiting"] {
t.Errorf("an injection attempt survived: %q", byID["sent_waiting"].Color)
}
if byID["to_send"].Color != rowColorDefaults["to_send"] {
t.Errorf("an empty colour was kept: %q", byID["to_send"].Color)
}
}
// Priority lives in the data, not in a chain of ifs: a contact confirmed on
// LoTW AND by card is confirmed, not "sent, awaiting reply". The panel shows
// the rules in the order they apply, so that order must survive a round trip.
func TestRowColorsKeepPriorityOrder(t *testing.T) {
// Saved in a jumbled order, as a hand-edited settings row could be.
got := normRowColors(RowColorSettings{Rules: []RowColorRule{
{ID: "to_send", Color: "#111111"},
{ID: "confirmed_lotw", Color: "#222222"},
}})
if len(got.Rules) != len(rowColorOrder) {
t.Fatalf("got %d rules, want every one present", len(got.Rules))
}
for i, id := range rowColorOrder {
if got.Rules[i].ID != id {
t.Errorf("rule %d is %q, want %q", i, got.Rules[i].ID, id)
}
}
// The saved colours survived the reordering.
if got.Rules[0].Color != "#222222" {
t.Errorf("confirmed_lotw lost its colour: %q", got.Rules[0].Color)
}
}
+41 -6
View File
@@ -21,6 +21,8 @@ import (
"hamlog/internal/applog" "hamlog/internal/applog"
"hamlog/internal/bandopen" "hamlog/internal/bandopen"
"hamlog/internal/cluster" "hamlog/internal/cluster"
"hamlog/internal/geo"
"hamlog/internal/gridcache"
"hamlog/internal/pskr" "hamlog/internal/pskr"
) )
@@ -127,24 +129,53 @@ func (a *App) startBandOpenFeed() {
} }
s := a.GetBandOpenSettings() s := a.GetBandOpenSettings()
a.bandOpen.on.Store(s.Enabled) a.bandOpen.on.Store(s.Enabled)
a.setBandOpenBands(s.Bands)
if !s.Enabled { if !s.Enabled {
// Put out whatever is currently lit. Leaving the badges up would keep // Put out whatever is currently lit. Leaving the badges up would keep
// announcing an opening from a watch that is now off, and they only fade // announcing an opening from a watch that is now off, and they only fade
// on a timer fed by spots this path no longer looks at — so they would // on a timer fed by spots this path no longer looks at — so they would
// hang there until the app restarted. // hang there until the app restarted.
a.clearBandOpenings() a.clearBandOpenings()
}
chaseGrids := a.gridStore != nil
if !s.Enabled && !chaseGrids {
return return
} }
// Every spot is measured from the operator's position. Without one there is // Every spot is measured from the operator's position. Without one there is
// nothing to measure, and a detector fed unmeasurable spots reports nothing // nothing to measure, and a detector fed unmeasurable spots reports nothing
// while looking like it is working. // while looking like it is working.
if !a.opSet { if !a.opSet {
applog.Printf("bandopen: no station grid set — the opening watch needs one to measure a path") applog.Printf("pskr: no station grid set — the feed needs one to measure a path")
return return
} }
// Grid chasing wants every band; the opening watch wants its four. "+" is the
// MQTT single-level wildcard, so one subscription per square covers the lot.
bands := s.Bands
if chaseGrids {
bands = []string{"+"}
}
// Filter at the BROKER on the receiver's square rather than receiving the
// world and discarding it here. Measured on the live feed: the four opening
// bands unfiltered are 83 messages a second, of which roughly one in a
// hundred survived the NearKm test below. One ring of squares — about the
// same 300 km — is 0.2 to 1.2 a second, and the same for every operator,
// where filtering by DXCC ranged from 1.2 (OH) to 72.5 (K).
rxGrids := geo.NeighbourGrids(a.opLat, a.opLon, 1)
var onGrid func(call, grid string)
if chaseGrids {
onGrid = func(call, grid string) { a.rememberDecodeGrid(call, grid, gridcache.SourceMQTT) }
}
var onSpot func(pskr.Spot)
if s.Enabled {
onSpot = a.feedBandOpen
}
a.pskr = pskr.New(pskr.Config{ a.pskr = pskr.New(pskr.Config{
Bands: s.Bands, Bands: bands,
OpLat: a.opLat, OpLon: a.opLon, RxGrids: rxGrids,
OpLat: a.opLat, OpLon: a.opLon,
Geo: func(grid string) (int, int, bool) { Geo: func(grid string) (int, int, bool) {
lat, lon, ok := gridToLatLon(grid) lat, lon, ok := gridToLatLon(grid)
if !ok { if !ok {
@@ -156,12 +187,16 @@ func (a *App) startBandOpenFeed() {
b := int(initialBearingDeg(a.opLat, a.opLon, lat, lon) + 0.5) b := int(initialBearingDeg(a.opLat, a.opLon, lat, lon) + 0.5)
return d, b, true return d, b, true
}, },
OnSpot: a.feedBandOpen, OnSpot: onSpot,
OnGrid: onGrid,
Logf: applog.Printf, Logf: applog.Printf,
}) })
if err := a.pskr.Start(); err != nil { if err := a.pskr.Start(); err != nil {
applog.Printf("bandopen: PSK Reporter feed did not start: %v", err) applog.Printf("pskr: feed did not start: %v", err)
return
} }
applog.Printf("pskr: feed up — bands %v, %d receiver squares (openings=%v, grids=%v)",
bands, len(rxGrids), s.Enabled, chaseGrids)
} }
// feedBandOpen hands one PSK Reporter decode to the detector. // feedBandOpen hands one PSK Reporter decode to the detector.
@@ -170,7 +205,7 @@ func (a *App) startBandOpenFeed() {
// so it does the least possible: the detector's own window and de-duplication // so it does the least possible: the detector's own window and de-duplication
// by callsign are what turn that flood into one announcement. // by callsign are what turn that flood into one announcement.
func (a *App) feedBandOpen(s pskr.Spot) { func (a *App) feedBandOpen(s pskr.Spot) {
if !bandopen.Watched(s.Band) { if !a.bandOpenWanted(s.Band) {
return return
} }
a.bandOpen.mu.Lock() a.bandOpen.mu.Lock()
+45 -3
View File
@@ -28,8 +28,15 @@ type bandOpenState struct {
// settings query per spot is exactly what the rest of this file avoids. // settings query per spot is exactly what the rest of this file avoids.
// startBandOpenFeed owns it — it runs at startup and again on every save, so // startBandOpenFeed owns it — it runs at startup and again on every save, so
// the switch takes effect without a restart. // the switch takes effect without a restart.
on atomic.Bool on atomic.Bool
mu sync.Mutex // bands is the operator's SELECTED set, held as map[string]bool.
//
// bandopen.Watched only says which bands the detector is capable of; it
// knows nothing about the chips in Settings. Nothing checked the selection
// on either feed path, so unticking 10 m and 2 m changed the subscription and
// left the cluster path announcing them anyway.
bands atomic.Value
mu sync.Mutex
det *bandopen.Detector det *bandopen.Detector
last []bandopen.Opening // most recent first, for the UI last []bandopen.Opening // most recent first, for the UI
// live holds the announced openings that are still going, keyed by band, and // live holds the announced openings that are still going, keyed by band, and
@@ -60,7 +67,7 @@ func (a *App) detectBandOpening(s cluster.Spot) {
// itself ran on every ordinary cluster spot. So an operator who had never // itself ran on every ordinary cluster spot. So an operator who had never
// enabled the watch still got opening banners, from a feature they had // enabled the watch still got opening banners, from a feature they had
// deliberately left off. // deliberately left off.
if !a.bandOpen.on.Load() { if !a.bandOpen.on.Load() || !a.bandOpenWanted(s.Band) {
return return
} }
// No operator grid = no distance and no bearing on the spot, and the whole // No operator grid = no distance and no bearing on the spot, and the whole
@@ -140,6 +147,41 @@ func (a *App) GetLiveOpenings() []bandopen.Opening {
return out return out
} }
// bandOpenWanted reports whether the operator has this band ticked. Falls back
// to the detector's own set until a selection has been stored, so a band is
// never silently dropped before the settings have been read.
func (a *App) bandOpenWanted(band string) bool {
if !bandopen.Watched(band) {
return false
}
sel, ok := a.bandOpen.bands.Load().(map[string]bool)
if !ok || len(sel) == 0 {
return true
}
return sel[strings.ToLower(strings.TrimSpace(band))]
}
// setBandOpenBands records the selection and puts out badges for bands that
// have just been unticked — they fade on a timer fed by spots the detector no
// longer looks at, so they would otherwise stay lit until the next restart.
func (a *App) setBandOpenBands(bands []string) {
sel := make(map[string]bool, len(bands))
for _, b := range bands {
sel[strings.ToLower(strings.TrimSpace(b))] = true
}
a.bandOpen.bands.Store(sel)
a.bandOpen.mu.Lock()
defer a.bandOpen.mu.Unlock()
for b := range a.bandOpen.live {
if len(sel) > 0 && !sel[b] {
delete(a.bandOpen.live, b)
delete(a.bandOpen.aliveUntil, b)
applog.Printf("bandopen: %s is no longer watched — badge cleared", strings.ToUpper(b))
}
}
}
// clearBandOpenings puts out every lit badge and forgets the detector's window. // clearBandOpenings puts out every lit badge and forgets the detector's window.
// Called when the watch is switched off: the badges fade on a timer fed by // Called when the watch is switched off: the badges fade on a timer fed by
// spots the detector no longer looks at, so without this they would stay up // spots the detector no longer looks at, so without this they would stay up
+53
View File
@@ -63,3 +63,56 @@ func TestClearBandOpeningsPutsTheBadgesOut(t *testing.T) {
t.Error("the remembered openings were thrown away") t.Error("the remembered openings were thrown away")
} }
} }
// Unticking a band in Settings must actually stop its announcements.
//
// Nothing checked the operator's selection on either feed path: bandopen.Watched
// only says which bands the detector is CAPABLE of. So a station with just 6 m
// ticked still got 10 m and 2 m badges, from the cluster path and — once grid
// chasing widened the subscription to every band — from PSK Reporter too.
func TestBandOpenHonoursTheSelectedBands(t *testing.T) {
a := &App{opSet: true, opLat: 48, opLon: 2}
a.bandOpen.on.Store(true)
a.setBandOpenBands([]string{"6m"})
if !a.bandOpenWanted("6m") {
t.Error("6m is ticked and was refused")
}
for _, b := range []string{"10m", "2m", "4m"} {
if a.bandOpenWanted(b) {
t.Errorf("%s is not ticked and was accepted", b)
}
}
// A band the detector cannot watch at all stays out whatever is stored.
if a.bandOpenWanted("20m") {
t.Error("20m is not a watched band and was accepted")
}
// And the cluster path must obey it too.
a.detectBandOpening(cluster.Spot{
DXCall: "K1ABC", Band: "10m", DistanceKm: 6000, ShortPath: 280, ReceivedAt: time.Now(),
})
if a.bandOpen.det != nil {
t.Error("a spot on an unticked band reached the detector")
}
}
// Unticking a band puts its badge out. They fade on a timer fed by spots the
// detector no longer looks at, so it would otherwise stay lit until a restart.
func TestUntickingABandClearsItsBadge(t *testing.T) {
a := &App{}
a.bandOpen.live = map[string]bandopen.Opening{
"6m": {Band: "6m", Calls: 9},
"10m": {Band: "10m", Calls: 5},
}
a.bandOpen.aliveUntil = map[string]time.Time{
"6m": time.Now().Add(time.Hour),
"10m": time.Now().Add(time.Hour),
}
a.setBandOpenBands([]string{"6m"})
live := a.GetLiveOpenings()
if len(live) != 1 || live[0].Band != "6m" {
t.Errorf("live openings = %v, want only 6m", live)
}
}
+24
View File
@@ -1,4 +1,28 @@
[ [
{
"version": "0.24.8",
"date": "",
"en": [
"Cluster: the grid cache now holds 100,000 callsigns and rotates instead of emptying itself, so locators stop vanishing from the list.",
"New option \"Chase new grids\": locators learnt from your decodes and from PSK Reporter are kept in their own database, with their source.",
"Band openings: the PSK Reporter feed is now filtered at the broker, which cuts it from about 83 messages a second to under two.",
"Band openings: unticking a band now actually stops its announcements, and puts its badge out.",
"Callsign lookup: the website, postal code and HamQTH profile picture are now read — the QSO web column was never filled by any lookup.",
"New selectable column \"QSO number\": position in the log, 1 for the oldest contact.",
"New Appearance settings: colour whole log rows by QSL status, with your own colours from a palette.",
"The QSO number was missing on contacts logged from WSJT-X — that path bypassed the numbering."
],
"fr": [
"Cluster : le cache de locators garde 100 000 indicatifs et tourne au lieu de se vider, les locators ne disparaissent donc plus de la liste.",
"Nouvelle option « Chasse aux nouveaux carrés » : les locators appris de tes décodes et de PSK Reporter sont conservés dans leur propre base, avec leur source.",
"Ouvertures de bande : le flux PSK Reporter est désormais filtré chez le broker, ce qui le fait passer d environ 83 messages par seconde à moins de deux.",
"Ouvertures de bande : décocher une bande arrête réellement ses annonces et éteint son badge.",
"Recherche d indicatif : le site web, le code postal et la photo de profil HamQTH sont désormais lus — la colonne web du QSO n était jamais remplie.",
"Nouvelle colonne sélectionnable « Numéro de QSO » : position dans le log, 1 pour le contact le plus ancien.",
"Nouveaux réglages Apparence : colorer les lignes entières du log selon le statut QSL, avec tes couleurs choisies dans une palette.",
"Le numéro de QSO manquait sur les contacts enregistrés depuis WSJT-X — ce chemin contournait la numérotation."
]
},
{ {
"version": "0.24.7", "version": "0.24.7",
"date": "", "date": "",
+160
View File
@@ -0,0 +1,160 @@
package main
import (
"fmt"
"path/filepath"
"sync"
"testing"
"hamlog/internal/gridcache"
)
// The grid cache used to drop EVERYTHING at its ceiling. That was survivable
// while the only feed was this station's own decodes, which never reached it.
// Fed by anything larger it is a cliff: every locator in the cluster list
// disappears at once, periodically, and the operator sees the column empty
// itself for no reason.
//
// Rotation keeps the previous generation, so a full cache costs the older half
// and nothing more.
func TestDecodeGridRotationKeepsThePreviousGeneration(t *testing.T) {
a := &App{}
// Fill exactly one generation.
for i := 0; i < decodeGridsCap; i++ {
a.rememberDecodeGrid(fmt.Sprintf("CALL%06d", i), "JN36", gridcache.SourceDecode)
}
if got := a.lookupDecodeGrid("CALL000000"); got != "JN36" {
t.Fatalf("first entry lost before any rotation: %q", got)
}
if a.decodeGridsOld != nil {
t.Fatal("rotated early — the cap is the ceiling of ONE generation")
}
// One more entry rotates.
a.rememberDecodeGrid("NEWCALL", "IO91", gridcache.SourceDecode)
if a.decodeGridsOld == nil {
t.Fatal("did not rotate at the cap")
}
if got := a.lookupDecodeGrid("NEWCALL"); got != "IO91" {
t.Errorf("the entry that caused the rotation was lost: %q", got)
}
// The whole previous generation is still readable — this is the point.
if got := a.lookupDecodeGrid("CALL000000"); got != "JN36" {
t.Errorf("a locator from the previous generation was dropped: %q — that is the cliff again", got)
}
if got := a.lookupDecodeGrid("CALL099999"); got != "JN36" {
t.Errorf("previous generation incomplete: %q", got)
}
// A second rotation is what finally retires the oldest half.
for i := 0; i < decodeGridsCap; i++ {
a.rememberDecodeGrid(fmt.Sprintf("SECOND%06d", i), "KP20", gridcache.SourceDecode)
}
if got := a.lookupDecodeGrid("CALL000000"); got != "" {
t.Errorf("the cache is unbounded: %q survived two rotations", got)
}
if got := a.lookupDecodeGrid("NEWCALL"); got != "IO91" {
t.Errorf("an entry one generation old was retired too early: %q", got)
}
}
// Callsigns are normalised on the way in AND on the way out, or a spot for
// "f4bpo" would miss a grid learnt as "F4BPO".
func TestDecodeGridCaseAndBlanks(t *testing.T) {
a := &App{}
a.rememberDecodeGrid(" f4bpo ", "JN36", gridcache.SourceDecode)
if got := a.lookupDecodeGrid("F4BPO"); got != "JN36" {
t.Errorf("lookup of the upper-case form failed: %q", got)
}
a.rememberDecodeGrid("", "JN36", gridcache.SourceDecode)
a.rememberDecodeGrid("K1ABC", "", gridcache.SourceDecode)
if got := a.lookupDecodeGrid("K1ABC"); got != "" {
t.Errorf("stored an empty grid: %q", got)
}
}
// The write path runs on the decode goroutine while the cluster status builder
// reads. Rotation swaps the map headers, so an unguarded read is a data race
// rather than a stale value.
//
// This build is CGO-free, so -race is not available here; the test exercises the
// interleaving and would fault on a concurrent map access even without it.
func TestDecodeGridConcurrentAccess(t *testing.T) {
a := &App{}
var wg sync.WaitGroup
wg.Add(2)
go func() {
defer wg.Done()
for i := 0; i < 20000; i++ {
a.rememberDecodeGrid(fmt.Sprintf("W%05d", i), "FN31", gridcache.SourceDecode)
}
}()
go func() {
defer wg.Done()
for i := 0; i < 20000; i++ {
_ = a.lookupDecodeGrid(fmt.Sprintf("W%05d", i))
}
}()
wg.Wait()
}
// Only a CHANGE may reach the write batch.
//
// The feeds repeat themselves — the same station is reported by dozens of
// receivers a minute — so queueing every report would put the whole stream in
// the batch instead of the news in it, and turn a cache into a write amplifier.
func TestOnlyChangesAreQueued(t *testing.T) {
st, err := gridcache.Open(filepath.Join(t.TempDir(), "grids.db"), nil)
if err != nil {
t.Fatal(err)
}
defer st.Close()
a := &App{gridStore: st}
a.rememberDecodeGrid("F4BPO", "JN36", gridcache.SourceDecode)
if n := st.Pending(); n != 1 {
t.Fatalf("a new locator queued %d writes, want 1", n)
}
if err := st.Flush(); err != nil {
t.Fatal(err)
}
// The same report, a hundred times over, is not news.
for i := 0; i < 100; i++ {
a.rememberDecodeGrid("F4BPO", "JN36", gridcache.SourceDecode)
}
if n := st.Pending(); n != 0 {
t.Errorf("unchanged reports queued %d writes — the batch would carry the whole feed", n)
}
// A station that moved is.
a.rememberDecodeGrid("F4BPO", "KP30", gridcache.SourceDecode)
if n := st.Pending(); n != 1 {
t.Errorf("a changed locator queued %d writes, want 1", n)
}
if got := a.lookupDecodeGrid("F4BPO"); got != "KP30" {
t.Errorf("the map kept the old locator: %q", got)
}
}
// Rotation must be OFF while a store is attached: it would discard callsigns the
// database still holds, and the lookup would then miss something we know.
func TestNoRotationWhilePersisting(t *testing.T) {
st, err := gridcache.Open(filepath.Join(t.TempDir(), "grids.db"), nil)
if err != nil {
t.Fatal(err)
}
defer st.Close()
a := &App{gridStore: st}
for i := 0; i < decodeGridsCap+10; i++ {
a.rememberDecodeGrid(fmt.Sprintf("CALL%06d", i), "JN36", gridcache.SourceDecode)
}
if a.decodeGridsOld != nil {
t.Error("rotated while a store was attached — locators the database holds would go missing")
}
if got := a.lookupDecodeGrid("CALL000000"); got != "JN36" {
t.Errorf("the first entry was dropped: %q", got)
}
}
+10 -1
View File
@@ -91,6 +91,7 @@ import { ShutdownProgress } from '@/components/ShutdownProgress';
import { ClusterGrid } from '@/components/ClusterGrid'; import { ClusterGrid } from '@/components/ClusterGrid';
import { cleanSpotter, inferSpotMode, spotModeCategory, spotStatusKey } from '@/lib/spot'; import { cleanSpotter, inferSpotMode, spotModeCategory, spotStatusKey } from '@/lib/spot';
import { applySpotDisplay, readSpotDisplayOptions, SPOT_DISPLAY_OPTIONS_EXPOSED } from '@/lib/spotDisplay'; import { applySpotDisplay, readSpotDisplayOptions, SPOT_DISPLAY_OPTIONS_EXPOSED } from '@/lib/spotDisplay';
import { GetRowColors } from '../wailsjs/go/main/App';
import { WorkedBeforeGrid } from '@/components/WorkedBeforeGrid'; import { WorkedBeforeGrid } from '@/components/WorkedBeforeGrid';
import { NetControlPanel } from '@/components/NetControlPanel'; import { NetControlPanel } from '@/components/NetControlPanel';
import { ContestPanel, CONTEST_DEFAULT, type ContestSession } from '@/components/ContestPanel'; import { ContestPanel, CONTEST_DEFAULT, type ContestSession } from '@/components/ContestPanel';
@@ -1923,6 +1924,10 @@ export default function App() {
// change). The grid reads them straight from the row — no per-page backend // change). The grid reads them straight from the row — no per-page backend
// recompute — so here we just parse the stored JSON string into the code→ref // recompute — so here we just parse the stored JSON string into the code→ref
// object the award columns expect (keys are already upper-case). // object the award columns expect (keys are already upper-case).
// Row colouring by QSL status (Settings → Appearance). Reloaded when the
// settings dialog closes, which is the only place it changes.
const [rowColors, setRowColors] = useState<any>(null);
useEffect(() => { GetRowColors().then(setRowColors).catch(() => {}); }, [showSettings]);
const qsosWithAwards = useMemo( const qsosWithAwards = useMemo(
() => (qsos as any[]).map((q) => ({ ...q, award_refs: parseAwardRefs(q.award_refs) })), () => (qsos as any[]).map((q) => ({ ...q, award_refs: parseAwardRefs(q.award_refs) })),
[qsos], [qsos],
@@ -3303,7 +3308,7 @@ export default function App() {
sat_name: details.sat_name, sat_mode: details.sat_mode, sat_name: details.sat_name, sat_mode: details.sat_mode,
contest_id: details.contest_id, contest_id: details.contest_id,
srx: srxE.num, stx: stxE.num, srx_string: srxE.str, stx_string: stxE.str, srx: srxE.num, stx: stxE.num, srx_string: srxE.str, stx_string: stxE.str,
email: details.email, email: details.email, web: details.web,
}; };
applyAwardRefs(payload, details.award_refs ?? '', awardFieldRef.current); applyAwardRefs(payload, details.award_refs ?? '', awardFieldRef.current);
// Contest mode overrides the exchange fields: CONTEST_ID, the sent // Contest mode overrides the exchange fields: CONTEST_ID, the sent
@@ -3666,6 +3671,7 @@ export default function App() {
ituz: d.ituz ?? (last.ituz || undefined), ituz: d.ituz ?? (last.ituz || undefined),
cont: d.cont || last.cont || '', cont: d.cont || last.cont || '',
email: d.email || last.email || '', email: d.email || last.email || '',
web: d.web || last.web || '',
qsl_via: d.qsl_via || last.qsl_via || '', qsl_via: d.qsl_via || last.qsl_via || '',
}; };
}); });
@@ -3728,6 +3734,7 @@ export default function App() {
ituz: d.ituz ?? (r.ituz || undefined), ituz: d.ituz ?? (r.ituz || undefined),
cont: d.cont || (r.cont ?? ''), cont: d.cont || (r.cont ?? ''),
email: d.email || (r.email ?? ''), email: d.email || (r.email ?? ''),
web: d.web || (r.web ?? ''),
qsl_via: d.qsl_via || (r.qsl_via ?? ''), qsl_via: d.qsl_via || (r.qsl_via ?? ''),
})); }));
// Backfill anything the provider didn't supply from the last time we worked // Backfill anything the provider didn't supply from the last time we worked
@@ -5068,6 +5075,7 @@ export default function App() {
// rewrote the other's widths. // rewrote the other's widths.
storageKey="mainpane" storageKey="mainpane"
rows={qsosWithAwards as any} rows={qsosWithAwards as any}
rowColors={rowColors}
myGrid={station.my_grid} myGrid={station.my_grid}
total={total} total={total}
awardCols={awardCols} awardCols={awardCols}
@@ -6351,6 +6359,7 @@ export default function App() {
<RecentQSOsGrid <RecentQSOsGrid
key={`rqg2-${activeProfileId ?? 'x'}`} key={`rqg2-${activeProfileId ?? 'x'}`}
rows={qsosWithAwards as any} rows={qsosWithAwards as any}
rowColors={rowColors}
myGrid={station.my_grid} myGrid={station.my_grid}
total={total} total={total}
awardCols={awardCols} awardCols={awardCols}
@@ -0,0 +1,88 @@
import { useEffect, useState } from 'react';
import { GetRowColors, SaveRowColors } from '../../wailsjs/go/main/App';
import { Checkbox } from '@/components/ui/checkbox';
import { useI18n } from '@/lib/i18n';
import { cn } from '@/lib/utils';
import type { RowColorSettings } from '@/lib/rowColors';
// A fixed palette plus a free picker. Muted values on purpose: they are
// composited at low opacity over a dark grid, where a saturated colour reads as
// an error state rather than a status.
const PALETTE = [
'#16a34a', '#0ea5e9', '#f59e0b', '#a855f7',
'#dc2626', '#14b8a6', '#eab308', '#ec4899',
'#64748b', '#84cc16', '#6366f1', '#f97316',
];
// The rule ids the backend orders; the labels live here so a translation never
// travels through the settings row.
const LABELS: Record<string, string> = {
confirmed_lotw: 'appr.confirmedLotw',
confirmed_paper: 'appr.confirmedPaper',
sent_waiting: 'appr.sentWaiting',
to_send: 'appr.toSend',
};
export function AppearancePanel() {
const { t } = useI18n();
const [cfg, setCfg] = useState<RowColorSettings | null>(null);
useEffect(() => {
(async () => {
try { setCfg((await GetRowColors()) as any); } catch { /* defaults on the backend */ }
})();
}, []);
const save = (next: RowColorSettings) => {
setCfg(next);
SaveRowColors(next as any).catch(() => {});
};
const patchRule = (id: string, patch: Partial<{ color: string; enabled: boolean }>) => {
if (!cfg) return;
save({ ...cfg, rules: cfg.rules.map((r) => (r.id === id ? { ...r, ...patch } : r)) });
};
if (!cfg) return <div className="p-1 text-sm text-muted-foreground"></div>;
return (
<div className="space-y-4">
<label className="flex items-start gap-2 text-sm cursor-pointer">
<Checkbox checked={cfg.enabled} className="mt-0.5"
onCheckedChange={(c) => save({ ...cfg, enabled: !!c })} />
<span>{t('appr.enable')} <span className="text-xs text-muted-foreground">{t('appr.enableHint')}</span></span>
</label>
{cfg.enabled && (
<div className="space-y-2">
{/* Order matters and is shown: a contact is usually several of these at
once, and the first match wins. */}
<p className="text-xs text-muted-foreground">{t('appr.orderHint')}</p>
{cfg.rules.map((r, i) => (
<div key={r.id} className="rounded-lg border border-border/60 p-2.5 space-y-2"
style={{ backgroundColor: r.enabled ? `color-mix(in srgb, ${r.color} 24%, transparent)` : undefined }}>
<label className="flex items-center gap-2 text-sm cursor-pointer">
<Checkbox checked={r.enabled} onCheckedChange={(c) => patchRule(r.id, { enabled: !!c })} />
<span className="font-mono text-xs text-muted-foreground">{i + 1}.</span>
<span className="font-medium">{t(LABELS[r.id] ?? r.id)}</span>
</label>
{r.enabled && (
<div className="flex items-center gap-1.5 flex-wrap pl-6">
{PALETTE.map((c) => (
<button key={c} type="button" title={c}
onClick={() => patchRule(r.id, { color: c })}
className={cn('size-6 rounded-md border-2 transition-transform hover:scale-110',
r.color.toLowerCase() === c ? 'border-foreground' : 'border-transparent')}
style={{ backgroundColor: c }} />
))}
<input type="color" value={r.color} title={t('appr.custom')}
onChange={(e) => patchRule(r.id, { color: e.target.value })}
className="size-6 rounded-md border border-border bg-transparent p-0 cursor-pointer" />
</div>
)}
</div>
))}
</div>
)}
</div>
);
}
+9 -2
View File
@@ -42,6 +42,7 @@ export interface DetailsState {
srx_string?: string; srx_string?: string;
stx_string?: string; stx_string?: string;
email: string; email: string;
web?: string;
// Award references for the contacted station (set via the Awards tab picker). // Award references for the contacted station (set via the Awards tab picker).
// Semicolon-delimited "AWARD@REF" entries, e.g. "POTA@FR-11553;IOTA@EU-064". // Semicolon-delimited "AWARD@REF" entries, e.g. "POTA@FR-11553;IOTA@EU-064".
// App.tsx maps these back to pota_ref/sota_ref/iota when saving the QSO. // App.tsx maps these back to pota_ref/sota_ref/iota when saving the QSO.
@@ -498,8 +499,14 @@ export function DetailsPanel({ callsign, prefix, operatorGrid, remoteGrid, qth,
<Field label="STX"> <Field label="STX">
<Input value={details.stx_string ?? ''} placeholder={t('detp.sentExchangePh')} onChange={(e) => onChange({ stx_string: e.target.value })} /> <Input value={details.stx_string ?? ''} placeholder={t('detp.sentExchangePh')} onChange={(e) => onChange({ stx_string: e.target.value })} />
</Field> </Field>
<Field label={t('detp.contactedEmail')} span={3}> {/* No placeholder: a greyed-out sample address in an empty field reads
<Input value={details.email} placeholder="[email protected]" onChange={(e) => onChange({ email: e.target.value })} /> as a value the lookup found, and that is exactly the field an
operator checks to see whether it found one. */}
<Field label={t('detp.contactedEmail')} span={2}>
<Input value={details.email} onChange={(e) => onChange({ email: e.target.value })} />
</Field>
<Field label={t('detp.contactedWeb')}>
<Input value={details.web ?? ''} onChange={(e) => onChange({ web: e.target.value })} />
</Field> </Field>
</div> </div>
)} )}
+9 -1
View File
@@ -18,6 +18,7 @@ import { Checkbox } from '@/components/ui/checkbox';
import { loadLocal, loadRemote, saveState, seedLocal } from '@/lib/gridPrefs'; import { loadLocal, loadRemote, saveState, seedLocal } from '@/lib/gridPrefs';
import { useI18n } from '@/lib/i18n'; import { useI18n } from '@/lib/i18n';
import { gridToLatLon, pathBetweenLatLon } from '@/lib/maidenhead'; import { gridToLatLon, pathBetweenLatLon } from '@/lib/maidenhead';
import { rowStyleFor, type RowColorSettings } from '@/lib/rowColors';
// Register every Community feature once. v32+ requires explicit registration; // Register every Community feature once. v32+ requires explicit registration;
// AllCommunityModule keeps it simple and pulls in sort/filter/resize/reorder/ // AllCommunityModule keeps it simple and pulls in sort/filter/resize/reorder/
@@ -80,6 +81,8 @@ type Props = {
// One column per defined award; the cell shows the reference this QSO counts // One column per defined award; the cell shows the reference this QSO counts
// for (from row.award_refs[CODE], attached by the parent). Hidden by default. // for (from row.award_refs[CODE], attached by the parent). Hidden by default.
awardCols?: { code: string; name: string }[]; awardCols?: { code: string; name: string }[];
// Whole-row colouring by QSL / LoTW status (Settings → Appearance).
rowColors?: RowColorSettings | null;
}; };
const BASE_COLSTATE_KEY = 'hamlog.qsoColState.v2'; const BASE_COLSTATE_KEY = 'hamlog.qsoColState.v2';
@@ -132,6 +135,10 @@ function qsoDistanceKm(d: any, myGrid?: string): number | undefined {
export const makeColCatalog = (t: TFn, myGrid?: string): ColEntry[] => [ export const makeColCatalog = (t: TFn, myGrid?: string): ColEntry[] => [
// ── QSO basics ── // ── QSO basics ──
// Position in the log, oldest = 1. Filled by the backend over the WHOLE log,
// so it does not change when a filter narrows what is shown.
{ group: 'QSO', label: t('rqg.c.number'), colId: 'number', headerName: t('rqg.h.number'), field: 'number' as any, width: 80, type: 'rightAligned', cellClass: 'font-mono',
comparator: (a, b) => (a ?? 0) - (b ?? 0) },
{ group: 'QSO', label: t('rqg.c.qso_date'), colId: 'qso_date', headerName: t('rqg.c.qso_date'), field: 'qso_date' as any, width: 150, cellClass: 'font-mono', valueFormatter: (p) => fmtDateUTC(p.value), sort: 'desc', defaultVisible: true }, { group: 'QSO', label: t('rqg.c.qso_date'), colId: 'qso_date', headerName: t('rqg.c.qso_date'), field: 'qso_date' as any, width: 150, cellClass: 'font-mono', valueFormatter: (p) => fmtDateUTC(p.value), sort: 'desc', defaultVisible: true },
{ group: 'QSO', label: t('rqg.c.qso_date_off'), colId: 'qso_date_off', headerName: t('rqg.c.qso_date_off'), field: 'qso_date_off' as any, width: 150, cellClass: 'font-mono', valueFormatter: (p) => fmtDateUTC(p.value) }, { group: 'QSO', label: t('rqg.c.qso_date_off'), colId: 'qso_date_off', headerName: t('rqg.c.qso_date_off'), field: 'qso_date_off' as any, width: 150, cellClass: 'font-mono', valueFormatter: (p) => fmtDateUTC(p.value) },
{ group: 'QSO', label: t('rqg.c.callsign'), colId: 'callsign', headerName: t('rqg.c.callsign'), field: 'callsign' as any, width: 110, cellClass: 'font-mono font-semibold', defaultVisible: true }, { group: 'QSO', label: t('rqg.c.callsign'), colId: 'callsign', headerName: t('rqg.c.callsign'), field: 'callsign' as any, width: 110, cellClass: 'font-mono font-semibold', defaultVisible: true },
@@ -299,7 +306,7 @@ const sanitizeAwardCols = (st: any[] | null | undefined): any[] =>
return rest; return rest;
}); });
export function RecentQSOsGrid({ rows, myGrid, selectAllSignal, selectRowSignal, rowDragCall, passOrder, onGridApi, storageKey, onRowDoubleClicked, onRowClicked, onRowSelected, onRowSelectedQso, onUpdateFromCty, onUpdateFromQRZ, onUpdateFromClublog, onSendTo, onSendRecording, onSendEQSL, onBulkEdit, onExportSelected, onExportSelectedFields, onExportFiltered, onExportCabrilloSelected, onExportCabrilloFiltered, onDelete, onFilteredCountChange, awardCols }: Props) { export function RecentQSOsGrid({ rows, myGrid, selectAllSignal, selectRowSignal, rowDragCall, passOrder, onGridApi, storageKey, onRowDoubleClicked, onRowClicked, onRowSelected, onRowSelectedQso, onUpdateFromCty, onUpdateFromQRZ, onUpdateFromClublog, onSendTo, onSendRecording, onSendEQSL, onBulkEdit, onExportSelected, onExportSelectedFields, onExportFiltered, onExportCabrilloSelected, onExportCabrilloFiltered, onDelete, onFilteredCountChange, awardCols, rowColors }: Props) {
const { t } = useI18n(); const { t } = useI18n();
const gridRef = useRef<any>(null); const gridRef = useRef<any>(null);
const [pickerOpen, setPickerOpen] = useState(false); const [pickerOpen, setPickerOpen] = useState(false);
@@ -696,6 +703,7 @@ export function RecentQSOsGrid({ rows, myGrid, selectAllSignal, selectRowSignal,
animateRows={false} animateRows={false}
suppressCellFocus suppressCellFocus
getRowId={(p) => String((p.data as any).id)} getRowId={(p) => String((p.data as any).id)}
getRowStyle={(p) => rowStyleFor(p.data, rowColors ?? null)}
/> />
</div> </div>
</div> </div>
+27 -2
View File
@@ -52,7 +52,7 @@ import {
GetADIFMonitor, SaveADIFMonitor, PickADIFMonitorFile, GetADIFMonitor, SaveADIFMonitor, PickADIFMonitorFile,
GetRelayAuto, SaveRelayAuto, GetStationDevices, GetRelayAuto, SaveRelayAuto, GetStationDevices,
GetAwardDefs, GetTrackedAwards, SaveTrackedAwards, GetAwardDefs, GetTrackedAwards, SaveTrackedAwards,
GetBandOpenSettings, SaveBandOpenSettings, GetPSKReporterStatus, GetBandOpenSettings, SaveBandOpenSettings, GetPSKReporterStatus, GetChaseNewGrids, SetChaseNewGrids, GetGridCacheStatus,
} from '../../wailsjs/go/main/App'; } from '../../wailsjs/go/main/App';
import type { profile as profileModels } from '../../wailsjs/go/models'; import type { profile as profileModels } from '../../wailsjs/go/models';
import type { LookupSettingsForm, StationSettingsForm, ListsSettingsForm, ModePresetForm } from '@/types'; import type { LookupSettingsForm, StationSettingsForm, ListsSettingsForm, ModePresetForm } from '@/types';
@@ -77,6 +77,7 @@ import { getDateFormat, setDateFormat, type DateFormat } from '@/lib/dateFormat'
import { useI18n, FlagGB, FlagFR, type Lang } from '@/lib/i18n'; import { useI18n, FlagGB, FlagFR, type Lang } from '@/lib/i18n';
import { useTheme, CONCRETE_THEMES, type ThemeChoice } from '@/lib/theme'; import { useTheme, CONCRETE_THEMES, type ThemeChoice } from '@/lib/theme';
import { OperatingPanel } from '@/components/OperatingPanel'; import { OperatingPanel } from '@/components/OperatingPanel';
import { AppearancePanel } from '@/components/AppearancePanel';
import { UDPIntegrationsPanel } from '@/components/UDPIntegrationsPanel'; import { UDPIntegrationsPanel } from '@/components/UDPIntegrationsPanel';
type LookupSettings = LookupSettingsForm; type LookupSettings = LookupSettingsForm;
@@ -176,6 +177,7 @@ interface Props {
`disabled: true` greys them out and shows the "coming soon" placeholder. */ `disabled: true` greys them out and shows the "coming soon" placeholder. */
type SectionId = type SectionId =
| 'general' | 'general'
| 'appearance'
| 'email' | 'email'
| 'station' | 'station'
| 'profiles' | 'profiles'
@@ -266,6 +268,7 @@ function buildTree(flexAvailable: boolean, t: (k: string) => string): TreeNode[]
{ {
kind: 'group', label: t('nav.software'), icon: Cog, defaultOpen: true, children: [ kind: 'group', label: t('nav.software'), icon: Cog, defaultOpen: true, children: [
{ kind: 'item', label: t('sec.general'), id: 'general' }, { kind: 'item', label: t('sec.general'), id: 'general' },
{ kind: 'item', label: t('sec.appearance'), id: 'appearance' },
{ kind: 'item', label: t('sec.email'), id: 'email' }, { kind: 'item', label: t('sec.email'), id: 'email' },
{ kind: 'item', label: t('sec.lookup'), id: 'lookup' }, { kind: 'item', label: t('sec.lookup'), id: 'lookup' },
{ kind: 'group', label: t('nav.lists'), icon: Database, defaultOpen: true, children: [ { kind: 'group', label: t('nav.lists'), icon: Database, defaultOpen: true, children: [
@@ -290,7 +293,7 @@ function buildTree(flexAvailable: boolean, t: (k: string) => string): TreeNode[]
// Map section id → i18n key (breadcrumb / placeholders). // Map section id → i18n key (breadcrumb / placeholders).
const SECTION_KEY: Partial<Record<SectionId, string>> = { const SECTION_KEY: Partial<Record<SectionId, string>> = {
station: 'sec.station', profiles: 'sec.profiles', operating: 'sec.operating', confirmations: 'sec.confirmations', station: 'sec.station', profiles: 'sec.profiles', operating: 'sec.operating', confirmations: 'sec.confirmations',
'external-services': 'sec.external', lookup: 'sec.lookup', 'lists-bands': 'sec.bands', 'lists-modes': 'sec.modes', 'external-services': 'sec.external', appearance: 'sec.appearance', lookup: 'sec.lookup', 'lists-bands': 'sec.bands', 'lists-modes': 'sec.modes',
cluster: 'sec.cluster', backup: 'sec.backup', database: 'sec.database', autostart: 'sec.autostart', udp: 'sec.udp', cluster: 'sec.cluster', backup: 'sec.backup', database: 'sec.database', autostart: 'sec.autostart', udp: 'sec.udp',
adifmon: 'sec.adifmon', adifmon: 'sec.adifmon',
webpublish: 'sec.webpublish', webpublish: 'sec.webpublish',
@@ -1552,6 +1555,8 @@ export function SettingsModal({ onClose, onSaved, initialSection, onMainPaneChan
// has side effects there — adding the RBN nodes, bringing the PSK Reporter // has side effects there — adding the RBN nodes, bringing the PSK Reporter
// feed up or down — so the write has to go where those live. // feed up or down — so the write has to go where those live.
const [bandOpen, setBandOpen] = useState<any>({ enabled: false, bands: [], available: [] }); const [bandOpen, setBandOpen] = useState<any>({ enabled: false, bands: [], available: [] });
const [chaseGrids, setChaseGrids] = useState(false);
const [gridStat, setGridStat] = useState<any>(null);
const [pskrStatus, setPskrStatus] = useState<any>(null); const [pskrStatus, setPskrStatus] = useState<any>(null);
const saveBandOpen = async (next: any) => { const saveBandOpen = async (next: any) => {
setBandOpen(next); setBandOpen(next);
@@ -1560,11 +1565,13 @@ export function SettingsModal({ onClose, onSaved, initialSection, onMainPaneChan
useEffect(() => { useEffect(() => {
(async () => { (async () => {
try { setBandOpen(await GetBandOpenSettings()); } catch { /* defaults stand */ } try { setBandOpen(await GetBandOpenSettings()); } catch { /* defaults stand */ }
try { setChaseGrids(await GetChaseNewGrids()); } catch { /* defaults stand */ }
})(); })();
// Poll the feed while the panel is open: a live count is the only thing that // Poll the feed while the panel is open: a live count is the only thing that
// distinguishes "connected" from "connected and receiving nothing". // distinguishes "connected" from "connected and receiving nothing".
const t = window.setInterval(async () => { const t = window.setInterval(async () => {
try { setPskrStatus(await GetPSKReporterStatus()); } catch { /* ignore */ } try { setPskrStatus(await GetPSKReporterStatus()); } catch { /* ignore */ }
try { setGridStat(await GetGridCacheStatus()); } catch { /* ignore */ }
}, 3000); }, 3000);
return () => window.clearInterval(t); return () => window.clearInterval(t);
}, []); }, []);
@@ -4227,6 +4234,23 @@ export function SettingsModal({ onClose, onSaved, initialSection, onMainPaneChan
things set up once. A preferences dialog you reopen every ten minutes things set up once. A preferences dialog you reopen every ten minutes
is a filter in the wrong place. */} is a filter in the wrong place. */}
{/* Grid chasing. Here rather than in the filter panel because it is set
up once: it decides whether locators learnt from decodes are KEPT
across restarts, not what the list shows right now. */}
<div className="border-t border-border/60 pt-3 space-y-2">
<label className="flex items-start gap-2 text-sm cursor-pointer">
<Checkbox checked={chaseGrids} className="mt-0.5"
onCheckedChange={(c) => { setChaseGrids(!!c); SetChaseNewGrids(!!c).catch(() => {}); }} />
<span>{t('clu.chaseGrids')} <span className="text-xs text-muted-foreground">{t('clu.chaseGridsHint')}</span></span>
</label>
{chaseGrids && (
<p className="pl-6 text-xs text-muted-foreground">
{t('clu.chaseGridsStat', { n: gridStat?.known ?? 0, p: gridStat?.pending ?? 0 })}
{pskrStatus?.running ? ` · ${t('bo.feedUp', { n: pskrStatus.received ?? 0 })}` : ''}
</p>
)}
</div>
{/* Band-opening watch. It lives HERE, with the cluster nodes, because {/* Band-opening watch. It lives HERE, with the cluster nodes, because
switching it on adds two of them the operator should see that switching it on adds two of them the operator should see that
happen where it happens rather than find nodes they did not add. */} happen where it happens rather than find nodes they did not add. */}
@@ -6142,6 +6166,7 @@ export function SettingsModal({ onClose, onSaved, initialSection, onMainPaneChan
// Map sections to their content + icon (for placeholder). // Map sections to their content + icon (for placeholder).
const PANELS: Record<SectionId, () => JSX.Element> = { const PANELS: Record<SectionId, () => JSX.Element> = {
general: GeneralPanel, general: GeneralPanel,
appearance: () => <AppearancePanel />,
email: EmailPanel, email: EmailPanel,
station: StationPanel, station: StationPanel,
profiles: ProfilesPanel, profiles: ProfilesPanel,
File diff suppressed because one or more lines are too long
+41
View File
@@ -0,0 +1,41 @@
// Row colouring for the log grid, by QSL / LoTW status.
//
// The rules are ORDERED and the first match wins: a contact is usually several
// things at once, and one confirmed on LoTW and by card is confirmed, not
// "sent, awaiting reply".
export type RowColorRule = { id: string; color: string; enabled: boolean };
export type RowColorSettings = { enabled: boolean; rules: RowColorRule[] };
// ADIF QSL fields are single letters. Y is the only one that means "yes";
// R (requested) and Q (queued) mean a card is owed, which is a different state
// and the one an operator is looking for when deciding what to post.
const yes = (v: any) => String(v ?? '').trim().toUpperCase() === 'Y';
const owed = (v: any) => {
const s = String(v ?? '').trim().toUpperCase();
return s === 'R' || s === 'Q';
};
export function matchRowRule(q: any): string | null {
if (!q) return null;
if (yes(q.lotw_rcvd)) return 'confirmed_lotw';
if (yes(q.qsl_rcvd) || yes(q.eqsl_rcvd)) return 'confirmed_paper';
if (yes(q.qsl_sent) || yes(q.lotw_sent) || yes(q.eqsl_sent)) return 'sent_waiting';
// Any route still queued, not just the paper card. LoTW marks a pending upload
// as R, which is the commonest "not gone out yet" state in a digital log and
// was matching nothing at all while this only looked at qsl_sent.
if (owed(q.qsl_sent) || owed(q.lotw_sent) || owed(q.eqsl_sent)) return 'to_send';
return null;
}
// The colour is applied as a TINT, not a fill. The grid is dark and a solid
// user-picked colour behind white text is unreadable at exactly the moment it
// matters — Logger32 gets away with it because its grid is white.
export function rowStyleFor(q: any, cfg: RowColorSettings | null): { backgroundColor: string } | undefined {
if (!cfg?.enabled) return undefined;
const id = matchRowRule(q);
if (!id) return undefined;
const rule = cfg.rules?.find((r) => r.id === id);
if (!rule?.enabled || !rule.color) return undefined;
return { backgroundColor: `color-mix(in srgb, ${rule.color} 24%, transparent)` };
}
+1 -1
View File
@@ -1,6 +1,6 @@
// Single source of truth for the app version shown in the UI (header + About). // Single source of truth for the app version shown in the UI (header + About).
// Bump this on a release (the release script updates it alongside telemetry.go). // Bump this on a release (the release script updates it alongside telemetry.go).
export const APP_VERSION = '0.24.7'; export const APP_VERSION = '0.24.8';
// Author / credits, shown in Help -> About. // Author / credits, shown in Help -> About.
export const APP_AUTHOR = 'F4BPO'; export const APP_AUTHOR = 'F4BPO';
+10
View File
@@ -404,6 +404,8 @@ export function GetCatalogCodes():Promise<Array<string>>;
export function GetChangelog():Promise<Array<main.ChangelogEntry>>; export function GetChangelog():Promise<Array<main.ChangelogEntry>>;
export function GetChaseNewGrids():Promise<boolean>;
export function GetChatHistory(arg1:number):Promise<Array<main.ChatMessage>>; export function GetChatHistory(arg1:number):Promise<Array<main.ChatMessage>>;
export function GetClublogCtyInfo():Promise<main.ClublogCtyInfo>; export function GetClublogCtyInfo():Promise<main.ClublogCtyInfo>;
@@ -440,6 +442,8 @@ export function GetFlexBandPower():Promise<Record<string, main.FlexBandPower>>;
export function GetFlexState():Promise<cat.FlexTXState>; export function GetFlexState():Promise<cat.FlexTXState>;
export function GetGridCacheStatus():Promise<main.GridCacheStatus>;
export function GetIcomState():Promise<cat.IcomTXState>; export function GetIcomState():Promise<cat.IcomTXState>;
export function GetListsSettings():Promise<main.ListsSettings>; export function GetListsSettings():Promise<main.ListsSettings>;
@@ -488,6 +492,8 @@ export function GetRotatorHeading():Promise<main.RotatorHeading>;
export function GetRotators():Promise<Array<main.RotatorDevice>>; export function GetRotators():Promise<Array<main.RotatorDevice>>;
export function GetRowColors():Promise<main.RowColorSettings>;
export function GetSPEStatus():Promise<spe.Status>; export function GetSPEStatus():Promise<spe.Status>;
export function GetScpStatus():Promise<main.ScpStatus>; export function GetScpStatus():Promise<main.ScpStatus>;
@@ -924,6 +930,8 @@ export function SaveRelayAuto(arg1:main.RelayAutoConfig):Promise<void>;
export function SaveRotators(arg1:Array<main.RotatorDevice>):Promise<void>; export function SaveRotators(arg1:Array<main.RotatorDevice>):Promise<void>;
export function SaveRowColors(arg1:main.RowColorSettings):Promise<void>;
export function SaveSelfSpotSettings(arg1:main.SelfSpotSettings):Promise<void>; export function SaveSelfSpotSettings(arg1:main.SelfSpotSettings):Promise<void>;
export function SaveStationDevices(arg1:Array<main.StationDevice>):Promise<void>; export function SaveStationDevices(arg1:Array<main.StationDevice>):Promise<void>;
@@ -970,6 +978,8 @@ export function SetCIVTrace(arg1:boolean):Promise<void>;
export function SetCWDecoderPitch(arg1:number):Promise<void>; export function SetCWDecoderPitch(arg1:number):Promise<void>;
export function SetChaseNewGrids(arg1:boolean):Promise<void>;
export function SetClublogCtyEnabled(arg1:boolean):Promise<void>; export function SetClublogCtyEnabled(arg1:boolean):Promise<void>;
export function SetClublogMostWantedEnabled(arg1:boolean):Promise<void>; export function SetClublogMostWantedEnabled(arg1:boolean):Promise<void>;
+20
View File
@@ -750,6 +750,10 @@ export function GetChangelog() {
return window['go']['main']['App']['GetChangelog'](); return window['go']['main']['App']['GetChangelog']();
} }
export function GetChaseNewGrids() {
return window['go']['main']['App']['GetChaseNewGrids']();
}
export function GetChatHistory(arg1) { export function GetChatHistory(arg1) {
return window['go']['main']['App']['GetChatHistory'](arg1); return window['go']['main']['App']['GetChatHistory'](arg1);
} }
@@ -822,6 +826,10 @@ export function GetFlexState() {
return window['go']['main']['App']['GetFlexState'](); return window['go']['main']['App']['GetFlexState']();
} }
export function GetGridCacheStatus() {
return window['go']['main']['App']['GetGridCacheStatus']();
}
export function GetIcomState() { export function GetIcomState() {
return window['go']['main']['App']['GetIcomState'](); return window['go']['main']['App']['GetIcomState']();
} }
@@ -918,6 +926,10 @@ export function GetRotators() {
return window['go']['main']['App']['GetRotators'](); return window['go']['main']['App']['GetRotators']();
} }
export function GetRowColors() {
return window['go']['main']['App']['GetRowColors']();
}
export function GetSPEStatus() { export function GetSPEStatus() {
return window['go']['main']['App']['GetSPEStatus'](); return window['go']['main']['App']['GetSPEStatus']();
} }
@@ -1790,6 +1802,10 @@ export function SaveRotators(arg1) {
return window['go']['main']['App']['SaveRotators'](arg1); return window['go']['main']['App']['SaveRotators'](arg1);
} }
export function SaveRowColors(arg1) {
return window['go']['main']['App']['SaveRowColors'](arg1);
}
export function SaveSelfSpotSettings(arg1) { export function SaveSelfSpotSettings(arg1) {
return window['go']['main']['App']['SaveSelfSpotSettings'](arg1); return window['go']['main']['App']['SaveSelfSpotSettings'](arg1);
} }
@@ -1882,6 +1898,10 @@ export function SetCWDecoderPitch(arg1) {
return window['go']['main']['App']['SetCWDecoderPitch'](arg1); return window['go']['main']['App']['SetCWDecoderPitch'](arg1);
} }
export function SetChaseNewGrids(arg1) {
return window['go']['main']['App']['SetChaseNewGrids'](arg1);
}
export function SetClublogCtyEnabled(arg1) { export function SetClublogCtyEnabled(arg1) {
return window['go']['main']['App']['SetClublogCtyEnabled'](arg1); return window['go']['main']['App']['SetClublogCtyEnabled'](arg1);
} }
+70
View File
@@ -1441,6 +1441,8 @@ export namespace lookup {
cont?: string; cont?: string;
email?: string; email?: string;
qsl_via?: string; qsl_via?: string;
web?: string;
zip?: string;
image_url?: string; image_url?: string;
source: string; source: string;
// Go type: time // Go type: time
@@ -1468,6 +1470,8 @@ export namespace lookup {
this.cont = source["cont"]; this.cont = source["cont"];
this.email = source["email"]; this.email = source["email"];
this.qsl_via = source["qsl_via"]; this.qsl_via = source["qsl_via"];
this.web = source["web"];
this.zip = source["zip"];
this.image_url = source["image_url"]; this.image_url = source["image_url"];
this.source = source["source"]; this.source = source["source"];
this.fetched_at = this.convertValues(source["fetched_at"], null); this.fetched_at = this.convertValues(source["fetched_at"], null);
@@ -2419,6 +2423,22 @@ export namespace main {
this.body = source["body"]; this.body = source["body"];
} }
} }
export class GridCacheStatus {
enabled: boolean;
known: number;
pending: number;
static createFrom(source: any = {}) {
return new GridCacheStatus(source);
}
constructor(source: any = {}) {
if ('string' === typeof source) source = JSON.parse(source);
this.enabled = source["enabled"];
this.known = source["known"];
this.pending = source["pending"];
}
}
export class ModePreset { export class ModePreset {
name: string; name: string;
default_rst_sent?: string; default_rst_sent?: string;
@@ -2972,6 +2992,54 @@ export namespace main {
this.motorized = source["motorized"]; this.motorized = source["motorized"];
} }
} }
export class RowColorRule {
id: string;
color: string;
enabled: boolean;
static createFrom(source: any = {}) {
return new RowColorRule(source);
}
constructor(source: any = {}) {
if ('string' === typeof source) source = JSON.parse(source);
this.id = source["id"];
this.color = source["color"];
this.enabled = source["enabled"];
}
}
export class RowColorSettings {
enabled: boolean;
rules: RowColorRule[];
static createFrom(source: any = {}) {
return new RowColorSettings(source);
}
constructor(source: any = {}) {
if ('string' === typeof source) source = JSON.parse(source);
this.enabled = source["enabled"];
this.rules = this.convertValues(source["rules"], RowColorRule);
}
convertValues(a: any, classs: any, asMap: boolean = false): any {
if (!a) {
return a;
}
if (a.slice && a.map) {
return (a as any[]).map(elem => this.convertValues(elem, classs));
} else if ("object" === typeof a) {
if (asMap) {
for (const key of Object.keys(a)) {
a[key] = new classs(a[key]);
}
return a;
}
return new classs(a);
}
return a;
}
}
export class ScpStatus { export class ScpStatus {
enabled: boolean; enabled: boolean;
count: number; count: number;
@@ -4186,6 +4254,7 @@ export namespace qso {
} }
export class QSO { export class QSO {
id: number; id: number;
number?: number;
callsign: string; callsign: string;
// Go type: time // Go type: time
qso_date: any; qso_date: any;
@@ -4326,6 +4395,7 @@ export namespace qso {
constructor(source: any = {}) { constructor(source: any = {}) {
if ('string' === typeof source) source = JSON.parse(source); if ('string' === typeof source) source = JSON.parse(source);
this.id = source["id"]; this.id = source["id"];
this.number = source["number"];
this.callsign = source["callsign"]; this.callsign = source["callsign"];
this.qso_date = this.convertValues(source["qso_date"], null); this.qso_date = this.convertValues(source["qso_date"], null);
this.qso_date_off = this.convertValues(source["qso_date_off"], null); this.qso_date_off = this.convertValues(source["qso_date_off"], null);
@@ -0,0 +1,7 @@
-- The lookup cache gains the two fields the providers were already sending and
-- nothing was reading: the operator's own site, and the postal code.
--
-- `web` matters beyond the cache: the qso table has had a `web` column since
-- 0003 and no lookup ever filled it.
ALTER TABLE callsign_cache ADD COLUMN web TEXT;
ALTER TABLE callsign_cache ADD COLUMN zip TEXT;
+51
View File
@@ -67,3 +67,54 @@ func DistanceBetweenGrids(a, b string) (km float64, ok bool) {
} }
return HaversineKm(lat1, lon1, lat2, lon2), true return HaversineKm(lat1, lon1, lat2, lon2), true
} }
// LatLonToGrid returns the 4-character Maidenhead square for a position.
func LatLonToGrid(lat, lon float64) string {
lon = math.Mod(lon+180, 360)
if lon < 0 {
lon += 360
}
lat = lat + 90
if lat < 0 {
lat = 0
} else if lat > 180 {
lat = 180
}
return string([]byte{
byte('A' + int(lon/20)),
byte('A' + int(lat/10)),
byte('0' + int(math.Mod(lon, 20)/2)),
byte('0' + int(math.Mod(lat, 10)/1)),
})
}
// NeighbourGrids returns the square holding (lat, lon) and the ring of squares
// around it — 9 squares for ring 1, 25 for ring 2.
//
// Used to filter the PSK Reporter feed at the BROKER rather than in OpsLog. A
// square is about 111 km tall and 150 km wide at mid latitudes, so one ring is
// roughly the 300 km "around here" the feed already meant, and the traffic that
// used to be received and discarded is never sent.
func NeighbourGrids(lat, lon float64, ring int) []string {
if ring < 0 {
ring = 0
}
seen := map[string]bool{}
out := []string{}
for dLat := -ring; dLat <= ring; dLat++ {
for dLon := -ring; dLon <= ring; dLon++ {
// One square step: 1° of latitude, 2° of longitude.
la := lat + float64(dLat)
lo := lon + float64(dLon)*2
if la > 90 || la < -90 {
continue // past a pole there is no square, not a wrapped one
}
g := LatLonToGrid(la, lo)
if !seen[g] {
seen[g] = true
out = append(out, g)
}
}
}
return out
}
+49
View File
@@ -0,0 +1,49 @@
package geo
import "testing"
// The squares the PSK Reporter feed is filtered on. A wrong ring means either
// receiving the world again or hearing nothing.
func TestNeighbourGrids(t *testing.T) {
// JN36 is around 46.5N 5.5E.
lat, lon, ok := GridToLatLon("JN36")
if !ok {
t.Fatal("JN36 did not resolve")
}
if got := LatLonToGrid(lat, lon); got != "JN36" {
t.Fatalf("round trip gave %q, want JN36", got)
}
ring := NeighbourGrids(lat, lon, 1)
if len(ring) != 9 {
t.Errorf("ring 1 has %d squares, want 9: %v", len(ring), ring)
}
found := false
for _, g := range ring {
if g == "JN36" {
found = true
}
if len(g) != 4 {
t.Errorf("not a 4-character square: %q", g)
}
}
if !found {
t.Errorf("the operator's own square is missing from %v", ring)
}
if n := len(NeighbourGrids(lat, lon, 0)); n != 1 {
t.Errorf("ring 0 has %d squares, want just the operator's", n)
}
if n := len(NeighbourGrids(lat, lon, 2)); n != 25 {
t.Errorf("ring 2 has %d squares, want 25", n)
}
}
// Near a pole a step north has nowhere to go; it must be dropped, not wrapped
// onto a square on the far side of the world.
func TestNeighbourGridsNearThePole(t *testing.T) {
for _, g := range NeighbourGrids(89.5, 25, 1) {
if len(g) != 4 {
t.Errorf("bad square near the pole: %q", g)
}
}
}
+226
View File
@@ -0,0 +1,226 @@
// Package gridcache is the long-term callsign→grid store behind grid chasing.
//
// Its own SQLite file, not a table in the settings database: that one sits
// wherever the operator put it, often a synchronised folder, and this rewrites
// itself every minute. Deleting the file costs a few days of listening.
//
// A callsign has one grid and the newest report wins — a stale locator reads as
// a square already worked, which is worse than none.
package gridcache
import (
"context"
"database/sql"
"fmt"
"strings"
"sync"
"time"
_ "modernc.org/sqlite"
)
// Retention bounds a store that would otherwise only ever grow. Two years is
// chosen to be far longer than any propagation interest and short enough that a
// reassigned callsign eventually stops carrying its previous holder's square —
// the one way this cache can be actively wrong rather than merely empty.
const Retention = 2 * 365 * 24 * time.Hour
// FlushEvery is the batch interval. Long enough that a burst of reports costs
// one transaction, short enough that a crash loses a minute of learning.
const FlushEvery = 60 * time.Second
// Sources a locator can come from.
const (
SourceDecode = "decode" // a CQ this station's own receiver decoded
SourceMQTT = "mqtt" // a PSK Reporter report
)
// Entry is one locator and where it came from.
type Entry struct {
Grid string
Source string
}
type Store struct {
db *sql.DB
mu sync.Mutex
dirty map[string]Entry // call → what to write
stop chan struct{}
stopOnce sync.Once
wg sync.WaitGroup
logf func(string, ...any)
}
// Open creates or opens the store at path and prunes what has aged out.
func Open(path string, logf func(string, ...any)) (*Store, error) {
if logf == nil {
logf = func(string, ...any) {}
}
// WAL so a flush never blocks a read, and a busy timeout because the flush
// goroutine and the startup load can overlap on a slow disk.
db, err := sql.Open("sqlite", path+"?_pragma=journal_mode(WAL)&_pragma=busy_timeout(5000)")
if err != nil {
return nil, fmt.Errorf("gridcache: open %s: %w", path, err)
}
if _, err := db.Exec("CREATE TABLE IF NOT EXISTS grids (" +
"call TEXT PRIMARY KEY, grid TEXT NOT NULL, updated_at INTEGER NOT NULL, " +
"source TEXT NOT NULL DEFAULT '')"); err != nil {
db.Close()
return nil, fmt.Errorf("gridcache: schema: %w", err)
}
// A file written before the column existed keeps its rows; this fails
// harmlessly when the column is already there.
db.Exec("ALTER TABLE grids ADD COLUMN source TEXT NOT NULL DEFAULT ''")
s := &Store{db: db, dirty: map[string]Entry{}, stop: make(chan struct{}), logf: logf}
if n, err := s.prune(); err != nil {
s.logf("gridcache: prune failed: %v", err)
} else if n > 0 {
s.logf("gridcache: pruned %d locators not heard in %d days", n, int(Retention.Hours()/24))
}
return s, nil
}
func (s *Store) prune() (int64, error) {
cut := time.Now().Add(-Retention).Unix()
res, err := s.db.Exec(`DELETE FROM grids WHERE updated_at < ?`, cut)
if err != nil {
return 0, err
}
return res.RowsAffected()
}
// LoadAll returns every stored locator, for seeding the in-memory map at
// startup. One query and one map build — the point of the whole package is that
// nothing afterwards has to ask the database anything.
func (s *Store) LoadAll() (map[string]string, error) {
rows, err := s.db.Query(`SELECT call, grid FROM grids`)
if err != nil {
return nil, fmt.Errorf("gridcache: load: %w", err)
}
defer rows.Close()
out := make(map[string]string, 4096)
for rows.Next() {
var call, grid string
if err := rows.Scan(&call, &grid); err != nil {
return nil, err
}
out[call] = grid
}
return out, rows.Err()
}
// Put queues a locator for writing. Callers pass only what CHANGED — an
// unchanged report is the common case by a wide margin and must not reach here,
// or the batch would carry the whole feed instead of the news in it.
func (s *Store) Put(call, grid, source string) {
call = strings.ToUpper(strings.TrimSpace(call))
grid = strings.TrimSpace(grid)
if call == "" || grid == "" {
return
}
s.mu.Lock()
s.dirty[call] = Entry{Grid: grid, Source: source}
s.mu.Unlock()
}
// Start runs the flush loop until Close.
func (s *Store) Start(ctx context.Context) {
s.wg.Add(1)
go func() {
defer s.wg.Done()
t := time.NewTicker(FlushEvery)
defer t.Stop()
for {
select {
case <-s.stop:
return
case <-ctx.Done():
return
case <-t.C:
if err := s.Flush(); err != nil {
s.logf("gridcache: flush failed: %v", err)
}
}
}
}()
}
// Flush writes the pending batch in one transaction. Safe to call with nothing
// pending, which is most of the time on a quiet band.
func (s *Store) Flush() error {
s.mu.Lock()
if len(s.dirty) == 0 {
s.mu.Unlock()
return nil
}
batch := s.dirty
s.dirty = map[string]Entry{}
s.mu.Unlock()
tx, err := s.db.Begin()
if err != nil {
s.requeue(batch)
return err
}
st, err := tx.Prepare(`INSERT INTO grids (call, grid, updated_at, source) VALUES (?, ?, ?, ?)
ON CONFLICT(call) DO UPDATE SET grid = excluded.grid, updated_at = excluded.updated_at,
source = excluded.source`)
if err != nil {
tx.Rollback()
s.requeue(batch)
return err
}
defer st.Close()
now := time.Now().Unix()
for call, e := range batch {
if _, err := st.Exec(call, e.Grid, now, e.Source); err != nil {
tx.Rollback()
s.requeue(batch)
return err
}
}
if err := tx.Commit(); err != nil {
s.requeue(batch)
return err
}
// Logged because a batch that never reaches the disk looks exactly like one
// that does: the locators are in memory either way until the next restart,
// which is the one moment the difference shows.
s.logf("gridcache: wrote %d locators", len(batch))
return nil
}
// requeue puts a failed batch back, without overwriting anything learnt while it
// was in flight — the newer value is the right one.
func (s *Store) requeue(batch map[string]Entry) {
s.mu.Lock()
defer s.mu.Unlock()
for call, e := range batch {
if _, newer := s.dirty[call]; !newer {
s.dirty[call] = e
}
}
}
// Pending reports how many locators are waiting to be written.
func (s *Store) Pending() int {
s.mu.Lock()
defer s.mu.Unlock()
return len(s.dirty)
}
// Close stops the loop and writes whatever is pending. A restart is the moment
// the cache is most valuable, so losing the last minute of learning to a clean
// shutdown would be a poor trade.
func (s *Store) Close() error {
s.stopOnce.Do(func() { close(s.stop) })
s.wg.Wait()
err := s.Flush()
if cerr := s.db.Close(); err == nil {
err = cerr
}
return err
}
+153
View File
@@ -0,0 +1,153 @@
package gridcache
import (
"path/filepath"
"testing"
"time"
)
func open(t *testing.T) (*Store, string) {
t.Helper()
path := filepath.Join(t.TempDir(), "grids.db")
s, err := Open(path, nil)
if err != nil {
t.Fatalf("open: %v", err)
}
return s, path
}
// The whole reason the store exists: what was learnt is still there after a
// restart, so the cluster's locator column is full in the first second instead
// of after an hour of listening.
func TestSurvivesRestart(t *testing.T) {
s, path := open(t)
s.Put("F4BPO", "JN36", SourceDecode)
s.Put("OH5CX", "KP30", SourceDecode)
if err := s.Flush(); err != nil {
t.Fatalf("flush: %v", err)
}
if err := s.Close(); err != nil {
t.Fatalf("close: %v", err)
}
again, err := Open(path, nil)
if err != nil {
t.Fatalf("reopen: %v", err)
}
defer again.Close()
got, err := again.LoadAll()
if err != nil {
t.Fatalf("load: %v", err)
}
if got["F4BPO"] != "JN36" || got["OH5CX"] != "KP30" {
t.Errorf("locators lost across a restart: %v", got)
}
}
// A callsign has ONE grid and the newest report wins. Operators move, go
// portable, go on expedition — and a stale locator is worse than none for grid
// chasing, because it reads as a square already worked.
func TestNewestReportWins(t *testing.T) {
s, _ := open(t)
defer s.Close()
s.Put("F4BPO", "JN36", SourceDecode)
if err := s.Flush(); err != nil {
t.Fatal(err)
}
s.Put("F4BPO", "KP30", SourceDecode) // moved
if err := s.Flush(); err != nil {
t.Fatal(err)
}
got, err := s.LoadAll()
if err != nil {
t.Fatal(err)
}
if got["F4BPO"] != "KP30" {
t.Errorf("grid = %q, want the newer KP30", got["F4BPO"])
}
if len(got) != 1 {
t.Errorf("a callsign must hold one row, got %d: %v", len(got), got)
}
}
// Nothing reaches the disk until a flush, and a flush with nothing pending is
// not an error — that is most minutes on a quiet band.
func TestBatching(t *testing.T) {
s, _ := open(t)
defer s.Close()
for _, c := range []string{"A1AA", "B2BB", "C3CC"} {
s.Put(c, "JN36", SourceDecode)
}
if n := s.Pending(); n != 3 {
t.Errorf("pending = %d, want 3 queued and unwritten", n)
}
if got, _ := s.LoadAll(); len(got) != 0 {
t.Errorf("wrote before the flush: %v", got)
}
if err := s.Flush(); err != nil {
t.Fatal(err)
}
if n := s.Pending(); n != 0 {
t.Errorf("pending = %d after a flush, want 0", n)
}
if got, _ := s.LoadAll(); len(got) != 3 {
t.Errorf("flush wrote %d rows, want 3", len(got))
}
if err := s.Flush(); err != nil {
t.Errorf("empty flush must be a no-op, got %v", err)
}
}
// Age is what bounds a store that would otherwise only grow. A callsign not
// heard in two years is likely reassigned, and carrying its previous holder's
// square is the one way this cache can be actively wrong rather than empty.
func TestPruneOnOpen(t *testing.T) {
s, path := open(t)
s.Put("FRESH", "JN36", SourceDecode)
s.Put("STALE", "IO91", SourceDecode)
if err := s.Flush(); err != nil {
t.Fatal(err)
}
// Backdate one row past the retention window.
old := time.Now().Add(-Retention - 24*time.Hour).Unix()
if _, err := s.db.Exec(`UPDATE grids SET updated_at = ? WHERE call = 'STALE'`, old); err != nil {
t.Fatal(err)
}
s.Close()
again, err := Open(path, nil)
if err != nil {
t.Fatal(err)
}
defer again.Close()
got, _ := again.LoadAll()
if _, ok := got["STALE"]; ok {
t.Error("an entry past the retention window survived — the store is unbounded")
}
if got["FRESH"] != "JN36" {
t.Error("pruning took a live entry with it")
}
}
// Close has to write what the last minute learnt: a restart is exactly when the
// cache is worth the most, so losing it to a clean shutdown would be a poor
// trade.
func TestCloseFlushes(t *testing.T) {
s, path := open(t)
s.Put("LATE", "JN36", SourceDecode)
if err := s.Close(); err != nil {
t.Fatalf("close: %v", err)
}
again, err := Open(path, nil)
if err != nil {
t.Fatal(err)
}
defer again.Close()
got, _ := again.LoadAll()
if got["LATE"] != "JN36" {
t.Error("what was pending at shutdown was dropped")
}
}
+29 -1
View File
@@ -94,6 +94,13 @@ func (h *HamQTH) fetch(ctx context.Context, sessionID, callsign string) (Result,
if err != nil { if err != nil {
return Result{}, err return Result{}, err
} }
return parseHamQTHSearch(body)
}
// parseHamQTHSearch turns one answer into a Result. Split out from the request
// so the field mapping can be checked against a captured payload — which is how
// the missing full name and picture were found.
func parseHamQTHSearch(body []byte) (Result, error) {
var resp hamqthRoot var resp hamqthRoot
if err := xml.Unmarshal(body, &resp); err != nil { if err := xml.Unmarshal(body, &resp); err != nil {
return Result{}, fmt.Errorf("hamqth: parse callsign: %w", err) return Result{}, fmt.Errorf("hamqth: parse callsign: %w", err)
@@ -112,11 +119,20 @@ func (h *HamQTH) fetch(ctx context.Context, sessionID, callsign string) (Result,
if s.Callsign == "" { if s.Callsign == "" {
return Result{}, ErrNotFound return Result{}, ErrNotFound
} }
// <nick> is the name the operator goes BY on the air, which is what belongs
// in a log — "Igor", not "Igor Vladimirovich Getman". adr_name is the postal
// name and is used only when there is nothing else, so a record carrying
// neither nick nor name no longer resolves to blank.
name := strings.TrimSpace(s.Nick + " " + s.LastName)
if name == "" {
name = strings.TrimSpace(s.AdrName)
}
r := Result{ r := Result{
Callsign: strings.ToUpper(s.Callsign), Callsign: strings.ToUpper(s.Callsign),
Name: strings.TrimSpace(s.Nick + " " + s.LastName), Name: name,
QTH: firstNonEmpty(s.QTH, s.AdrCity), QTH: firstNonEmpty(s.QTH, s.AdrCity),
Address: s.AdrStreet1, Address: s.AdrStreet1,
Zip: s.AdrZip,
State: strings.ToUpper(s.USState), State: strings.ToUpper(s.USState),
County: s.USCounty, County: s.USCounty,
Country: firstNonEmpty(s.AdrCountry, s.Country), Country: firstNonEmpty(s.AdrCountry, s.Country),
@@ -124,6 +140,8 @@ func (h *HamQTH) fetch(ctx context.Context, sessionID, callsign string) (Result,
Continent: strings.ToUpper(s.Continent), Continent: strings.ToUpper(s.Continent),
Email: s.Email, Email: s.Email,
QSLVia: s.QSLVia, QSLVia: s.QSLVia,
Web: s.Web,
ImageURL: s.Picture,
} }
r.Lat, _ = strconv.ParseFloat(s.Latitude, 64) r.Lat, _ = strconv.ParseFloat(s.Latitude, 64)
r.Lon, _ = strconv.ParseFloat(s.Longitude, 64) r.Lon, _ = strconv.ParseFloat(s.Longitude, 64)
@@ -182,4 +200,14 @@ type hamqthSearch struct {
Continent string `xml:"continent"` Continent string `xml:"continent"`
Email string `xml:"email"` Email string `xml:"email"`
QSLVia string `xml:"qsl_via"` QSLVia string `xml:"qsl_via"`
// AdrName is the full postal name. Many records carry it and no <name> at
// all, so building the name from <nick> + <name> silently kept the first
// name and dropped the rest.
AdrName string `xml:"adr_name"`
AdrZip string `xml:"adr_zip"`
// Picture is HamQTH's profile photo. Result.ImageURL was documented as "QRZ
// only" because this element was never read — QRZ calls the same thing
// <image>.
Picture string `xml:"picture"`
Web string `xml:"web"`
} }
+128
View File
@@ -0,0 +1,128 @@
package lookup
import (
"encoding/xml"
"testing"
)
// A real HamQTH answer, captured from the live API.
//
// Two things it settles. The email IS returned and must reach the QSO. And the
// element names are not the ones a reading of the documentation suggests:
// there is no <name> here at all, the full name is <adr_name>, and the picture
// is <picture> where QRZ calls it <image>. The stray <div> is an advert the
// server injects into its own XML; the parser has to shrug it off.
const hamqthEU1EU = `<?xml version="1.0"?>
<HamQTH xmlns="https://www.hamqth.com" version="2.8">
<div id="in-page-channel-node-id" data-channel-name="in_page_channel_Sux7_K"/>
<search>
<callsign>eu1eu</callsign>
<nick>Igor</nick>
<qth>Minsk-5</qth>
<country>Belarus</country>
<adif>27</adif>
<itu>29</itu>
<cq>16</cq>
<grid>KO33SV</grid>
<adr_name>Igor Vladimirovich Getman</adr_name>
<adr_street1>A/ya 143</adr_street1>
<adr_city>Minsk-5</adr_city>
<adr_zip>220005</adr_zip>
<adr_country>Belarus</adr_country>
<adr_adif>27</adr_adif>
<district>WAARB-LE</district>
<lotw>?</lotw>
<qsldirect>?</qsldirect>
<qsl>?</qsl>
<eqsl>Y</eqsl>
<email>[email protected]</email>
<latitude>53.88999938964844</latitude>
<longitude>27.59000015258789</longitude>
<continent>EU</continent>
<utc_offset>-2</utc_offset>
<picture>https://www.hamqth.com/images/default/ts-930_qith_old_radios.jpg</picture>
</search>
</HamQTH>`
func TestHamQTHParsesEveryUsefulField(t *testing.T) {
var root hamqthRoot
if err := xml.Unmarshal([]byte(hamqthEU1EU), &root); err != nil {
t.Fatalf("parse: %v", err)
}
s := root.Search
for _, c := range []struct{ name, got, want string }{
{"callsign", s.Callsign, "eu1eu"},
{"email", s.Email, "[email protected]"},
{"grid", s.Grid, "KO33SV"},
{"qth", s.QTH, "Minsk-5"},
{"street", s.AdrStreet1, "A/ya 143"},
{"city", s.AdrCity, "Minsk-5"},
{"country", s.AdrCountry, "Belarus"},
{"continent", s.Continent, "EU"},
{"dxcc", s.DXCC, "27"},
{"cq", s.CQ, "16"},
{"itu", s.ITU, "29"},
} {
if c.got != c.want {
t.Errorf("%s = %q, want %q", c.name, c.got, c.want)
}
}
}
// The fields the parser was blind to. Each is data OpsLog has somewhere to put.
func TestHamQTHParsesTheFieldsThatWereMissing(t *testing.T) {
var root hamqthRoot
if err := xml.Unmarshal([]byte(hamqthEU1EU), &root); err != nil {
t.Fatalf("parse: %v", err)
}
s := root.Search
// The postal name, the fallback when a record carries no nick and no name.
if s.AdrName != "Igor Vladimirovich Getman" {
t.Errorf("adr_name = %q", s.AdrName)
}
if s.LastName != "" {
t.Errorf("this record has no <name>; got %q", s.LastName)
}
// The profile picture. Result.ImageURL existed and said "QRZ only" — HamQTH
// sends one too, under a different element name.
if s.Picture == "" {
t.Error("picture not parsed")
}
if s.AdrZip != "220005" {
t.Errorf("adr_zip = %q", s.AdrZip)
}
}
// End to end: what the provider hands back must carry the email, the full name
// and the picture.
func TestHamQTHResultCarriesTheLot(t *testing.T) {
r, err := parseHamQTHSearch([]byte(hamqthEU1EU))
if err != nil {
t.Fatalf("parse: %v", err)
}
if r.Email != "[email protected]" {
t.Errorf("Email = %q — this is what was reported missing", r.Email)
}
// The nick, not the postal name: a log wants the name the operator goes by
// on the air. adr_name is only the fallback when there is nothing else.
if r.Name != "Igor" {
t.Errorf("Name = %q, want the on-air nick", r.Name)
}
if r.ImageURL == "" {
t.Error("ImageURL empty — HamQTH sent a picture")
}
if r.Grid != "KO33SV" {
t.Errorf("Grid = %q", r.Grid)
}
if r.Lat < 53.8 || r.Lat > 53.9 {
t.Errorf("Lat = %v", r.Lat)
}
if r.DXCC != 27 || r.CQZ != 16 || r.ITUZ != 29 {
t.Errorf("dxcc/cq/itu = %d/%d/%d", r.DXCC, r.CQZ, r.ITUZ)
}
if r.Address != "A/ya 143" {
t.Errorf("Address = %q", r.Address)
}
}
+16 -7
View File
@@ -37,7 +37,13 @@ type Result struct {
Continent string `json:"cont,omitempty"` Continent string `json:"cont,omitempty"`
Email string `json:"email,omitempty"` Email string `json:"email,omitempty"`
QSLVia string `json:"qsl_via,omitempty"` QSLVia string `json:"qsl_via,omitempty"`
ImageURL string `json:"image_url,omitempty"` // profile picture URL (QRZ only for now) // Web is the operator's own site. The QSO table has had a `web` column all
// along and nothing ever filled it, because no provider mapping read the
// field.
Web string `json:"web,omitempty"`
// Zip is the postal code. HamQTH and QRZ both send one.
Zip string `json:"zip,omitempty"`
ImageURL string `json:"image_url,omitempty"` // profile picture URL
Source string `json:"source"` // "qrz", "hamqth", or "cache" Source string `json:"source"` // "qrz", "hamqth", or "cache"
FetchedAt time.Time `json:"fetched_at"` FetchedAt time.Time `json:"fetched_at"`
} }
@@ -446,12 +452,13 @@ func (c *Cache) Get(ctx context.Context, callsign string) (Result, bool) {
row := c.db.QueryRowContext(ctx, ` row := c.db.QueryRowContext(ctx, `
SELECT callsign, name, qth, address, state, cnty, country, grid, SELECT callsign, name, qth, address, state, cnty, country, grid,
lat, lon, dxcc, cqz, ituz, cont, email, qsl_via, image_url, lat, lon, dxcc, cqz, ituz, cont, email, qsl_via, image_url,
source, fetched_at web, zip, source, fetched_at
FROM callsign_cache WHERE callsign = ?`, callsign) FROM callsign_cache WHERE callsign = ?`, callsign)
var ( var (
r Result r Result
name, qth, addr, state, cnty sql.NullString name, qth, addr, state, cnty sql.NullString
country, grid, cont, email, qslVia, image sql.NullString country, grid, cont, email, qslVia, image sql.NullString
web, zip sql.NullString
src string src string
dxcc, cqz, ituz sql.NullInt64 dxcc, cqz, ituz sql.NullInt64
lat, lon sql.NullFloat64 lat, lon sql.NullFloat64
@@ -459,7 +466,7 @@ func (c *Cache) Get(ctx context.Context, callsign string) (Result, bool) {
) )
if err := row.Scan(&r.Callsign, &name, &qth, &addr, &state, &cnty, if err := row.Scan(&r.Callsign, &name, &qth, &addr, &state, &cnty,
&country, &grid, &lat, &lon, &country, &grid, &lat, &lon,
&dxcc, &cqz, &ituz, &cont, &email, &qslVia, &image, &dxcc, &cqz, &ituz, &cont, &email, &qslVia, &image, &web, &zip,
&src, &fetched); err != nil { &src, &fetched); err != nil {
return Result{}, false return Result{}, false
} }
@@ -481,6 +488,8 @@ func (c *Cache) Get(ctx context.Context, callsign string) (Result, bool) {
r.Lon = lon.Float64 r.Lon = lon.Float64
r.Continent = cont.String r.Continent = cont.String
r.Email = email.String r.Email = email.String
r.Web = web.String
r.Zip = zip.String
r.QSLVia = qslVia.String r.QSLVia = qslVia.String
r.ImageURL = image.String r.ImageURL = image.String
r.DXCC = int(dxcc.Int64) r.DXCC = int(dxcc.Int64)
@@ -497,7 +506,7 @@ func (c *Cache) Put(ctx context.Context, r Result) error {
updateCols := []string{ updateCols := []string{
"name", "qth", "address", "state", "cnty", "name", "qth", "address", "state", "cnty",
"country", "grid", "lat", "lon", "country", "grid", "lat", "lon",
"dxcc", "cqz", "ituz", "cont", "email", "qsl_via", "image_url", "dxcc", "cqz", "ituz", "cont", "email", "qsl_via", "image_url", "web", "zip",
"source", "fetched_at", "source", "fetched_at",
} }
// The lookup cache always lives in the local SQLite database, so SQLite // The lookup cache always lives in the local SQLite database, so SQLite
@@ -510,8 +519,8 @@ func (c *Cache) Put(ctx context.Context, r Result) error {
INSERT INTO callsign_cache(callsign, name, qth, address, state, cnty, INSERT INTO callsign_cache(callsign, name, qth, address, state, cnty,
country, grid, lat, lon, country, grid, lat, lon,
dxcc, cqz, ituz, cont, email, qsl_via, image_url, dxcc, cqz, ituz, cont, email, qsl_via, image_url,
source, fetched_at) web, zip, source, fetched_at)
VALUES(?,?,?,?,?,?, ?,?,?,?, ?,?,?,?,?,?,?, ?,?) VALUES(?,?,?,?,?,?, ?,?,?,?, ?,?,?,?,?,?,?, ?,?, ?,?)
ON CONFLICT(callsign) DO UPDATE SET ` + strings.Join(sets, ", ") ON CONFLICT(callsign) DO UPDATE SET ` + strings.Join(sets, ", ")
_, err := c.db.ExecContext(ctx, q, _, err := c.db.ExecContext(ctx, q,
r.Callsign, nullable(r.Name), nullable(r.QTH), nullable(r.Address), r.Callsign, nullable(r.Name), nullable(r.QTH), nullable(r.Address),
@@ -520,7 +529,7 @@ func (c *Cache) Put(ctx context.Context, r Result) error {
nullableFloat(r.Lat), nullableFloat(r.Lon), nullableFloat(r.Lat), nullableFloat(r.Lon),
nullableInt(r.DXCC), nullableInt(r.CQZ), nullableInt(r.ITUZ), nullableInt(r.DXCC), nullableInt(r.CQZ), nullableInt(r.ITUZ),
nullable(r.Continent), nullable(r.Email), nullable(r.QSLVia), nullable(r.Continent), nullable(r.Email), nullable(r.QSLVia),
nullable(r.ImageURL), nullable(r.ImageURL), nullable(r.Web), nullable(r.Zip),
r.Source, db.NowISO(), r.Source, db.NowISO(),
) )
return err return err
+44 -5
View File
@@ -81,7 +81,16 @@ type Config struct {
// OnSpot receives every accepted decode. Called from the MQTT goroutine, so // OnSpot receives every accepted decode. Called from the MQTT goroutine, so
// it must not block: the broker's buffer is what pays for it if it does. // it must not block: the broker's buffer is what pays for it if it does.
OnSpot func(Spot) OnSpot func(Spot)
Logf func(string, ...any) // OnGrid receives the transmitter of EVERY message, before any geographic
// filtering, for the callsign-to-locator store. Same goroutine as OnSpot and
// the same rule: do not block.
OnGrid func(call, grid string)
// RxGrids filters at the BROKER: only reports collected by a receiver in one
// of these squares are sent at all. Empty keeps the old behaviour, which was
// to receive the world and discard it here — measured at 83 messages a second
// for the four opening bands, of which about one in a hundred survived.
RxGrids []string
Logf func(string, ...any)
} }
// Watcher owns the MQTT connection and its subscriptions. // Watcher owns the MQTT connection and its subscriptions.
@@ -115,6 +124,32 @@ func New(cfg Config) *Watcher {
return &Watcher{cfg: cfg} return &Watcher{cfg: cfg}
} }
// topics builds the subscription list.
//
// The v2 topic is
//
// pskr/filter/v2/<band>/<mode>/<tx call>/<rx call>/<tx grid>/<rx grid>/<tx dxcc>/<rx dxcc>
//
// so the receiver's square is a level the broker can filter on, and a band of
// "+" means every band. Filtering by RECEIVER square rather than by receiver
// DXCC is deliberate: measured on 20 m, one country ranged from 1.2 messages a
// second (OH) to 72.5 (K), because a DXCC can be a continent. By square the
// same measurement is 0.2 to 1.2 — the load follows distance, which is what the
// feed is actually about, and it is the same for every operator.
func (w *Watcher) topics() []string {
out := []string{}
for _, b := range w.cfg.Bands {
if len(w.cfg.RxGrids) == 0 {
out = append(out, "pskr/filter/v2/"+b+"/#")
continue
}
for _, g := range w.cfg.RxGrids {
out = append(out, "pskr/filter/v2/"+b+"/+/+/+/+/"+strings.ToUpper(g)+"/+/+")
}
}
return out
}
// Start connects and subscribes. Safe to call when already running. // Start connects and subscribes. Safe to call when already running.
func (w *Watcher) Start() error { func (w *Watcher) Start() error {
w.mu.Lock() w.mu.Lock()
@@ -143,10 +178,7 @@ func (w *Watcher) Start() error {
opts.OnConnect = func(c mqtt.Client) { opts.OnConnect = func(c mqtt.Client) {
w.cfg.Logf("pskr: connected to %s", w.cfg.Broker) w.cfg.Logf("pskr: connected to %s", w.cfg.Broker)
for _, b := range w.cfg.Bands { for _, topic := range w.topics() {
// Every mode, every pair of stations, on this band. That firehose IS
// the point: the detector's job is to find the shape in it.
topic := "pskr/filter/v2/" + b + "/#"
if tok := c.Subscribe(topic, 0, w.handle); tok.Wait() && tok.Error() != nil { if tok := c.Subscribe(topic, 0, w.handle); tok.Wait() && tok.Error() != nil {
w.cfg.Logf("pskr: subscribe %s failed: %v", topic, tok.Error()) w.cfg.Logf("pskr: subscribe %s failed: %v", topic, tok.Error())
continue continue
@@ -206,6 +238,13 @@ func (w *Watcher) handle(_ mqtt.Client, m mqtt.Message) {
return return
} }
// The locator store takes every transmitter, before any of the geography
// below. What it wants is "which square is this callsign in", and that is
// true whoever happened to hear the report.
if w.cfg.OnGrid != nil {
w.cfg.OnGrid(call, grid[:4])
}
// THE RECEIVER HAS TO BE NEAR THE OPERATOR. This is the whole difference // THE RECEIVER HAS TO BE NEAR THE OPERATOR. This is the whole difference
// between a useful feed and a world map. // between a useful feed and a world map.
// //
+46
View File
@@ -0,0 +1,46 @@
package pskr
import (
"strings"
"testing"
)
// The receiver square is a level the BROKER can filter on, which is the whole
// point: measured on the live feed, the four opening bands unfiltered are 83
// messages a second of which about one in a hundred survives the NearKm test.
// One ring of squares is under two a second, and the same for every operator —
// where filtering by DXCC ranged from 1.2 (OH) to 72.5 (K) on one band.
func TestTopicsFilterOnTheReceiverSquare(t *testing.T) {
w := New(Config{Bands: []string{"6m"}, RxGrids: []string{"jn36", "JN37"}})
got := w.topics()
if len(got) != 2 {
t.Fatalf("want one subscription per band × square, got %v", got)
}
// Level order: band/mode/txcall/rxcall/txgrid/RXGRID/txdxcc/rxdxcc
want := "pskr/filter/v2/6m/+/+/+/+/JN36/+/+"
if got[0] != want {
t.Errorf("topic = %q, want %q", got[0], want)
}
if !strings.Contains(got[1], "/JN37/") {
t.Errorf("square not upper-cased into the topic: %q", got[1])
}
}
// With no squares the old behaviour stands: receive the band and decide here.
func TestTopicsWithoutSquares(t *testing.T) {
w := New(Config{Bands: []string{"10m", "2m"}})
got := w.topics()
if len(got) != 2 || got[0] != "pskr/filter/v2/10m/#" {
t.Errorf("unfiltered topics = %v", got)
}
}
// Grid chasing wants every band. "+" is the MQTT single-level wildcard, so one
// subscription per square covers the lot instead of one per band per square.
func TestTopicsAllBands(t *testing.T) {
w := New(Config{Bands: []string{"+"}, RxGrids: []string{"JN36"}})
got := w.topics()
if len(got) != 1 || got[0] != "pskr/filter/v2/+/+/+/+/+/JN36/+/+" {
t.Errorf("all-band topic = %v", got)
}
}
+64
View File
@@ -0,0 +1,64 @@
package qso
import (
"context"
"database/sql"
"path/filepath"
"testing"
"time"
_ "modernc.org/sqlite"
)
// OrderedIDs against a real SQLite file, with qso_date stored the way the repo
// actually writes it.
//
// The first version scanned that column straight into a time.Time. It compiles,
// it reads correctly, and it fails at run time — the column holds a formatted
// STRING. The error was swallowed into "the column will be empty", which is
// exactly what shipped: a column that was always blank.
func TestOrderedIDsAgainstRealSQLite(t *testing.T) {
conn, err := sql.Open("sqlite", "file:"+filepath.Join(t.TempDir(), "t.db"))
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { conn.Close() })
if _, err := conn.Exec(`CREATE TABLE qso (
id INTEGER PRIMARY KEY AUTOINCREMENT,
callsign TEXT NOT NULL,
qso_date TEXT NOT NULL
)`); err != nil {
t.Fatal(err)
}
// Inserted NEWEST first, so the ids run opposite to the dates — what an
// imported ADIF produces, and the whole reason the id is not the number.
for _, row := range []struct {
call string
at time.Time
}{
{"NEWEST", time.Date(2026, 8, 13, 10, 0, 0, 0, time.UTC)},
{"MIDDLE", time.Date(2020, 1, 1, 10, 0, 0, 0, time.UTC)},
{"OLDEST", time.Date(1999, 5, 5, 10, 0, 0, 0, time.UTC)},
} {
if _, err := conn.Exec(`INSERT INTO qso (callsign, qso_date) VALUES (?, ?)`,
row.call, row.at.UTC().Format(isoMillis)); err != nil {
t.Fatal(err)
}
}
ids, newest, err := NewRepo(conn).OrderedIDs(context.Background())
if err != nil {
t.Fatalf("OrderedIDs: %v — this is the failure that emptied the column", err)
}
if len(ids) != 3 {
t.Fatalf("got %d ids, want 3", len(ids))
}
// Rows 1,2,3 were inserted newest→oldest, so chronological order is 3,2,1.
if ids[0] != 3 || ids[1] != 2 || ids[2] != 1 {
t.Errorf("order = %v, want the oldest contact first (3,2,1)", ids)
}
if newest.Year() != 2026 {
t.Errorf("newest = %v, want the 2026 contact", newest)
}
}
+45 -1
View File
@@ -47,7 +47,15 @@ func MergeNonZero(dst *QSO, src QSO) {
// import/export. Pointers are used to distinguish "absent" from "zero". // import/export. Pointers are used to distinguish "absent" from "zero".
// Anything in ADIF that is not a promoted column lands in Extras. // Anything in ADIF that is not a promoted column lands in Extras.
type QSO struct { type QSO struct {
ID int64 `json:"id"` ID int64 `json:"id"`
// Number is the contact's position in the log, oldest = 1.
//
// NOT a column and NOT the ID: the primary key follows insertion order, so
// importing an old ADIF gives the oldest contacts the highest ids. This is a
// rank over qso_date, computed by the App layer and stamped on the way out —
// which is why it is `json:"-"`-adjacent in spirit: nothing reads or writes
// it in SQL.
Number int `json:"number,omitempty"`
Callsign string `json:"callsign"` Callsign string `json:"callsign"`
QSODate time.Time `json:"qso_date"` // start, UTC QSODate time.Time `json:"qso_date"` // start, UTC
QSODateOff time.Time `json:"qso_date_off,omitempty"` // end, UTC QSODateOff time.Time `json:"qso_date_off,omitempty"` // end, UTC
@@ -3170,3 +3178,39 @@ func (r *Repo) WorkedGridKeys(ctx context.Context, normMode func(string) string)
} }
return out, rows.Err() return out, rows.Err()
} }
// OrderedIDs returns every QSO id in chronological order, oldest first.
//
// The mirror of ListFiltered's "qso_date DESC, id DESC", so the tie-break is the
// same one the grid displays and a contact cannot change number depending on
// which way it is read.
//
// One query, ids only: the caller turns it into a rank once and keeps it in
// memory. Asking the database for a row's rank per row would be a correlated
// count over the whole log for each of thirty thousand rows.
func (r *Repo) OrderedIDs(ctx context.Context) ([]int64, time.Time, error) {
rows, err := r.db.QueryContext(ctx, `SELECT id, qso_date FROM qso ORDER BY qso_date ASC, id ASC`)
if err != nil {
return nil, time.Time{}, err
}
defer rows.Close()
out := make([]int64, 0, 4096)
// qso_date is stored as a formatted STRING, not a driver date. Scanning it
// into a time.Time silently fails on SQLite, which is how this returned an
// error and left the whole column empty.
var lastDate string
for rows.Next() {
var id int64
var at string
if err := rows.Scan(&id, &at); err != nil {
return nil, time.Time{}, err
}
out = append(out, id)
lastDate = at // ascending, so the last row read is the newest
}
if err := rows.Err(); err != nil {
return nil, time.Time{}, err
}
// Parsed once, for the newest row only — the ordering came from SQL.
return out, parseTimeLoose(lastDate), nil
}
+95
View File
@@ -0,0 +1,95 @@
package main
import (
"testing"
"time"
"hamlog/internal/qso"
)
func at(s string) time.Time {
t, err := time.Parse("2006-01-02 15:04", s)
if err != nil {
panic(err)
}
return t
}
// The number is the contact's rank in the WHOLE log, not its position in what
// the grid happens to be showing. Ranking inside the result would renumber
// every contact the moment a filter is applied, and QSO #1 would change
// identity as the operator typed.
func TestQSONumberIsGlobalNotPerPage(t *testing.T) {
a := &App{}
// The log, oldest first: ids are deliberately NOT in date order, which is
// exactly what an imported ADIF produces.
a.qsoNumbers = map[int64]int{
770: 1, // oldest, imported last so it has the highest id
101: 2,
102: 3,
103: 4,
}
// A filtered page holding only two of them, newest first as the grid asks.
page := []qso.QSO{{ID: 103}, {ID: 770}}
a.stampQSONumbers(page)
if page[0].Number != 4 {
t.Errorf("id 103 numbered %d, want 4", page[0].Number)
}
if page[1].Number != 1 {
t.Errorf("id 770 numbered %d, want 1 — the oldest contact, whatever its id", page[1].Number)
}
}
// A contact logged now is the newest, so it takes the next number without
// rereading the log — a full scan per QSO would be felt in a contest run.
func TestNewQSOTakesTheNextNumber(t *testing.T) {
a := &App{}
a.qsoNumbers = map[int64]int{1: 1, 2: 2, 3: 3}
a.qsoNumMax = at("2026-08-13 10:00")
a.noteQSONumbered(9, at("2026-08-13 11:00"))
if got := a.qsoNumbers[9]; got != 4 {
t.Errorf("new QSO numbered %d, want 4", got)
}
if !a.qsoNumMax.Equal(at("2026-08-13 11:00")) {
t.Error("the newest date was not carried forward")
}
}
// A contact entered with an OLDER date belongs in the middle of the order.
// Appending it would number it last, which is wrong — so the map is dropped and
// rebuilt correctly instead.
func TestBackdatedQSOForcesARebuild(t *testing.T) {
a := &App{}
a.qsoNumbers = map[int64]int{1: 1, 2: 2, 3: 3}
a.qsoNumMax = at("2026-08-13 10:00")
a.noteQSONumbered(9, at("2020-01-01 09:00"))
if a.qsoNumbers != nil {
t.Errorf("a back-dated QSO was appended as the newest: %v", a.qsoNumbers)
}
}
// Nothing built yet: the lazy build will see the new contact anyway, so this
// must not create a one-entry map that then numbers the whole log wrongly.
func TestNoteBeforeAnyBuildDoesNothing(t *testing.T) {
a := &App{}
a.noteQSONumbered(9, at("2026-08-13 11:00"))
if a.qsoNumbers != nil {
t.Errorf("built a map from a single contact: %v", a.qsoNumbers)
}
}
// With no index available the column is simply empty — never wrong.
func TestStampWithoutIndexLeavesZero(t *testing.T) {
a := &App{} // no qso repo, so the index cannot be built
page := []qso.QSO{{ID: 1}, {ID: 2}}
a.stampQSONumbers(page)
for _, q := range page {
if q.Number != 0 {
t.Errorf("invented a number without an index: %d", q.Number)
}
}
}
+1 -1
View File
@@ -21,7 +21,7 @@ import (
const ( const (
// appVersion is stamped on every heartbeat (and could feed the About box). // appVersion is stamped on every heartbeat (and could feed the About box).
appVersion = "0.24.7" appVersion = "0.24.8"
// posthogHost is the PostHog ingestion endpoint. EU cloud by default; change // posthogHost is the PostHog ingestion endpoint. EU cloud by default; change
// to https://us.i.posthog.com for a US project. // to https://us.i.posthog.com for a US project.