rouggy/OpsLog
0
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 88 Wiki Activity

feat(decodes): an FT decodes tab fed by the inbound UDP link

Browse Source 
Every FTx decode WSJT-X, JTDX or MSHV puts on the wire, grouped by T/R
period. Optional and closable, from Tools -> FT decodes; its open state is
remembered, because an operator running digital modes leaves it open for
the session rather than consulting and closing it.

The period is the point, and what separates this from the cluster list.
FT8 is a sequence of fifteen-second slots and a band is read by watching
them go by: who called CQ this slot, who answered, what I was sending while
they did. A flat list sorted by time loses exactly that, so the list is
grouped one section per period, newest first, with the operator's own
transmission shown inside the slot it went out in.

Three fields had to be carried up from the wire to make it possible:

  - the decode's OWN timestamp, which the parser read and threw away. It is
    what assigns a slot: a period's decodes arrive in one burst a second or
    two after it closes, so arrival time piles a whole period into the next
    one. Rebuilt to UTC from milliseconds-since-midnight, with the
    day-boundary case handled - a decode stamped 23:59:58 arriving at
    00:00:01 would otherwise be dated a day ahead and sit at the top of the
    list for the rest of the session.
  - the decoded line itself. The exchange is what says where a station is in
    a QSO, and no set of extracted fields reads like "R-09" does.
  - tx_message and transmitting from Status, which nothing parsed before.
    Recorded once per message rather than on every Status, which repeats it
    about once a second for the whole over.

Also picked up on the way: is_new, low_confidence, off_air, the operator's
own call and grid, and the T/R period itself - better authority on slot
length than the mode name, which says nothing about a custom period. The
Status tail is read defensively: those fields were appended over successive
schema versions and JTDX and MSHV each stop at their own point, so a short
packet is normal and keeps whatever parsed.

Status flags come from ClusterSpotStatuses, the resolver the cluster list
and band map already use, filling the same cache. One verdict per call:
"new band" in this panel and plain worked in the cluster two seconds later
would be worse than no flag at all. Clicking a call goes through the same
handler as a cluster spot, so answering a station is one gesture whether it
came off telnet or off the receiver.

Filters: CQ only, new-anything only, band, mode, continent, an SNR floor
and a free search. The band, mode and continent choices are built from what
is actually on the feed - offering 160 m to a station whose receivers are
all on 6 m is noise.

Decodes are held in the frontend and pruned to a rolling half hour: they
are a live view, not data, nothing outside the panel reads them, and a
night of FT8 on 20 m would otherwise grow a list no filter can rescue.
Arrivals are staged on a 300 ms timer so a period landing as fifty packets
costs one status lookup and one render.
This commit is contained in:
rouggy
2026-08-18 05:51:34 +02:00
parent 9599c3e0b9
commit a197d124dc
7 changed files with 791 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files
internal/integrations/udp/decodetime_test.go
+52
View File
@@ -0,0 +1,52 @@
package udp
import (
"testing"
"time"
)
// WSJT-X stamps a decode with a time of DAY and no date, so the date has to come
// from our own clock — and around midnight the two disagree. A decode stamped
// 23:59:58 that reaches us at 00:00:01 would be dated the NEW day, putting it
// almost 24 hours in the future: it would sort to the top of the decodes panel
// and stay there for the rest of the session, and its period would never line up
// with the ones around it.
func TestDecodeTimeCrossesMidnight(t *testing.T) {
const ms = 1000
sec := func(h, m, s int) uint32 { return uint32((h*3600 + m*60 + s) * ms) }
got := decodeTime(sec(23, 59, 58))
now := time.Now().UTC()
// Whatever the clock says, a decode must never land in the future beyond the
// slack of a single period, nor more than a day in the past.
if d := got.Sub(now); d > time.Minute {
t.Errorf("decode at 23:59:58 resolved to %s, %s in the FUTURE", got.Format(time.RFC3339), d)
}
if d := now.Sub(got); d > 24*time.Hour {
t.Errorf("decode at 23:59:58 resolved to %s, %s in the past", got.Format(time.RFC3339), d)
}
// And the ordinary case: a stamp close to now stays on today.
near := decodeTime(sec(now.Hour(), now.Minute(), now.Second()))
if diff := near.Sub(now); diff > 2*time.Second || diff < -2*time.Second {
t.Errorf("a decode stamped at the current time resolved to %s (%s off)", near.Format(time.RFC3339), diff)
}
}
// The whole point of the timestamp is grouping, so two decodes from the same
// fifteen-second slot must floor to the same period however far apart in the
// slot they were heard.
func TestDecodesInOneSlotShareAPeriod(t *testing.T) {
const ms = 1000
at := func(h, m, s int) time.Time { return decodeTime(uint32((h*3600 + m*60 + s) * ms)) }
floor := func(x time.Time) int64 { return x.Unix() / 15 * 15 }
a, b := at(12, 30, 0), at(12, 30, 14)
if floor(a) != floor(b) {
t.Errorf("12:30:00 and 12:30:14 fell in different periods (%d vs %d)", floor(a), floor(b))
}
c := at(12, 30, 15)
if floor(a) == floor(c) {
t.Error("12:30:00 and 12:30:15 shared a period — the slot boundary was not honoured")
}
}
internal/integrations/udp/server.go
+67 -2
View File
@@ -77,6 +77,26 @@ func reusingListenConfig() net.ListenConfig {
}
}
// decodeTime turns WSJT-X's milliseconds-since-midnight into a UTC instant.
//
// The sender gives a time of DAY with no date, so the date comes from our own
// clock — and the two can straddle midnight: a decode stamped 23:59:58 that
// reaches us at 00:00:01 would otherwise be dated a day late and sort to the top
// of the list for the rest of the session. More than half a day apart is read as
// the wrong side of midnight and moved.
func decodeTime(msSinceMidnight uint32) time.Time {
now := time.Now().UTC()
midnight := time.Date(now.Year(), now.Month(), now.Day(), 0, 0, 0, 0, time.UTC)
at := midnight.Add(time.Duration(msSinceMidnight) * time.Millisecond)
switch {
case at.Sub(now) > 12*time.Hour:
at = at.AddDate(0, 0, -1) // stamped late yesterday, arrived after midnight
case now.Sub(at) > 12*time.Hour:
at = at.AddDate(0, 0, 1) // stamped just after midnight, our clock still on the old day
}
return at
}
// Event is what a Server emits to its consumer for every parsed packet.
// At most one of the fields is populated per event.
type Event struct {
@@ -96,6 +116,23 @@ type Event struct {
DecodeFreqHz int64 // RF frequency (dial + audio offset)
DecodeSNR int // reported SNR (dB)
DecodeCQ bool // the decode was a CQ
DecodeMsg string // the decoded line as printed ("CQ K1ABC FN42")
// DecodeAt is the decode's own UTC timestamp, rebuilt from the sender's
// milliseconds-since-midnight. It is what groups decodes into T/R periods:
// a period's worth arrives in one burst, so arrival time would put them all
// in whichever slot the burst happened to land in.
DecodeAt time.Time
// DecodeTRPeriod is the transmit/receive period in seconds, from the last
// Status of the same program (15 = FT8). 0 when the sender never said.
DecodeTRPeriod int
DecodeDial int64 // dial frequency the decode was heard on, for the band
DecodeOffAir bool // decoded from a file rather than off the air
// TxMessage is what the operator's digital app is sending, with Transmitting
// true while the carrier is actually up. From Status, so ~1 Hz.
TxMessage string
Transmitting bool
DECall string // the operator's own call, as the digital app knows it
// ClearCall is set when a WSJT/JTDX/MSHV Status message reports an EMPTY DX
// Call after previously reporting one — i.e. the operator cleared the call in
@@ -127,7 +164,10 @@ type Server struct {
// 50.400 panadapter. WSJT-X requires --rig-name for a second instance, so the
// id is distinct whenever there is more than one.
dialHz map[string]int64
lastDX string // WSJT: last non-empty DX Call seen, to detect a clear
// trPeriod is the T/R period (seconds) from each program's last Status —
// what tells a decode which slot it belongs to.
trPeriod map[string]int
lastDX string // WSJT: last non-empty DX Call seen, to detect a clear
// badPkts counts datagrams this listener could not parse, so the diagnostic
// dump below stays bounded. A misconfigured port is not a one-off: the
@@ -359,11 +399,28 @@ func (s *Server) handle(pkt []byte, remote *net.UDPAddr) {
s.dialHz = map[string]int64{}
}
s.dialHz[w.ProgramID] = w.FreqHz
// The T/R period travels with Status, and a decode has to be told
// which slot it belongs to — so it is remembered per program the
// same way the dial is.
if w.TRPeriod > 0 {
if s.trPeriod == nil {
s.trPeriod = map[string]int{}
}
s.trPeriod[w.ProgramID] = w.TRPeriod
}
s.mu.Unlock()
}
if !w.IsDecode && (w.TxMessage != "" || w.DECall != "") {
// What the operator is sending. Carried on every Status, so the
// consumer sees it change as the QSO progresses.
ev.TxMessage = w.TxMessage
ev.Transmitting = w.Transmitting
ev.DECall = w.DECall
}
if w.IsDecode {
s.mu.Lock()
dial := s.dialHz[w.ProgramID]
tr := s.trPeriod[w.ProgramID]
s.mu.Unlock()
if dial <= 0 {
// No Status from THIS instance yet. Guessing with another
@@ -377,6 +434,11 @@ func (s *Server) handle(pkt []byte, remote *net.UDPAddr) {
ev.DecodeSNR = w.SNR
ev.DecodeCQ = w.IsCQ
ev.Mode = w.Mode
ev.DecodeMsg = w.DecodeMsg
ev.DecodeAt = decodeTime(w.DecodeMsSinceMidnight)
ev.DecodeTRPeriod = tr
ev.DecodeDial = dial
ev.DecodeOffAir = w.OffAir
break
}
// Only a logged QSO is worth a line — WSJT-X/MSHV send a Status packet
@@ -501,7 +563,10 @@ func (s *Server) handle(pkt []byte, remote *net.UDPAddr) {
// Empty events are useless; skip — EXCEPT a clear signal, which is meant to be
// empty (the DX Call was cleared in the digital app), and a tune-only
// request (freq with no callsign).
if ev.DXCall == "" && ev.LoggedADIF == "" && ev.DecodeCall == "" && !ev.ClearCall && ev.TuneFreqHz == 0 {
// TxMessage rides on Status, which also carries the DX call — but a Status
// with an empty DX call and a live transmit message (calling CQ) used to be
// dropped here, and that is exactly the message the decodes panel needs.
if ev.DXCall == "" && ev.LoggedADIF == "" && ev.DecodeCall == "" && ev.TxMessage == "" && !ev.ClearCall && ev.TuneFreqHz == 0 {
return
}
select {
internal/integrations/udp/wsjt.go
+89 -11
View File
@@ -55,6 +55,34 @@ type WSJTEvent struct {
DeltaFreqHz int64 // audio offset within the passband (Hz)
SNR int // reported signal-to-noise (dB)
IsCQ bool // the decode was a CQ call
// DecodeMsg is the decoded text as WSJT-X printed it ("CQ K1ABC FN42",
// "F4BPO K1ABC -07"). Kept whole rather than only its parsed pieces: the
// exchange is what tells an operator where a station is in a QSO, and no set
// of extracted fields says "R-09" the way the line itself does.
DecodeMsg string
// DecodeMsSinceMidnight is the decode's own timestamp, in milliseconds since
// 00:00 UTC, as the sender reported it. It is what groups decodes into T/R
// PERIODS — arrival time cannot, since a whole period's decodes land in one
// burst and a slow link shifts the lot into the next slot.
DecodeMsSinceMidnight uint32
DecodeIsNew bool // sender's "is_new": first time this line was decoded
LowConfidence bool // sender is unsure of the decode
OffAir bool // decoded from a file, not off the air
// ---- Status extras ----
// TxMessage is what the operator is sending right now ("CQ F4BPO JN18"),
// with Transmitting saying whether the carrier is actually up. Both come
// from Status, so they arrive about once a second.
TxMessage string
Transmitting bool
DECall string // the operator's own callsign, as the digital app knows it
DEGrid string // and their square
// TRPeriod is the transmit/receive period in seconds (15 for FT8, 7 or 8 for
// FT4 depending on the sender's rounding). The authority on how long a slot
// is — better than inferring it from the mode name, which says nothing about
// a custom period.
TRPeriod int
}
// maxFwdHeader bounds how far into a packet the WSJT-X magic may sit behind a
@@ -166,40 +194,80 @@ func ParseWSJT(pkt []byte) (WSJTEvent, bool, error) {
return WSJTEvent{}, false, err
}
ev.DXCall = strings.ToUpper(strings.TrimSpace(dxCall))
// Skip report, tx_mode (QUtf8), tx_enabled (bool), transmitting,
// decoding, rx_df (qint32), tx_df (qint32), de_call (QUtf8),
// de_grid (QUtf8) → then dx_grid.
// report, tx_mode → skipped.
for _, name := range []string{"report", "tx_mode"} {
if _, err := readQString(r); err != nil {
return ev, true, fmt.Errorf("read %s: %w", name, err)
}
}
// 3 booleans (each 1 byte)
for i := 0; i < 3; i++ {
var b uint8
if err := binary.Read(r, binary.BigEndian, &b); err != nil {
// tx_enabled, transmitting, decoding (1 byte each). The middle one is
// worth keeping: it says the carrier is up, which is what turns TxMessage
// from "what I would send" into "what is going out".
var txEnabled, transmitting, decoding uint8
for _, p := range []*uint8{&txEnabled, &transmitting, &decoding} {
if err := binary.Read(r, binary.BigEndian, p); err != nil {
return ev, true, err
}
}
// 2 int32
ev.Transmitting = transmitting != 0
// rx_df, tx_df
var i32 int32
for i := 0; i < 2; i++ {
if err := binary.Read(r, binary.BigEndian, &i32); err != nil {
return ev, true, err
}
}
// de_call, de_grid, dx_grid
if _, err := readQString(r); err != nil {
deCall, err := readQString(r)
if err != nil {
return ev, true, err
}
if _, err := readQString(r); err != nil {
ev.DECall = strings.ToUpper(strings.TrimSpace(deCall))
deGrid, err := readQString(r)
if err != nil {
return ev, true, err
}
ev.DEGrid = strings.ToUpper(strings.TrimSpace(deGrid))
dxGrid, err := readQString(r)
if err != nil {
return ev, true, err
}
ev.DXGrid = strings.ToUpper(strings.TrimSpace(dxGrid))
// Everything past here was APPENDED to the schema over successive
// releases, and JTDX and MSHV each stop at their own point. A short
// packet is therefore normal, not an error: read as far as the sender
// went and keep what we got. That is why the tail below swallows its
// errors instead of reporting them — the fields already parsed are good.
var b uint8
if binary.Read(r, binary.BigEndian, &b) != nil { // tx_watchdog
return ev, true, nil
}
if _, err := readQString(r); err != nil { // sub_mode
return ev, true, nil
}
if binary.Read(r, binary.BigEndian, &b) != nil { // fast_mode
return ev, true, nil
}
if binary.Read(r, binary.BigEndian, &b) != nil { // special_operation_mode
return ev, true, nil
}
var u32 uint32
if binary.Read(r, binary.BigEndian, &u32) != nil { // frequency_tolerance
return ev, true, nil
}
if binary.Read(r, binary.BigEndian, &u32) != nil { // tr_period (seconds)
return ev, true, nil
}
// 0xFFFFFFFF is WSJT-X's "not applicable" for the quint32 fields.
if u32 > 0 && u32 < 3600 {
ev.TRPeriod = int(u32)
}
if _, err := readQString(r); err != nil { // configuration_name
return ev, true, nil
}
if txMsg, err := readQString(r); err == nil {
ev.TxMessage = strings.TrimSpace(txMsg)
}
return ev, true, nil
case wsjtMsgDecode:
@@ -217,6 +285,7 @@ func ParseWSJT(pkt []byte) (WSJTEvent, bool, error) {
if err := binary.Read(r, binary.BigEndian, &b); err != nil { // is_new
return WSJTEvent{}, false, err
}
ev.DecodeIsNew = b != 0
var t32, df uint32
var snr int32
if err := binary.Read(r, binary.BigEndian, &t32); err != nil { // time
@@ -240,6 +309,11 @@ func ParseWSJT(pkt []byte) (WSJTEvent, bool, error) {
if err != nil {
return WSJTEvent{}, false, err
}
// low_confidence and off_air were appended later; absent on older senders.
var lowConf, offAir uint8
_ = binary.Read(r, binary.BigEndian, &lowConf)
_ = binary.Read(r, binary.BigEndian, &offAir)
call, isCQ, grid := wsjtSender(msg)
if call == "" {
return WSJTEvent{}, false, nil // free-text / telemetry / unparseable → ignore
@@ -251,6 +325,10 @@ func ParseWSJT(pkt []byte) (WSJTEvent, bool, error) {
ev.DeltaFreqHz = int64(df)
ev.SNR = int(snr)
ev.Mode = strings.ToUpper(strings.TrimSpace(mode))
ev.DecodeMsg = strings.TrimSpace(msg)
ev.DecodeMsSinceMidnight = t32
ev.LowConfidence = lowConf != 0
ev.OffAir = offAir != 0
return ev, true, nil
case wsjtMsgLoggedADIF: