From c5e0ec903312b4b06adc155b184a56f156ffa3e5 Mon Sep 17 00:00:00 2001 From: Gregory Salaun Date: Tue, 11 Aug 2026 21:02:33 +0200 Subject: [PATCH] fix(filter): "equals nothing" found nothing MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A NULL never equals '', so a condition written that way returned zero rows. The question is perfectly clear — the operator wants the ones with nothing in that field — and answering it with silence makes the filter look broken rather than mis-stated. eq and ne with a blank value now run the empty / not-empty test. Empty on a NUMERIC column also had a real fault behind it. IFNULL(col,'')='' compares 0 against '', which SQLite calls false and MySQL calls true — one expression quietly answering two different questions depending on where the logbook lives. Numeric columns test NULL or zero explicitly now; text keeps the string test, where '' is a real value and 0 is not. --- changelog.json | 6 +++-- internal/qso/filter_empty_test.go | 41 +++++++++++++++++++++++++++++++ internal/qso/qso.go | 33 +++++++++++++++++++++++++ 3 files changed, 78 insertions(+), 2 deletions(-) create mode 100644 internal/qso/filter_empty_test.go diff --git a/changelog.json b/changelog.json index 1617174..f19ad47 100644 --- a/changelog.json +++ b/changelog.json @@ -6,13 +6,15 @@ "Bulk edit can now set mode, submode and RST. They were excluded as per-QSO fields, which missed the point: bulk edit is for repairing a batch — an import that mapped every contact to SSB, an ADIF with no mode at all — and refusing meant editing a hundred rows one at a time. Setting the mode clears the submode, since one left over from the old mode contradicts the new one. Band stays with frequency, which already sets the two together.", "Web publishing: the page now widens to fit the table. It was capped at a comfortable reading width, so with more than about eight columns the rest sat behind a scrollbar on a screen wide enough to show them all — and Windows hides that scrollbar until something moves, which made a table that scrolls look like a table missing columns. Columns also take the width their contents need instead of being squeezed to fit first.", "Every QSO now carries its distance. Nothing ever recorded one, so the field went out empty in every ADIF export and left the Distance column blank on a published page. It is computed from the two locators when a contact is logged and when an ADIF is imported, and a one-time pass fills in the QSOs already in your log the first time this version runs — in the background, without asking. A distance the imported file supplied is always kept.", - "RDA: 1015 districts were filed under the wrong DXCC entity. Every reference sat on European Russia; 991 belong to Asiatic Russia and the 24 KA- districts to Kaliningrad, which is a separate entity altogether. Corrected against the reference list, and Kaliningrad added to the award filter so those 24 can be claimed at all." + "RDA: 1015 districts were filed under the wrong DXCC entity. Every reference sat on European Russia; 991 belong to Asiatic Russia and the 24 KA- districts to Kaliningrad, which is a separate entity altogether. Corrected against the reference list, and Kaliningrad added to the award filter so those 24 can be claimed at all.", + "QSO filter: asking a field to equal nothing now finds the empty ones. SQL answers that question with nothing at all — a missing value never equals an empty one — so the filter looked broken rather than wrong. Empty on a numeric field also covers zero as well as missing, which SQLite and MySQL disagreed about." ], "fr": [ "L édition groupée sait enfin régler le mode, le sous-mode et le RST. Ils étaient exclus comme champs propres à chaque QSO, ce qui manquait l essentiel : l édition groupée sert à RÉPARER un lot — un import qui a tout mis en SSB, un ADIF sans aucun mode — et refuser obligeait à corriger cent lignes une par une. Régler le mode efface le sous-mode, celui de l ancien mode contredisant le nouveau. La bande reste avec la fréquence, qui pose déjà les deux ensemble.", "Publication web : la page s élargit désormais à la taille du tableau. Elle était bridée à une largeur de lecture confortable, donc au-delà de huit colonnes environ le reste passait derrière une barre de défilement sur un écran assez large pour tout montrer — et Windows masque cette barre tant que rien ne bouge, si bien qu un tableau qui défile ressemblait à un tableau amputé. Les colonnes prennent aussi la largeur qu il leur faut au lieu d être comprimées d abord.", "Chaque QSO porte désormais sa distance. Rien ne l enregistrait, elle partait donc vide dans chaque export ADIF et laissait la colonne Distance blanche sur une page publiée. Elle est calculée depuis les deux locators à l enregistrement d un contact et à l import d un ADIF, et une passe unique complète les QSO déjà présents au premier lancement de cette version — en tâche de fond, sans rien demander. Une distance fournie par le fichier importé est toujours conservée.", - "RDA : 1015 districts étaient rangés sous la mauvaise entité DXCC. Toutes les références étaient sur la Russie européenne ; 991 relèvent de la Russie asiatique et les 24 districts KA- de Kaliningrad, qui est une entité à part entière. Corrigé d après la liste de référence, et Kaliningrad ajouté au filtre de l award pour que ces 24 puissent être revendiqués." + "RDA : 1015 districts étaient rangés sous la mauvaise entité DXCC. Toutes les références étaient sur la Russie européenne ; 991 relèvent de la Russie asiatique et les 24 districts KA- de Kaliningrad, qui est une entité à part entière. Corrigé d après la liste de référence, et Kaliningrad ajouté au filtre de l award pour que ces 24 puissent être revendiqués.", + "Filtre QSO : demander à un champ d être égal à rien trouve désormais les vides. SQL répond à cette question par rien du tout — une valeur absente n est jamais égale à une valeur vide — et le filtre paraissait cassé plutôt que mal posé. Vide sur un champ numérique couvre aussi le zéro autant que l absence, ce sur quoi SQLite et MySQL n étaient pas d accord." ] }, { diff --git a/internal/qso/filter_empty_test.go b/internal/qso/filter_empty_test.go new file mode 100644 index 0000000..6461959 --- /dev/null +++ b/internal/qso/filter_empty_test.go @@ -0,0 +1,41 @@ +package qso + +import ( + "strings" + "testing" +) + +// "equals nothing" and "is empty" are the same question. SQL answers the first +// with nothing at all — a NULL never equals ” — so a filter written in plain +// words returned zero rows and looked broken rather than wrong. +func TestEqualsBlankMeansEmpty(t *testing.T) { + sql, args, err := conditionSQL(Condition{Field: "freq_hz", Op: "eq", Value: ""}) + if err != nil { + t.Fatalf("eq blank: %v", err) + } + if len(args) != 0 || !strings.Contains(sql, "IS NULL") { + t.Errorf("sql = %q args = %v — want the empty test", sql, args) + } + sql, _, _ = conditionSQL(Condition{Field: "name", Op: "ne", Value: " "}) + if !strings.Contains(sql, "<> ''") { + t.Errorf("ne blank on text gave %q — want the not-empty test", sql) + } +} + +// A numeric column is empty when NULL *or* zero, and that must be explicit: +// SQLite compares 0 against ” as false while MySQL calls it true, so one +// expression would answer two different questions depending on the backend. +func TestEmptyOnNumericCoversZeroAndNull(t *testing.T) { + sql, _, err := conditionSQL(Condition{Field: "freq_hz", Op: "empty"}) + if err != nil { + t.Fatalf("empty: %v", err) + } + if !strings.Contains(sql, "IS NULL") || !strings.Contains(sql, "= 0") { + t.Errorf("sql = %q — want both NULL and zero", sql) + } + // Text keeps the string test: '' is a real value there, 0 is not. + sql, _, _ = conditionSQL(Condition{Field: "name", Op: "empty"}) + if !strings.Contains(sql, "IFNULL") || strings.Contains(sql, "= 0") { + t.Errorf("text empty gave %q", sql) + } +} diff --git a/internal/qso/qso.go b/internal/qso/qso.go index 9dfe84d..b8002cf 100644 --- a/internal/qso/qso.go +++ b/internal/qso/qso.go @@ -1266,6 +1266,17 @@ var filterableColumns = map[string]bool{ // value compares on the date part (see conditionSQL) so day filters are exact. var dateColumns = map[string]bool{"qso_date": true, "qso_date_off": true} +// numericColumns are the filterable columns holding numbers rather than text. +// +// "Empty" means something different for them: NULL *or* zero. It has to be said +// explicitly because the two backends disagree — SQLite compares 0 against ” +// as false, MySQL calls it true — so one expression would quietly answer two +// different questions depending on where the logbook lives. +var numericColumns = map[string]bool{ + "freq_hz": true, "freq_rx_hz": true, "dxcc": true, "cqz": true, "ituz": true, + "srx": true, "stx": true, "tx_pwr": true, +} + // bareDateRe matches a plain calendar date with no time component. var bareDateRe = regexp.MustCompile(`^\d{4}-\d{2}-\d{2}$`) @@ -1366,6 +1377,18 @@ func conditionSQL(c Condition) (string, []any, error) { col = "substr(" + col + ",1,10)" v = strings.TrimSpace(v) } + // "equals nothing" and "is empty" are the same question, and SQL answers the + // first with nothing at all: a NULL never equals '', so a filter written that + // way returns zero rows and looks broken rather than wrong. Asking it in + // plain words is not a mistake worth punishing. + if strings.TrimSpace(v) == "" { + switch c.Op { + case "eq": + c.Op = "empty" + case "ne": + c.Op = "notempty" + } + } switch c.Op { case "eq": return col + " = ?", []any{v}, nil @@ -1416,8 +1439,18 @@ func conditionSQL(c Condition) (string, []any, error) { } return col + " IN (" + ph + ")", args, nil case "empty": + // A numeric column is empty when it is NULL *or* zero, and that has to be + // said explicitly: SQLite compares 0 against '' as false while MySQL calls + // it true, so IFNULL(col,'')='' quietly means different things on the two + // backends OpsLog supports. + if numericColumns[strings.ToLower(strings.TrimSpace(c.Field))] { + return "(" + col + " IS NULL OR " + col + " = 0)", nil, nil + } return "IFNULL(" + col + ",'') = ''", nil, nil case "notempty": + if numericColumns[strings.ToLower(strings.TrimSpace(c.Field))] { + return "(" + col + " IS NOT NULL AND " + col + " <> 0)", nil, nil + } return "IFNULL(" + col + ",'') <> ''", nil, nil default: return "", nil, fmt.Errorf("unknown operator %q", c.Op)