fix: convert MySQL logbook to utf8mb4 (fixes 1366 on Cyrillic/Polish)

A QSO with a non-Latin character (Cyrillic Я = \xD0\xAF, Polish ł, …) failed to
save on the shared MySQL backend with "Error 1366: Incorrect string value …
for column 'qso'.'address'". Cause: the database had been pre-created by a
hosting panel (o2switch/cPanel) as latin1, so OpsLog's CREATE DATABASE IF NOT
EXISTS … utf8mb4 was a no-op and every table inherited latin1 — which can't
store those letters, even over a utf8mb4 connection.

OpenMySQL now runs ensureMySQLUTF8MB4 after migrations: ALTER DATABASE to
utf8mb4 (for future tables) + CONVERT any table with a non-utf8mb4 text column
via an information_schema probe. Idempotent and cheap on a healthy DB (the probe
returns nothing → no ALTERs); best-effort so a charset issue never blocks
logbook access. SQLite is unaffected.
This commit is contained in:
2026-07-23 23:59:47 +02:00
parent 1a0a349fb3
commit 75510a1161
2 changed files with 51 additions and 0 deletions
+2
View File
@@ -3,6 +3,7 @@
"version": "0.20.12",
"date": "2026-07-23",
"en": [
"Fixed 'Incorrect string value' (MySQL error 1366) when a QSO contained non-Latin characters — Cyrillic (Я), Polish ł, etc. — e.g. updating from QRZ. It happened when the shared MySQL database had been pre-created by the hosting panel as latin1, so OpsLog's tables couldn't store those letters. OpsLog now converts the database and its tables to utf8mb4 on connect (once, automatically), and everything stores correctly. Local SQLite logbooks were never affected.",
"NET Control: the on-air list now follows a mic-pass order you control instead of log time. A pinned '#' column numbers the round, ⬆⬇ buttons move the selected station up/down the queue, new check-ins join the end, and 'Log & end' advances to the next in line. The order is remembered per net. Header sorting is disabled on this list so a click can't shuffle the round.",
"The CW decoder (RX audio → text) is back, with a rebuilt decoding engine. The first version struggled on real signals; the new one adds a windowed tone detector, an adaptive dB envelope with noise squelch, two-way debouncing (a brief fade inside a dash no longer shatters it into dots), per-character dit/dah classification (even the first letter of an over decodes at any speed), and gap-fed speed tracking. It rides QSB, ignores QRM on other pitches, follows 1240 WPM and sloppy hand keying — all proven by a synthetic-signal test suite. Tools → CW decoder (or the ear button): decodes while the mode is CW, with WPM/pitch/level display, pitch lock (follows the FlexRadio CW pitch automatically), and click-a-word to fill the callsign. Weak signals in heavy QRM remain hard — that's CW Skimmer territory — but clean-to-moderate signals now decode solidly.",
"The CW Keyer settings panel is now fully translated (it had stayed in English), and the long serial-engine explanation paragraph was removed to keep it clean.",
@@ -11,6 +12,7 @@
"Fixed the colour theme sometimes resetting to light after an update/relaunch: an update can clear the WebView's localStorage, and the fallback that restores the theme from the local settings database gave up after ~2.4s — occasionally too soon during the brief startup window before that store is ready. It now keeps retrying until the store actually answers, so a dark theme is reliably restored."
],
"fr": [
"Correction de « Incorrect string value » (erreur MySQL 1366) quand un QSO contenait des caractères non latins — cyrillique (Я), polonais ł, etc. — p. ex. lors d'une mise à jour depuis QRZ. Ça arrivait quand la base MySQL partagée avait été pré-créée en latin1 par le panel d'hébergement, empêchant les tables d'OpsLog de stocker ces lettres. OpsLog convertit désormais la base et ses tables en utf8mb4 à la connexion (une seule fois, automatiquement), et tout s'enregistre correctement. Les logbooks SQLite locaux n'étaient pas concernés.",
"NET Control : la liste on-air suit maintenant un ordre de passage du micro que tu contrôles, au lieu de l'heure de log. Une colonne « # » épinglée numérote le tour, les boutons ⬆⬇ montent/descendent la station sélectionnée dans la file, les nouveaux arrivants rejoignent la fin, et « Logger & terminer » passe au suivant. L'ordre est mémorisé par NET. Le tri par en-tête est désactivé sur cette liste pour qu'un clic ne bouscule pas le tour.",
"Le décodeur CW (audio RX → texte) est de retour, avec un moteur de décodage reconstruit. La première version peinait sur les vrais signaux ; la nouvelle ajoute un détecteur de tonalité fenêtré, une enveloppe dB adaptative avec squelch de bruit, un anti-rebond bilatéral (un bref fading dans un trait ne le brise plus en points), une classification point/trait par caractère (même la première lettre d'un over se décode à n'importe quelle vitesse) et un suivi de vitesse nourri par les espaces. Il tient le QSB, ignore le QRM sur d'autres tonalités, suit de 12 à 40 WPM et la manipulation humaine approximative — le tout prouvé par une suite de tests sur signaux synthétiques. Outils → Décodeur CW (ou le bouton oreille) : décode quand le mode est CW, avec affichage WPM/tonalité/niveau, verrouillage de tonalité (suit automatiquement le pitch CW du FlexRadio) et clic-sur-un-mot pour remplir l'indicatif. Les signaux faibles dans un fort QRM restent difficiles — c'est le territoire de CW Skimmer — mais les signaux propres à moyens se décodent maintenant solidement.",
"Le panneau de réglages du Manipulateur CW est maintenant entièrement traduit (il était resté en anglais), et le long paragraphe d'explication du moteur série a été retiré pour l'alléger.",
+49
View File
@@ -207,9 +207,58 @@ func OpenMySQL(c MySQLConfig) (*sql.DB, error) {
Dialect = "sqlite" // migration failed; we'll fall back to SQLite
return nil, err
}
// Guarantee the schema can store non-Latin text (Cyrillic, Polish ł, …).
// Best-effort: never block access to the logbook over the charset probe.
if err := ensureMySQLUTF8MB4(conn, name); err != nil {
fmt.Printf("OpsLog: utf8mb4 conversion: %v\n", err)
}
return conn, nil
}
// ensureMySQLUTF8MB4 makes the database default and every OpsLog table utf8mb4,
// converting any that isn't. It repairs databases pre-created as latin1 or
// utf8mb3 by a hosting panel (cPanel/o2switch default), where OpsLog's
// "CREATE DATABASE IF NOT EXISTS … utf8mb4" was a no-op and the tables inherited
// a charset that can't store non-Latin callsigns/names/QTHs — the MySQL 1366
// "Incorrect string value: '\xD0\xAF'" (Cyrillic Я) seen updating a QSO from QRZ.
//
// Idempotent and cheap on a healthy database: one information_schema probe finds
// the tables with a non-utf8mb4 text column, and only those are converted (none
// on an already-correct DB). Conversion is safe for the wide qso table because
// the bulk of its columns are off-page TEXT (see the row-size note above); only
// the ~20 indexed VARCHARs sit in-row, well under InnoDB's 65535-byte limit even
// at 4 bytes/char.
func ensureMySQLUTF8MB4(conn *sql.DB, dbName string) error {
// Fix the database default so any FUTURE table is utf8mb4 too.
_, _ = conn.Exec("ALTER DATABASE `" + dbName + "` CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci")
rows, err := conn.Query(`
SELECT DISTINCT TABLE_NAME
FROM information_schema.COLUMNS
WHERE TABLE_SCHEMA = ?
AND CHARACTER_SET_NAME IS NOT NULL
AND CHARACTER_SET_NAME <> 'utf8mb4'`, dbName)
if err != nil {
return fmt.Errorf("probe charsets: %w", err)
}
var tables []string
for rows.Next() {
var t string
if err := rows.Scan(&t); err == nil && validDBIdent(t) {
tables = append(tables, t)
}
}
rows.Close()
var firstErr error
for _, t := range tables {
if _, err := conn.Exec("ALTER TABLE `" + t + "` CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci"); err != nil && firstErr == nil {
firstErr = fmt.Errorf("convert %s: %w", t, err)
}
}
return firstErr
}
// isFreshMySQL reports whether the database has no OpsLog schema yet (no qso
// table), so the baseline fast-path applies. A partially-migrated database is
// NOT fresh and goes through the incremental migrator.