fix: ADIF import silently dropped a file with no header
Reported with screenshots: a dozen records lifted out of wsjtx_log.adi into a new .adi imported as "0 imported, 0 ignored, 0 total". The parser only emitted records after <EOH>, and a hand-assembled file has no header — so the whole file was read as header text and thrown away. The count says the file was empty; there is nothing on screen to suggest a missing tag, which is why this was reported as "import does nothing". An <eor> is proof that a record ended, header or no header. Fields are now collected always and DISCARDED at <eoh> — that tag is exactly the statement "everything before this was the header", so real headers stay out of the data just as reliably, and a headerless file keeps its first record instead of losing it to the same rule. An existing test asserted the old behaviour. It encoded a defensible reading of the spec, but the file it rejects is one operators really make and the rejection is silent, so the test is rewritten with the evidence that changed it rather than deleted.
This commit is contained in:
+23
-7
@@ -22,9 +22,11 @@ import (
|
||||
// Record is a single ADIF record. Keys are lowercased field names.
|
||||
type Record map[string]string
|
||||
|
||||
// Parse reads an ADI stream and invokes fn for each record (after <EOH>).
|
||||
// Parse reads an ADI stream and invokes fn for each record.
|
||||
// Returning a non-nil error from fn stops parsing and is propagated.
|
||||
// The header (text before <EOH>) is silently discarded.
|
||||
//
|
||||
// The header (text before <EOH>) is silently discarded. A file with NO header is
|
||||
// read as records from the first tag — hand-assembled files often have none.
|
||||
func Parse(r io.Reader, fn func(Record) error) error {
|
||||
return parseWith(r, nil, fn)
|
||||
}
|
||||
@@ -44,7 +46,6 @@ func parseWith(r io.Reader, decodeValue func([]byte) string, fn func(Record) err
|
||||
br := bufio.NewReaderSize(r, 64*1024)
|
||||
|
||||
rec := Record{}
|
||||
headerDone := false
|
||||
|
||||
for {
|
||||
// Seek next '<'. Bytes before it are either header text or
|
||||
@@ -65,11 +66,19 @@ func parseWith(r io.Reader, decodeValue func([]byte) string, fn func(Record) err
|
||||
name, length := parseSpec(spec)
|
||||
switch name {
|
||||
case "eoh":
|
||||
headerDone = true
|
||||
rec = Record{}
|
||||
rec = Record{} // everything collected so far was the header
|
||||
continue
|
||||
case "eor":
|
||||
if headerDone && len(rec) > 0 {
|
||||
// An <eor> proves we are in the record section, whether or not an <eoh>
|
||||
// came first. Files pasted together by hand — a few lines lifted out of
|
||||
// wsjtx_log.adi into a new file — have no header at all, and requiring
|
||||
// one made every record invisible: the import reported 0 imported,
|
||||
// 0 ignored, 0 total, which reads as "the file is empty" rather than
|
||||
// "there was no header".
|
||||
//
|
||||
// A real header cannot contain <eor>, so accepting it costs nothing for
|
||||
// well-formed files: theirs ends at <eoh> long before any record.
|
||||
if len(rec) > 0 {
|
||||
if err := fn(rec); err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -95,7 +104,14 @@ func parseWith(r io.Reader, decodeValue func([]byte) string, fn func(Record) err
|
||||
if decodeValue == nil && !utf8.Valid(val) {
|
||||
val = extendToRunes(br, val, length)
|
||||
}
|
||||
if headerDone && name != "" {
|
||||
// Fields are collected ALWAYS, and thrown away at <eoh>.
|
||||
//
|
||||
// Collecting them only after the header meant a headerless file lost its
|
||||
// FIRST record whole: its fields were discarded as header text, and the
|
||||
// <eor> that followed found nothing to emit. Discarding at <eoh> instead
|
||||
// keeps real headers out of the data just as well — that tag is exactly
|
||||
// what says "everything before this was the header".
|
||||
if name != "" {
|
||||
if decodeValue != nil {
|
||||
rec[name] = decodeValue(val)
|
||||
} else {
|
||||
|
||||
Reference in New Issue
Block a user