ADIF Multitool: a Swiss Army knife for ham radio logs
Tuesday, February 28th, 2023 09:47 pm![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
flwydI got involved in amateur radio in 2021 in part because it was a hobby I could do during pandemic stay-at-home periods that didn't involve staring at a computer screen. Getting a home station set up that's capable of transmitting on HF bands is kind of complicated, and involves making significant decisions like attaching masts to the roof and drilling holes through a wall. Fortunately, hams have come up with ways to encourage radio operation outdoors, with a temporary station setup. Parks on the Air awards fake Internet points to hams who set up in a state or national park and make contact with other amateur radio operators. I've found POTA to be a great motivation for getting out of the house and exploring some interesting places, while also improving radio skills and experimenting with emergency communication setup options.
After "activating" a park, the operator submits a log file of all their contacts to the Parks on the Air website so everyone can get credit. These log files use the ADIF format, commonly used for ham radio log interchange. ADIF uses a fairly simple format that predates general-purpose schemes like XML and JSON. Fields are encoded with a name, length, optional type, and value like <CALL:5:S>WT0RJ <NAME:6>Trevor <EOR>. These files are reasonably easy to read, but can be tricky to write by hand, since counting the number of characters in a string that's longer than about 7 is easy to get wrong. Hams typically use a logging program which can export an ADIF file for upload. But I don't really want to take a laptop on a camping trip, so I keep my Parks on the Air logs on paper in a notebook. Rather than transcribe that log directly into ADIF format I decided to enter them in Google Sheets, which makes it easy to fill in repeated fields like my park identifier, state, and the radio frequency where I called CQ. I then export the sheet to a CSV file.
There are a handful of CSV-to-ADIF converters that folks have published, but I wasn't particularly impressed by any of them. So of course I decided to make my own open source program. During the busy summer period that conversion was all it could do. Then after spending November organizing books and not spreading COVID and December staying up late writing Advent of Code, I was motivated to spend some time outside. Since Colorado got a lot of snow and cold air in January, "Go for a long hike" wasn't very attractive, but "brush the snow off the end of a picnic table and make ham radio contacts" was totally feasible. (Also "operate from the truck when the wind is too cold".) And since I wasn't camping, I had plenty of computer time to add more logging features in the evening, with park activations each weekend to provide insight on feature usefulness.
I decided to call the program ADIF Multitool, a "Swiss Army knife" that combines lots of single-use log processing tools into a convenient package. It's a command-line tool, written in the Go programming language, following the Unix pipeline philosophy. Several
Using Go for this tool has been a fairly positive experience. Go's I/O APIs provided useful levels of abstraction for handling the ADIF spec while also making it easy to provide my own fixed-string "filesystem" in test cases. Go strings are byte arrays, which is great for working with data interchange formats, while also providing robust Unicode support, which is great for working with user-provided text. The lightweight type system made it pretty easy to implement command line options like "a repeatable key=value flag that collects into a map." The field-tag approach to marshaling and serializing XML and JSON took some mental stretching to get used to (the simplistic examples in the package documentation contribute to this), but in practice they're quite nice and avoid the need to interact with a SAX or DOM parser. One of my main motivations for picking Go was the ability to generate code, which allowed me to convert the XML version of the ADIF specification into Go code for every data type, field, and enumeration value; when the next version of the spec is released I'll just need to run
After "activating" a park, the operator submits a log file of all their contacts to the Parks on the Air website so everyone can get credit. These log files use the ADIF format, commonly used for ham radio log interchange. ADIF uses a fairly simple format that predates general-purpose schemes like XML and JSON. Fields are encoded with a name, length, optional type, and value like <CALL:5:S>WT0RJ <NAME:6>Trevor <EOR>. These files are reasonably easy to read, but can be tricky to write by hand, since counting the number of characters in a string that's longer than about 7 is easy to get wrong. Hams typically use a logging program which can export an ADIF file for upload. But I don't really want to take a laptop on a camping trip, so I keep my Parks on the Air logs on paper in a notebook. Rather than transcribe that log directly into ADIF format I decided to enter them in Google Sheets, which makes it easy to fill in repeated fields like my park identifier, state, and the radio frequency where I called CQ. I then export the sheet to a CSV file.
There are a handful of CSV-to-ADIF converters that folks have published, but I wasn't particularly impressed by any of them. So of course I decided to make my own open source program. During the busy summer period that conversion was all it could do. Then after spending November organizing books and not spreading COVID and December staying up late writing Advent of Code, I was motivated to spend some time outside. Since Colorado got a lot of snow and cold air in January, "Go for a long hike" wasn't very attractive, but "brush the snow off the end of a picnic table and make ham radio contacts" was totally feasible. (Also "operate from the truck when the wind is too cold".) And since I wasn't camping, I had plenty of computer time to add more logging features in the evening, with park activations each weekend to provide insight on feature usefulness.
I decided to call the program ADIF Multitool, a "Swiss Army knife" that combines lots of single-use log processing tools into a convenient package. It's a command-line tool, written in the Go programming language, following the Unix pipeline philosophy. Several
adifmt commands can be chained together; each prints a log file to standard output which is then consumed as standard input by the next command. My latest POTA CSV file had just the fields that varied for each contact; I then added all the repetitive details about my own station, fixed data formats, inferred some field values from others, ensured all fields matched the specification, and saved it as an ADI file with this pipeline:adifmt edit --remove-blank --add qso_date=20230225 --add submode=USB
--add my_pota_ref=K-1213 --add my_lat=39.644388 --add my_lon=-104.846254 \
--add my_state=CO --add my_country="United States of America" \
--add operator=WT0RJ mylog.csv \
| adifmt fix \
| adifmt infer --fields station_callsign,mode,band \
--fields my_gridsquare,my_gridsquare_ext \
--fields pota_ref,sota_ref,my_sig_info,dxcc,my_dxcc \
| adifmt validate \
| adifmt save --overwrite-existing ~/WT0RJ@K-1213-20230225.adiUsing Go for this tool has been a fairly positive experience. Go's I/O APIs provided useful levels of abstraction for handling the ADIF spec while also making it easy to provide my own fixed-string "filesystem" in test cases. Go strings are byte arrays, which is great for working with data interchange formats, while also providing robust Unicode support, which is great for working with user-provided text. The lightweight type system made it pretty easy to implement command line options like "a repeatable key=value flag that collects into a map." The field-tag approach to marshaling and serializing XML and JSON took some mental stretching to get used to (the simplistic examples in the package documentation contribute to this), but in practice they're quite nice and avoid the need to interact with a SAX or DOM parser. One of my main motivations for picking Go was the ability to generate code, which allowed me to convert the XML version of the ADIF specification into Go code for every data type, field, and enumeration value; when the next version of the spec is released I'll just need to run
go generate again to add all of the additions and changes. Another expected benefit that I haven't tried yet: I can build releases for Windows and other operating systems I don't have access to, and distribute them without worrying that users might not have the latest version of Python or Java installed.
