Weekly report 20-36

This one is certainly useful, if only for corporates who tend to have such needs in the first place. Here the biggest deficiencies were the potential to mangle unusual documents, and the inability to change the size of the window reserved for signature data at runtime. Both were relatively easy to fix.

Then I spent a lot of hours making a test suite. Certificates are always such a pain to deal with. I figured out how to use openssl to create a custom CA with a related S/MIME leaf certificate, then imported the CA into a custom NSS database for use in pdfsig, just so that it could say the signer is trusted. Of course, I had wasted about three hours on the -addext switch making a CA that couldn’t be paired with its leaf certificate before I took a step back and made a minimal reproducer that helped me find out about it.

The effort wasn’t in vain, as I discovered a slight issue in the Go port, where its infamous range operator iterated through xref chunks in a randomised order. The fix for that even ended up cleaning up the code, which is a win-win.

Status: Finished now

A few years back, I’ve successfully managed to bend GNU Readline to do the most unusual of things, including an IRC client with switchable buffers and a multiline prompt, reminding the uninformed of a Curses interface. Since I don’t exactly like Readline because of its messy internals, as well as its strong copyleft license, I later tried to port this to the BSD Editline library, also available as a separate port called libedit. Even though I managed to make it mostly functional in both of my applications, some problems still remain, such as history not working correctly, and recently I’ve noticed that it sometimes misrenders.

While checking to see if this ‘frontend’ is still usable, I’ve discovered that one thing has finally started to work there: attributed prompts. With one caveat, it seemed that the escape character can’t be at the end of the prompt, or the part that it encloses would get ignored, and the user’s input would be rendered bold as well. So I swapped the attribute reset sequence with the trailing space character, which fixed it.

Of course, something different had to break in exchange. I mostly used the wide character interface of the library but in one place I dared to call the multibyte el_gets function in order to force it to redraw, and this started to crash. Easily enough, I just had to replace it with el_wgets.

As soon as I was about to try out some libedit changes in the IRC client, I was faced with an unpleasant surprise: it failed to build! It turned out that my Arch (btw I use Arch) system only had the ‘lua’ package installed, and it got upgraded to Lua 5.4, which has an incompatible coroutine API. I quickly patched it up without greater testing, though it appears to work.

Status: Forever a personal tool, a GUI rewrite is necessary

While fixing up my other program that sports a fancy command line, I noticed one thing: when I ran an external editor (VIM in my case) on the input, the editor would automatically insert a newline at the end, and the cursor would end up on a blank line. This looked rather inappropriate, so I’ve resolved it by trimming them off.

Then there was one more issue to address: when you pipe something into a libedit program, which is a valid use case for json-rpc-shell, it gets silently switched into a special mode that doesn’t work together with the asynchronous ‘unbuffered’ mode, and my program got stuck in a busy loop where the library would never invoke the carriage return handler, nor would it return anything sane from el_wgets. It was easy enough to prevent this, using the buffered mode instead, and reading entire lines at once.

Moving on from obvious deficiencies, it was time to make the defaults saner: it used to be that you had to run with -a and -p options on in order for bare method calls to not be sent as notifications, and to pretty-print the result. Not anymore.

Finally, as I wanted to extend the test server program with a new method that would return some nontrivial data, which ended up being ‘date’ for lack of better ideas, I noticed an old bug on each side of the connection: the server didn’t send the ‘id’ field in its reply, and the client didn’t bother to check it.

Status: Finished now

My trusty companion, an offline dictionary. I finally arrived at the idea to scratch one itch of mine: I didn’t have an English thesaurus in it yet, and I tend to need this a lot as of late, for my perfectionist writing. I went through my collection of StarDict dictionaries of questionable legal status (you can easily find these online), looking for a thesaurus that could be transformed so that my measly program would be able to display it, and went for the Collins Thesaurus.

Which lead me to the immediate problem: how do I transform all of its XDXF entries into the required plain-text format? I had to make a tool. Since I didn’t want to make it too specific to the task at hand, it had to be able to filter entries through a helper program. And since it costs a lot of time to launch a process in Linux and I didn’t want the operation to take several minutes, it had to be streamed through as a whole. Also, seeing as entries often contain newlines, they had to be separated from each other by something like the NUL character, which is actually the internal separator for all textual StarDict entries.

Perl, my favourite text transformation tool, has a switch for exactly that situation. So I dusted off my knowledge of the somewhat ugly GLib ecosystem and wrote an utility that first writes all textual entries to a filter program written in Perl, then progressively copies the entire StarDict dictionary to a new one, replacing textual entries with whatever it finds in the filter’s output. What then surprised me was that it only took a split second to fix up the thesaurus, and so I could iterate fast on this lovely piece of line noise:

There was one more thing to do. I already had ten dictionaries loaded at all times in my sdtui, and this being the eleventh, I ran out of Alt-number bindings to jump to it. I ended up adding Ctrl-PageUp/Down for that purpose, as most people know them from web browsers, as well as Ctrl-Left/Right, which someone claimed worked in irssi.

Status: Forever a personal tool, a GUI rewrite is necessary

Poor documentation I think is the biggest defect of all my projects. I used to generate man pages with GNU help2man, though that just isn’t enough. The user often has no way of learning how to use the program at all.

I still don’t want to write roff by hand, I hate scdoc syntax for the same reasons I hate Markdown, DocBook is annoying XML and complex machinery around it, xmltoman is still annoying XML and also GPL-licensed. However AsciiDoc can do this job and I had already been invested in it, so that is what what I’ve chosen to write extended new manuals in. My Gitea can even render it directly as HTML.

It took me a while to do this properly but now, at last, json-rpc-shell as well as pdf-simple-sign come equipped with good manual pages! They’re a joy for me to look at.