Markup

I.e. writing device-independent documents with complex formatting in ‘plain text’^[1].

Current choices

roff is pain

Technically, it can be used to typeset the same text to paper, to a terminal, or to a web page, though there are too many different macro packages (with extremely cryptic macro names as a rule), the processing pipeline is wild (many preprocessors for various features—​very UNIX), and the syntax serves the machine better than the human.

HTML and XML belong on the backend

SGML-derived languages such as HTML and XML are very strong in all the ways they can be processed, but for general use, they’re overly verbose: <p>a&mdash;b</p> where a--b would do. Also, SGML comments are annoying: <!-- -- --> is invalid.

Markdown sucks

Out of all options for a lightweight markup language, why did programmers need to choose such a majestic fuck-up for the ultimate answer?

• There are too many incompatible variants. CommonMark is merely another one on the list. Actually, its specification is a great argument against Markdown.

• The syntax for bold and italic text is nonsensical: * is supposed to mean bold, _ is supposed to be italic or at least underlined. Instead, Markdown mixes all meanings together.

• As pertains to programmers: the _ often matches in the middle of a word, depending on the variant, so you may need to write a\_long\_identifier.

• List syntax is optimised for reading on an unusually configured teletype^[2], not for writing. And at least in Obsidian, you can’t add a hard break and expect the following line to align, which might be a side effect of the syntax.

• There are too many ways to do anything.

AsciiDoc is okay, I guess

AsciiDoc maps to DocBook XML, which is a pretty extensive semantic markup for publications, and comes with many backends for various kinds of output. AsciiDoc predates Markdown. It has its own warts:

• While the original Python AsciiDoc has been basically obsoleted, it still forms a second standard next to the current Asciidoctor. The syntax shifted between the two—​Git still uses the old one. And if a standard ever appears, it will be slightly incompatible still.

  (Other implementations target Asciidoctor—​while they may differ in syntax, the differences are categorically bugs or omissions. There aren’t many. The one in Go which I’ve been using is pretty broken, and the project has been archived. Most recently, a partial one in Haskell has popped up.)

• List syntax is optimised for writing, not reading. Nested levels need ** etc. prefixes. If you don’t like the way that looks, combining * with - can get you two levels and that’s it. More importantly, list continuations can’t be indented at all. But see further below for why this may not matter at all.

• The mapping to DocBook XML is incomplete. For example, you can’t specify a figure with multiple images; this needs a workaround.

• With all of its features, it gets fairly complex. And even with compatibility mode turned off, there are too many ways of doing things—​mainly because of syntax imported from Markdown.

That said, for anything nontrivial it’s a clear improvement over Markdown, and it’s decently well supported: GitHub and GitLab render it, and Asciidoctor packages are widely available.

The Eclipse Foundation has an AsciiDoc Working Group. It hasn’t achieved much of anything in years, yet I’m looking forward to them finally releasing a real specification. Partly because the current Ruby, Javascript, and JVM options are all needlessly heavy-weight for a document processor. Once the format is formalised, compatible implementations should easily fall out of the specification, and making them won’t feel dirty. Until then, this markup appears to be stuck with broken/incomplete naïve attempts. (Asciidoctor’s parsing is slightly broken by definition, because it makes multiple passes over HTML.)

Besides my webpage and basic project documentation, I particularly like AsciiDoc for writing UNIX manual pages. I’ve made my own simplified processor for the relevant language subset as a hyperportable AWK script.

Setext vs ATX headings

In other words, are headings supposed to be underlined or prefixed? For a long time, I was on team Setext, because it looks infinitely better in monospace, and the heading level is more obvious. Compare:

 Title            = Title
 =====            Paragraph.
 Paragraph.

 Subtitle         == Subtitle
 --------         Paragraph.
 Paragraph.

 Subsubtitle      === Subsubtitle
 ~~~~~~~~~~~      Paragraph.
 Paragraph.

Because AsciiDoc restricts Setext headings to its ‘compatibility mode’, that made me a proponent of keeping the older format alive. Now I am on team ATX, because with variable pitch fonts, underline matching is meaningless busywork. Live preview in editors takes care of presentation, keeping the ugly prefixes out of sight.

Custom markups

It’s never been simpler to define your own format that matches your needs. The machines themselves can generate syntax files, entire editors with Vim emulation that understand the format, and any other tooling you might think of. You lose interoperability, which doesn’t always matter in the first place. So when I decided to retire Vim, which I had been using for my trivially formatted personal notes, I thought for a day, defined a syntax for the new age, and had the machine make me an editor with live preview. Old notes got converted. It didn’t take much time, and it works miraculously well.

Once you get rid of the shackles imposed by requiring other people to be able to view and edit the files as intended, you can add cool stuff like elastic tabstops: voilà, tables with close to no syntax. If you want numbers right-aligned, add autodetection. Busywork gone.

Rendering

Ultimately, you might stop to consider whether you need to stay with ‘plain text’ in the first place. Though in my experience, the invisible state of WYSIWYG, and the cursor positioning issues of such editors seemed to add more friction than magical in-band ornaments. Also, ‘plain text’ version control stays simple, if this matters to you.

With live preview, which I’ve already mentioned above, you can get the advantages of both. This feature was seemingly properly pioneered in Typora; now you can find it in Obsidian^[3] and many other editors. It’s hard for me to imagine editing without it anymore.

Live preview has some interesting consequences. It favours simple formatting, thus ATX headings, AsciiDoc-style list bullets, and doing away with hard wrapping. The smaller the ornaments, and the less you pre-format, the better it works—​it decides the visual layout by itself, dynamically.

Still, can we do better?

If you zoom out a bit, you will notice two areas where markup starts failing: complex layouts, and dynamic content. There, block-based editing will serve you better. However, even there, markup can find its use on a small scale, such as within inline runs.

1. In reality, approximating WYSIWYG using a stream of Unicode characters with line feeds and perhaps tabulation. Often outright assuming a teletype.

2. 4-space tabs where 8 is the norm.

3. I spent close to a year with Obsidian. It is overall a decent piece of software, despite its papercuts. Nowadays, you can even find plugins to teach it AsciiDoc.