Why boxes and glue?

Why boxes and glue exists

boxes and glue is TeX’s typesetting engine without TeX: the algorithms that make TeX output beautiful, reimplemented in Go as a library you can embed, script, and ship as a single binary.

This page explains how it came about, and why it looks the way it does.

It started with LuaTeX

For many years I have been developing the speedata Publisher, a database publishing system that creates catalogs, data sheets, and price lists from XML data. Its typesetting backend is LuaTeX, and LuaTeX is excellent: Knuth’s line breaking, a precise box model, and Lua access to almost everything inside the engine.

But running a product on top of it has a price:

  • I can’t fix it myself. LuaTeX is a large C code base. When something goes wrong deep inside the engine, I can report it, but I can’t realistically debug and fix it on my own.
  • I depend on upstream. What LuaTeX does next, and when, is decided elsewhere.
  • Deployment is a collection of parts. The Publisher ships LuaTeX, a number of libraries, and helper programs. A Go library is loaded into LuaTeX at runtime, and when something breaks on a customer’s Windows machine, it is very hard to reproduce and understand.
  • The font model is dated. In TeX, a font is loaded at a fixed size, so every size needs its own font definition.
  • Much of it I don’t need. LuaTeX carries a whole TeX system with it. The Publisher uses only a fraction.
  • Lua is the only way in. Lua is a fine extension language, but a large application written entirely in Lua is hard to maintain.

What I wanted was simple to state: one binary that I understand and control.

Algorithms, not macros

TeX is really two things: a set of brilliant algorithms, and a macro language with backslash commands on top of them. The Publisher never used the macro language. Everything ran through Lua.

So the question became: what if you keep only the algorithms? Knuth’s paragraph builder, hyphenation, and the model of a page as a list of nodes, where every piece is either a box (something with a size, such as a glyph or an image) or glue (stretchable space between boxes). That model gave the project its name.

Dropping the macro language made everything else possible. The engine is a normal Go library with a normal API. There is no macro expansion to reason about, and you can use the parts you need and ignore the rest.

One library pulled in the next

A typesetting engine in pure Go needs everything underneath it in pure Go as well, otherwise you are back to linking C libraries and fighting deployment. So one library led to the next, not because it was fun (though it was), but because the engine needed them:

  • textshape for text shaping, a Go port of HarfBuzz, so that ligatures, kerning, variable fonts, and complex scripts like Arabic work,
  • baseline-pdf for writing PDF,
  • hyphenation patterns, SVG, images, PDF import, and eventually htmlbag for HTML and CSS.

The result cross-compiles to every platform Go supports, and the output is reproducible.

Why Lua, again

A library alone is not enough. People also need a way to use it without writing Go. I experimented for quite a while:

  • gopher-lua implements Lua 5.1, with a programming interface that feels unusual.
  • Shopify’s go-lua implements Lua 5.2 and lacks parts of the standard library, for example Lua’s pattern matching.
  • JavaScript (via the Go interpreter Otto) didn’t feel right for this kind of work, and it lacks much of modern JavaScript.

None of these was something I wanted to offer users. So I built a Lua 5.4 VM on top of Shopify’s go-lua, and glu on top of that.

glu is where the circle closes. It turns Markdown or HTML into PDF with sensible defaults, but it also opens up the engine to Lua, the way LuaTeX does: callbacks, direct access to nodes, fonts, and pages. You don’t hit a wall when your document needs something unusual.

What this means for you

  • You get TeX-quality typesetting without installing a TeX distribution.
  • You get one binary, or one Go dependency, that runs the same on Linux, macOS, and Windows.
  • You can go as deep as you need: Markdown with glu, HTML and CSS with bagme, or every node yourself with the Go API.

What you don’t get is TeX itself. If you have existing LaTeX documents or rely on LaTeX packages, a TeX system is still the right tool. boxes and glue is for the cases where you generate documents from data or text and want the engine in your own hands.