A design token pipeline that outlived its designer

A design token pipeline built in 2022, four stages, no documentation, and the person who wrote it left in November. The next colour change took three days and touched five files, two of which turned out not to matter.

The symptom

$ ls tools/tokens/
  build.mjs  transform.mjs  formats.mjs  resolve.mjs
  tokens.source.json  aliases.json  overrides.json

$ npm run tokens
  ✓ resolved 188 tokens
  ✓ transformed
  ✓ wrote 4 files

$ git log --oneline tools/tokens/ | wc -l
3
# three commits, all from 2022, all by one person

the colour change, as performed:

  day 1  edit tokens.source.json. no effect.
  day 1  discover overrides.json. edit it. partial
         effect — the button changed, the badge did not.
  day 2  discover aliases.json, which maps semantic
         names to primitives. edit it. the badge changes
         and a chart breaks.
  day 2  discover that the chart reads a fifth file,
         generated by a stage nobody had run.
  day 3  run everything, verify, ship.

Why it happens

A pipeline built by one person over a fortnight encodes their model of the problem, and the model is the documentation. When they leave, what remains is four files that each do something and no statement of what the whole is for.

The fix

Reading it, and writing down what each stage does

  resolve.mjs     reads tokens.source.json and
                  aliases.json, resolves {reference}
                  syntax to values. produces a flat map.

  transform.mjs   applies unit conversions: px to rem,
                  hex to hsl for the ones that need
                  alpha variants.

  formats.mjs     writes four outputs — a CSS file, a
                  JSON file for the charts, a Sass
                  partial, and a JS module.

  overrides.json  applied AFTER transform, replacing
                  values wholesale. undocumented, and
                  the reason day 1 had no effect.

  build.mjs       calls the other three in order.

Two hours of reading produced five paragraphs that should have taken five minutes to write in 2022. The overrides file is the one that caused the three days — it exists to hold values that could not be expressed in the source format, and there are four of them, all from a single afternoon in 2022.

The stage that did nothing

// transform.mjs, the px-to-rem conversion
export function toRem(value) {
  if (typeof value === 'string' && value.endsWith('px')) {
    return `${parseFloat(value) / 16}rem`
  }
  return value
}

// tokens.source.json, every spacing value:
//   "s": "1rem", "m": "1.5rem", "l": "2.5rem"
//
// there are no px values. the function has run 188
// times per build since 2022 and changed nothing.

The source file had used pixels in 2022 and was converted to rem units by hand three months later, which made the transform redundant without anybody removing it. Dead code in a build pipeline is invisible because the output is correct — the only signal is that the function is never observably doing anything.

Two sources of truth that had drifted

$ jq -r '.color | to_entries[] | "(.key) (.value)"' 
    tokens.source.json | sort > /tmp/source
$ jq -r '.color | to_entries[] | "(.key) (.value)"' 
    overrides.json | sort > /tmp/override
$ join /tmp/source /tmp/override
  brand    #1f6feb  #2178ec
  danger   #d1242f  #d1242f
  warning  #9a6700  #bf8700

# three overridden, two of which differ from the source.
# which is correct? the design file says #2178ec.

The overrides had been the truth for two years and the source file had been edited since, by somebody assuming it was. Resolving this meant finding the design file, which was the only artefact anybody trusted — and the answer was that the overrides were right, which means every edit to the source since 2022 had been silently discarded.

Collapsing four stages into two

// tools/tokens/build.mjs — the whole pipeline
import { readFile, writeFile } from 'node:fs/promises'

const tokens = resolve(
  JSON.parse(await readFile('tokens/tokens.json', 'utf8'))
)

await Promise.all([
  writeFile('assets/css/tokens.css', toCss(tokens)),
  writeFile('assets/js/tokens.js', toJs(tokens)),
  writeFile('theme.json', toThemeJson(tokens)),
])

// resolve() is 40 lines. the three writers are 20 each.
// the transform stage and the overrides file are gone.

what was removed:

  transform.mjs        dead
  overrides.json       merged into the source, which is
                       now the only file anybody edits
  the Sass partial     nothing had imported it since
                       the Sass build was removed in 2023
  aliases.json         merged — the {reference} syntax
                       works within one file

412 lines → 120.

A test on the output rather than the transforms

test('the generated CSS matches the snapshot', async () => {
  await build()

  await expect(await readFile('assets/css/tokens.css', 'utf8'))
    .toMatchFileSnapshot('./__snapshots__/tokens.css')
})

// which caught, on the first run: the new writer emitted
// custom properties in object-key order, and one
// referenced another declared below it.

Unit tests on each transform would have passed and said nothing about the composition, which is where every bug in this pipeline had been. A snapshot of the final artefact is a weak assertion and it is the only one that covers the whole thing — and unlike most snapshots, this file is genuinely reviewed when it changes.

Verifying it worked

# the generated output, before and after the rewrite
$ diff <(git show HEAD~1:assets/css/tokens.css) 
       assets/css/tokens.css
  # 3 lines: the three colours that had been overridden,
  # now correct in the source. intentional.

$ npm run tokens && git diff --exit-code assets/ theme.json
  # clean — the committed output matches a fresh build

# the actual test
# somebody who had never touched it changed a colour:
#   files edited: 1
#   time: 6 minutes
#   incorrect first attempts: 0

Six minutes by somebody who had never opened the directory is the measurement that matters, and it is the only one that would have told us whether the rewrite achieved anything. Diffing the generated output before and after is what makes a rewrite of a build pipeline reviewable at all.

What this costs

A simpler pipeline that still nobody else has run. One person understood the old one and one person now understands the new one, which is the same failure with better odds — a hundred and twenty lines is readable in an afternoon where four hundred and twelve was not.

The three colours that had been wrong in the source since 2022 also mean that anybody who read the source file to find a brand colour got the wrong answer for two years. That is the cost of two sources of truth and it is not recoverable — there is no way to know what was built on the strength of a value that was never actually used.