The asset pipeline Laravel 5.4 shipped

The build broke on a Node upgrade for the third time and nobody could say why, because the failure was four layers down: Elixir calling gulp calling a plugin calling a library that had dropped support for the Node version installed. 5.4 replaces Elixir with Mix, which is a thin wrapper over webpack, and the migration is mostly deleting.

The symptom

$ npm run dev

module.js:471
    throw err;
Error: Cannot find module 'graceful-fs'
    at Function.Module._resolveFilename
    at Object.<anonymous> (node_modules/gulp-util/lib/log.js:2:12)
    at node_modules/laravel-elixir/dist/index.js:9:1

The stack trace passes through three packages before reaching anything anyone wrote. The gulpfile was fourteen lines and none of them were the problem.

Why it happens

Elixir was a fluent API over gulp, and gulp is a task runner rather than a bundler — so every capability came from a gulp plugin, each of which wrapped a library, each with its own version constraints. The abstraction was pleasant and had no escape hatch: when a plugin did not do what was needed, the answer was to write gulp directly and lose the Elixir configuration entirely.

webpack solves a different problem. It is a module bundler, so the dependency graph is the primary concept and the transformations hang off it — which is the right shape for an application whose JavaScript imports things.

The fix

The migration, which is mostly removal

// gulpfile.js — before
const elixir = require('laravel-elixir');
require('laravel-elixir-vue-2');

elixir(mix => {
    mix.sass('app.scss')
       .webpack('app.js')
       .version(['css/app.css', 'js/app.js']);
});

// webpack.mix.js — after
const mix = require('laravel-mix');

mix.js('resources/assets/js/app.js', 'public/js')
   .sass('resources/assets/sass/app.scss', 'public/css')
   .version();

The Vue extension is gone because Mix handles single-file components out of the box. version() takes no arguments because it versions whatever was compiled. The package.json lost eleven gulp plugins.

$ git diff --stat package.json
 package.json | 24 +++--------------------

$ du -sh node_modules
182M    node_modules      # was 341M

Versioning, and why the query string was wrong

Elixir’s versioning renamed files with a hash and wrote a manifest. Plenty of projects had skipped it and used a query string instead, which looks equivalent and is not.

<!-- the version a lot of sites ship -->
<link rel="stylesheet" href="/css/app.css?v={{ config('app.asset_version') }}">

<!-- what Mix produces -->
<link rel="stylesheet" href="{{ mix('/css/app.css') }}">
<!-- /css/app.css?id=8a7f1c2e9b — read from mix-manifest.json -->

Some proxies and CDNs ignore the query string when constructing a cache key, so a query-string buster changes nothing for the caches between you and the visitor. A changed filename cannot be ignored by anything. Mix uses a query string too — which works because the value is a content hash rather than a deploy counter, so a proxy that strips it still has a correct entry for the unchanged file.

The escape hatch

The reason this migration is worth doing is not that Mix is nicer than Elixir. It is that when the wrapper is not enough, the thing underneath is a configuration format with a decade of documentation rather than a chain of plugins.

mix.webpackConfig({
    resolve: {
        alias: { '@': path.resolve(__dirname, 'resources/assets/js') }
    },
    module: {
        rules: [
            { test: /.svg$/, use: 'svg-inline-loader' }
        ]
    }
});

That merges into the generated configuration rather than replacing it, so the Mix defaults survive. Anything not expressible there can take over entirely by exporting a full config — an exit that Elixir did not have.

Verifying it worked

$ npm run production

       Asset      Size  Chunks
  /js/app.js   184 kB       0  [emitted]
 /css/app.css  41.2 kB      0  [emitted]

$ diff <(md5sum public/js/app.js) old-app.js.md5
# identical output, different toolchain

$ nvm use 8 && npm ci && npm run production
# builds — which the old pipeline did not

Byte-identical bundles is the assertion that matters: the pipeline changed and the artefact did not, which makes the deploy safe to do on an ordinary afternoon.

The build has to move into the pipeline

Elixir projects frequently committed the compiled assets, because the build was fragile enough that nobody wanted it on a server. Mix removes the excuse and the mix() helper removes the option — it throws when the manifest is missing, so a deploy that has not built cannot serve a page at all.

build:
  stage: build
  image: node:8
  script:
    - npm ci
    - npm run production
  artifacts:
    paths: [public/js, public/css, public/mix-manifest.json]
    expire_in: 1 week

npm ci rather than npm install is the line that matters: it installs exactly the lock file and fails if the lock and the manifest disagree, where install quietly resolves something new. A build that can produce different bundles from the same commit is not a build.

Removing the compiled assets from version control is the other half, and it wants a .gitignore entry and a deliberate deletion in the same commit — otherwise the stale committed copies sit in the tree and are served whenever someone forgets to build, which is the failure this arrangement exists to make impossible.

What this costs

The escape hatch is webpack configuration, which is a real skill and a less forgiving one than gulp. A team that could previously get by with the fluent API now has a category of problem — a loader ordering, a resolve path, a chunk boundary — that requires understanding the bundler. That is a fair trade for having the option at all, and it is not free.

The build is also slower. gulp streamed files; webpack builds a dependency graph, which is more work and produces a better result. On a large application the difference is thirty seconds against eight, and watch mode narrows it considerably.