The template ended with thirty script tags and one stylesheet link, and the page could not do anything until the last of them had arrived. Most were under 2 KB. The problem was never the bytes.
The symptom
requests 31
transferred 412 KB
DOMContentLoaded 1.94 s
load 2.61 s
# the six largest files account for 340 KB of the 412
# the other twenty-five average 2.9 KB eachTwenty-five small files, each one a DNS-free but round-trip-expensive fetch against a browser willing to open six connections per host. Nearly two seconds before the DOM was ready, and the transfer itself accounts for a small fraction of it — the rest is request overhead and the parser blocking on each tag in turn.
Why it happens
Every plugin was added the way its README said, one tag at a time, over about three years. Nobody ever added thirty tags; somebody added the twenty-fourth. There was no moment at which the decision looked wrong, and no file anywhere recording what depended on what — the load order was whatever the accumulated sequence of edits had produced, and it worked, which is the least useful kind of correct.
The second half is that a script tag without defer blocks the parser. Thirty of them means the browser stops, fetches, executes and resumes thirty times, and the connection limit turns that into a queue rather than a burst.
It is worth ruling out the cheaper answers before building anything, because two of them look like they should work. Moving the assets to a second hostname raises the connection limit and buys perhaps 300 ms, at the cost of a second DNS lookup and a second TLS handshake. Putting a CDN in front changes where the round trips terminate but not how many there are. Both are real improvements to the wrong quantity: the problem is the count, and only concatenation reduces the count.
The fix
Write the order down, then automate it
The build cannot be written until the order is known, and working it out is most of the job. The listing below is the first time the dependency order existed as an artefact rather than as a sequence of accidents, and two entries in it turned out to be loaded twice.
module.exports = function (grunt) {
grunt.initConfig({
concat: {
options: { separator: ';n', sourceMap: true },
app: {
src: [
'src/js/vendor/jquery-1.11.1.js',
'src/js/vendor/bootstrap.js',
'src/js/vendor/jquery.validate.js',
'src/js/vendor/slick.js',
'src/js/app/namespace.js',
'src/js/app/cart.js',
'src/js/app/filters.js',
'src/js/app/*.js'
],
dest: 'build/app.js'
}
},
uglify: {
options: { sourceMap: true, sourceMapIn: 'build/app.js.map' },
app: { src: 'build/app.js', dest: 'build/app.min.js' }
}
});
grunt.registerTask('build', ['concat', 'uglify', 'filerev', 'manifest']);
};The wildcard at the end is deliberate and the three explicit entries before it are the point: the files with an ordering constraint are named, and everything else is alphabetical and does not care. A build that names all thirty files is a build nobody updates.
Warning
The separator: ';n' is not decoration. One vendor file in this set had no trailing semicolon, and concatenating it directly onto the next produced a syntax error that only appeared in the built bundle — which is to say, only in production.
Source maps, so a stack trace still names a file
Minified code turns every error report into app.min.js line 1 column 24817. A source map is a second file mapping those positions back to the original, generated by the same step that destroyed the information.
The subtlety is that both steps have to participate. Concatenation produces a map from the bundle to the sources; minification then needs to be told about that map with sourceMapIn, or its own map points at the concatenated intermediate and the chain breaks silently. A map that resolves to the wrong file is worse than no map, because it is believed.
Cache busting in the filename, not the query string
A bundle should be cached for a year, which means the URL has to change when the contents do. The two options are a query string — app.min.js?v=1411 — and a hash in the filename. They are not equivalent.
// build/rev-manifest.json is written by the build:
// { "app.min.js": "a41c9f2e.app.min.js" }
function asset($file)
{
static $manifest;
if ($manifest === null) {
$manifest = json_decode(file_get_contents(BUILD . '/rev-manifest.json'), true);
}
return '/build/' . (isset($manifest[$file]) ? $manifest[$file] : $file);
}Several proxies and a number of CDNs refuse to cache a URL with a query string at all, or strip the string and cache the bare path — which is the worst outcome available, since two different builds then share a cache entry. A filename hash has neither problem, and it makes deploys atomic: the old file is still there, so a page served from cache mid-deploy still finds the assets it asked for.
Verifying it worked
before after
requests 31 4
transferred 412 KB 198 KB
DOMContentLoaded 1.94 s 0.61 s
load 2.61 s 0.94 sFour requests: the document, one bundle, one stylesheet and a font. The transferred figure halved because minification and gzip finally had something worth compressing — thirty small files compress far worse individually than one large one, since each carries its own dictionary.
The number worth watching afterwards is not any of those. It is the count on a repeat visit, which is one, because the bundle is now immutable and cached for a year.
What this costs
There is now a step between editing a file and seeing the change, and that is a genuine loss. A watcher makes it nearly invisible, but “nearly” is doing work in that sentence, and everyone forgets to start it at least once and spends ten minutes debugging a change that was never built.
watch: {
js: {
files: ['src/js/**/*.js'],
// concat only: uglify on every keystroke is 4 seconds nobody has
tasks: ['concat'],
options: { spawn: false }
},
css: {
files: ['src/less/**/*.less'],
tasks: ['less']
}
}The watcher deliberately runs a shorter task list than the build: concatenation is 200 ms and minification is four seconds, and the development page loads the unminified bundle, so there is nothing to be gained by paying for the second one on every save. That does mean the two paths differ, which is worth being uneasy about — a bug that only exists after minification will not appear locally, and the only defence is that the deploy runs the real build and somebody looks at the result.
The build also has to run on deploy, which means the deploy now needs Node installed, and the version of Node on the server is a thing that can differ from the version on a laptop. Checking the built files into the repository avoids that and creates a different problem: every branch conflicts in app.min.js. Building on deploy is the better trade, but it is a trade.