The development image had Apache, Nginx, PHP-FPM, MySQL, Redis and a cron daemon in it, started by a shell script, and it was 1.4 gigabytes. Nobody built it deliberately; it grew one apt-get install at a time over eighteen months because adding a line to the existing Dockerfile was always easier than making a new one. Changing a PHP extension rebuilt everything and took six minutes.
The symptom
$ docker images app-dev
REPOSITORY TAG SIZE
app-dev latest 1.41GB
$ time docker build -t app-dev .
real 6m12.884s
$ docker history app-dev --format '{{.Size}}t{{.CreatedBy}}' | head -6
412MB RUN apt-get install -y mysql-server
188MB RUN apt-get install -y nginx apache2
141MB RUN docker-php-ext-install ...
94MB RUN apt-get install -y nodejs
0B COPY docker/start.sh /start.sh
0B CMD ["/start.sh"]The start.sh at the bottom was ninety lines of starting daemons in the right order with sleeps between them. It worked most of the time. When it did not, the container was up, the health check passed, and MySQL had not started.
Why it happens
A development environment grows by addition. Somebody needs Node for the asset build, so it goes in the image. Somebody needs the Apache configuration reproduced because production runs Apache on one host, so that goes in too. Each addition is smaller than the work of splitting, and nobody is ever assigned the split.
The structural problem underneath is that Docker’s model is one process per container and the shared image is fighting it. Everything unpleasant about the arrangement — the start script, the health check that lies, the six-minute rebuild, the inability to restart one service — follows from having six processes where the tooling expects one.
The fix
One file per service, and what each may contain
docker/
php/Dockerfile php-fpm, extensions, composer
nginx/Dockerfile nginx + the site config
apache/Dockerfile apache + mod_proxy_fcgi, for parity with one host
node/Dockerfile the asset build only — not in the runtime
# mysql, redis and the mail catcher are official images with
# configuration passed in. writing a Dockerfile for them is a
# sign of wanting something that belongs in a config file.The rule that keeps this from re-collapsing is that a Dockerfile may contain the software and its configuration, and nothing else. The moment one of them installs a second daemon “just for development”, the split is undone and the next person will add a third.
The PHP image, and the build dependencies that must not survive
FROM php:7.2-fpm-alpine
RUN apk add --no-cache --virtual .build-deps
$PHPIZE_DEPS libpng-dev icu-dev libzip-dev
&& apk add --no-cache libpng icu libzip
&& docker-php-ext-install -j"$(nproc)"
pdo_mysql gd intl zip opcache
&& pecl install redis
&& docker-php-ext-enable redis
&& apk del .build-deps
COPY --from=composer:1.7 /usr/bin/composer /usr/bin/composer
RUN addgroup -g 1000 app && adduser -u 1000 -G app -D app
USER app
WORKDIR /appThe virtual package group is the mechanism that matters. Compilers and header files are needed to build the extensions and are dead weight afterwards, and because they are installed and removed in a single RUN they never appear in a layer — deleting them in a later instruction would leave them in the image regardless, since layers are additive.
The runtime libraries have to be installed separately and kept, which is the part that catches people: removing .build-deps takes libpng-dev and leaves libpng, and forgetting the second apk add produces an image where gd is compiled and cannot load.
Copying Composer from its own image rather than piping an installer script through PHP is both shorter and verifiable — the tag pins the version and the layer is cached. It is also one fewer thing downloading and executing a script at build time.
Apache and nginx side by side
Two web server images in a development stack sounds like indulgence and is the opposite: production ran nginx on the new hosts and Apache on one legacy machine, and the differences had bitten twice.
# docker/nginx/Dockerfile
FROM nginx:1.15-alpine
COPY docker/nginx/site.conf /etc/nginx/conf.d/default.conf
RUN nginx -t # fail the BUILD, not the container
# docker/apache/Dockerfile
FROM httpd:2.4-alpine
COPY docker/apache/httpd.conf /usr/local/apache2/conf/httpd.conf
RUN httpd -tValidating the configuration during the build is a one-line habit worth having everywhere. A syntactically invalid config in an image is a container that exits immediately on start, and the message is buried in logs nobody is tailing yet; the same error at build time stops the build with the line number.
Both point at the same php-fpm container on port 9000, so the application code is identical and only the web server differs. That made the two Apache-specific bugs reproducible locally in about ten minutes, which is the entire justification for keeping the second image around.
Layer order, and the rebuild that takes four seconds
# rebuilds vendor on every source change
COPY . /app
RUN composer install --no-dev
# rebuilds vendor only when the dependencies change
COPY composer.json composer.lock /app/
RUN composer install --no-dev --no-scripts --no-autoloader
COPY . /app
RUN composer dump-autoload --optimizeThe cache is invalidated by the first changed file, so anything that changes often belongs as late as possible. Splitting the install into dependencies-then-autoloader exists because the packages do not need the source and the autoloader does; --no-scripts in the first step is what stops a post-install hook reaching for a file that has not been copied yet.
$ time docker build -t app-php docker/php
real 0m4.118s # source change only
$ touch composer.lock && time docker build -t app-php docker/php
real 1m38.204s # dependency change
# and the .dockerignore that made the context transferable
$ du -sh .
412M .
$ docker build . 2>&1 | head -1
Sending build context to Docker daemon 11.4MBExcluding .git alone took the build context from 340 megabytes to 11, and that transfer happens before any instruction runs. There is a correctness angle as well: a COPY . . without a .dockerignore pulls in the local .env and bakes development credentials into an image that gets pushed to a registry.
The node image that must not be in the runtime
The asset build needs Node and the running application does not, and keeping them in one image is how a production PHP container ends up with ninety megabytes of JavaScript tooling and a package manager in it.
# docker/node/Dockerfile — development only
FROM node:10-alpine
WORKDIR /app
CMD ["npm", "run", "watch"]
# and for the production image, a multi-stage build:
#
# FROM node:10-alpine AS assets
# COPY package*.json ./
# RUN npm ci
# COPY resources ./resources
# RUN npm run production
#
# FROM app-php:latest
# COPY --from=assets /app/public/build /app/public/buildThe multi-stage form is what keeps the toolchain out of the artefact: the second stage copies the compiled files and nothing else, so Node never appears in the image that gets deployed. Locally the watcher runs as its own service instead, which means a change to a stylesheet rebuilds in a second without touching the PHP container at all.
npm ci rather than npm install is the line that matters in the build stage. It installs exactly the lock file and fails when the lock and the manifest disagree, where install quietly resolves something new — and a build that can produce different assets from the same commit is not a build.
Making the stack start in a predictable state
The old start script sequenced daemons with sleeps. Compose cannot promise ordering across containers and should not be asked to, so the application has to tolerate a dependency being briefly absent.
services:
php:
restart: unless-stopped # the actual dependency mechanism
depends_on: [mysql, redis] # start order only, not readiness
mysql:
image: mariadb:10.3
healthcheck:
test: ["CMD", "mysqladmin", "ping", "-h", "localhost"]
interval: 10s
start_period: 30s # grace before failures countA PHP container that cannot reach MariaDB exits, is restarted, and succeeds on the third attempt fifteen seconds later. That is uglier than waiting and is the behaviour you want, because the database will also go away at some point when nothing is starting — and a stack that only survives a clean boot is a stack that has not been tested against the failure it will actually see.
start_period is the flag that stops a forty-second database initialisation being reported as unhealthy. Without it, every normal cold start looks like an incident to anything watching health status, which trains people to ignore it.
The uid problem, which is the one that wastes an afternoon
On macOS the file sharing layer papers over ownership. On Linux it does not, and a container writing as uid 1000 into a bind mount owned by uid 1001 gets permission denied — and everybody has a different uid.
# docker-compose.override.yml — local only
services:
php:
build:
context: .
dockerfile: docker/php/Dockerfile
args:
UID: ${UID:-1000}
GID: ${GID:-1000}
# in the Dockerfile
# ARG UID=1000
# ARG GID=1000
# RUN apk add --no-cache shadow
# && usermod -u $UID app && groupmod -g $GID app
# make setup:
# printf 'UID=%snGID=%sn' "$(id -u)" "$(id -g)" > .env.dockerThis is an hour of everyone’s life, once per team, and it never appears on macOS — so it is entirely possible to ship a stack that works for half the people and fails silently for the other half with a permissions error in a log they have not learned to read yet. Generating the two variables in a setup command means nobody has to know why they exist.
Warning
The alternative some teams take — running the container as root and living with root-owned files appearing in the repository — trades a one-time build problem for a permanent one. Files created by a build step then cannot be deleted without sudo, which surfaces during an unrelated git clean a month later.
Verifying it worked
$ docker images | grep app-
app-php latest 184MB
app-nginx latest 22MB
app-apache latest 58MB
# was one image at 1.41GB
$ time docker-compose build
real 2m04.118s # cold, all services
$ touch src/Kernel.php && time docker-compose build php
real 0m4.402s
# the test that matters: a machine that has never seen this project
$ git clone ... && cd app && make setup && make up
$ curl -fsS localhost:8080/health
okThe clean-machine test is the only verification that means anything, because everything else is being run by somebody whose environment already has the missing piece. Doing it on a colleague’s laptop rather than your own is what catches the step that lives in your shell history and not in the repository.
Restarting one service without the others is the property the split was for, and it is worth demonstrating once so that people believe it — docker-compose restart php taking two seconds while nginx keeps its connections is a different working experience from the six-minute rebuild.
What this costs
More files, and a build that has to be understood before it can be changed. One Dockerfile was legible to anybody; four plus a compose file plus an override is a small system, and somebody joining the team now has to learn it before they can add a PHP extension. Writing a short README next to the docker/ directory saying which file owns what is worth more than it sounds, because the alternative is people guessing and putting things in the wrong one.
The honest counter-argument is that a single image is genuinely simpler for a project that will never have more than one developer, and the split pays for itself through onboarding and through the ability to change one thing at a time. On a two-person project it is probably not worth an afternoon. On a team of six it paid for itself in the first week, mostly in rebuild time that nobody had been counting because it was spent waiting rather than working.