Extracting a service without a rewrite

Reporting was 14% of the codebase, had one entry point, and owned four of the six hours the test suite took. Nothing about it needed to deploy with the shop, and everything about it did — a change to a product page waited on a reporting test that had nothing to do with it.

The symptom

$ vendor/bin/phpunit --testdox-summary

Orders           412 tests   00:41
Catalogue        388 tests   00:36
Checkout         290 tests   01:12
Reporting        144 tests   04:02   <-- 66% of the wall time

Total            1234 tests  06:31

A hundred and forty-four tests taking four hours, because each one built a year of fixture data to aggregate. Every deploy of anything paid for it.

Why it happens

Shared deployment is shared risk regardless of how well the code is separated. The reporting namespace had no incoming dependencies and it did not matter: it was in the same repository, the same pipeline and the same release, so its slowness and its failures were everyone’s.

The fix

The seam first, in place

Nothing moves until there is exactly one way in. The first change is entirely within the monolith and ships on its own.

interface Reports
{
    /** @return array */
    public function salesByBrand(DateTimeInterface $from, DateTimeInterface $to);

    /** @return array */
    public function stockMovement(DateTimeInterface $from, DateTimeInterface $to);
}

// today: the existing code, unchanged, behind the interface
final class LocalReports implements Reports { /* ... */ }

Two methods. The controllers that had been calling eleven different classes now call these, and the fact that a service is coming is invisible. This step is worth doing even if the extraction never happens, which is the test of whether it is the right step.

The database is the hard part

The reporting code reads eleven tables the shop owns. There is no version of this where that stops being true on the day of the split, and pretending otherwise is how these projects fail.

-- the honest interim: a read-only user, and it is written down
CREATE USER 'reports'@'%' IDENTIFIED BY '...';
GRANT SELECT ON shop.orders       TO 'reports'@'%';
GRANT SELECT ON shop.order_lines  TO 'reports'@'%';
GRANT SELECT ON shop.products     TO 'reports'@'%';
-- no INSERT, no UPDATE, no DELETE, ever

This is shared-database coupling and it is deliberate. What the grant buys is that the coupling is now enumerated — eleven tables, read-only, in a file — rather than implied by whatever the ORM happens to touch. A schema change to any of them is a coordinated release, and now everyone can see which ones.

The path off it is a nightly copy into the service’s own schema, which is the right end state for reporting because the data does not need to be current to the second. That was scheduled as a later piece of work rather than a prerequisite, which is what kept this deliverable.

Route gradually rather than switching

final class RoutedReports implements Reports
{
    public function salesByBrand(DateTimeInterface $from, DateTimeInterface $to)
    {
        if (! $this->features->enabled('reports.remote')) {
            return $this->local->salesByBrand($from, $to);
        }

        try {
            return $this->remote->salesByBrand($from, $to);
        } catch (TransportException $e) {
            $this->log->warning('reports fell back to local', array('e' => $e));
            return $this->local->salesByBrand($from, $to);
        }
    }
}

A flag, a fallback and a log line. The old implementation stays until the new one has been serving for a month, which costs a little duplication and buys the ability to turn the whole thing off from a configuration change rather than a deploy.

The comparison mode ran first: call both, return the local answer, log any difference. Two discrepancies surfaced in the first week, both timezone handling at month boundaries, and both would have been reported by finance rather than by a log if the switch had been immediate.

Verifying it worked

# the shop's pipeline, after
Orders           412 tests   00:41
Catalogue        388 tests   00:36
Checkout         290 tests   01:12

Total            1090 tests  02:29

# and the reporting service's own pipeline, in parallel
Reporting        144 tests   03:51

Six and a half hours to two and a half on the path that blocks a product change. The reporting tests did not get faster — they stopped being in the way, which was the actual complaint.

The comparison run, before the switch

A feature flag makes the switch reversible and does nothing to say whether the new implementation is correct. Running both and comparing does, and it costs one release.

public function salesByBrand(DateTimeInterface $from, DateTimeInterface $to)
{
    $local = $this->local->salesByBrand($from, $to);

    if ($this->features->enabled('reports.compare')) {
        try {
            $remote = $this->remote->salesByBrand($from, $to);

            if ($remote !== $local) {
                $this->log->warning('report mismatch', array(
                    'method' => 'salesByBrand',
                    'args'   => array($from->format('c'), $to->format('c')),
                ));
            }
        } catch (Exception $e) {
            $this->log->warning('comparison failed', array('e' => $e->getMessage()));
        }
    }

    return $local;   // still the source of truth
}

Two discrepancies surfaced in the first week, both timezone handling at month boundaries — the old code used the server timezone and the new one used UTC, so a report run at 00:30 on the first of the month disagreed with itself by a day of sales.

That would have been found by finance rather than by a log if the switch had been immediate, which is a materially different conversation. The comparison mode came out after a fortnight; the cost was one wasted call per report and it bought the only evidence that mattered.

What this costs

A function call became a network call, and network calls fail. The fallback covers it today and the fallback is temporary by design, so there is a date after which a reporting outage is a reporting outage. That is a real reduction in reliability traded for independent deployment, and it should be stated rather than discovered.

There are now two repositories, two pipelines, two sets of dependencies to keep patched and two things to be on call for. For one service extracted from one monolith that overhead is easily worth it; the arithmetic changes at the fifth service, and the answer is not obviously the same.

And the shared schema is still there. It is documented, constrained and read-only, which makes it a known liability rather than an unknown one — but a service that cannot change its own storage is only partly extracted, and calling it done at this point would be the mistake this whole exercise was meant to avoid.