With prefork and mod_php, every Apache child has the interpreter and all of its extensions linked into it, regardless of what that child is currently doing. Forty workers at roughly 35 MB resident is the whole of a small VPS, and most of them are serving a stylesheet, a logo, or nothing at all while a keep-alive connection stays open.
$ ps -ylC apache2 --sort:rss | awk 'NR>1 { rss += $8; n++ } END { printf "%d workers, %.1f MB averagen", n, rss/n/1024 }'
41 workers, 34.7 MB average
$ free -m
total used free shared buffers cached
Mem: 512 487 24 0 4 61
-/+ buffers/cache: 421 90
Swap: 1024 212 812
That box is swapping while the CPU is largely idle, which is the signature of this arrangement rather than of a traffic problem. There are two independent ways out and they can be taken in either order. Serving static files from something that is not an Apache-with-PHP worker — nginx in front on port 80, Apache moved to 8080 — removes most of the processes without touching the application at all, and it is the change that can be made in an afternoon and reversed in ten minutes. Moving PHP out of the worker entirely, with mod_fcgid or PHP-FPM, is the larger one: Apache workers become small, and the pool of interpreters is sized separately, which is the real gain because connection handling and script execution want completely different concurrency numbers. What leaving mod_php costs is .htaccess and per-directory php_value. Both stop working, every setting has to move into a pool config or an ini_set(), and finding all of them in a tree that has accumulated them for five years is the bulk of the work.