The default pool ships with pm.max_children = 50, which on a small server is not a limit so much as an invitation to swap. The number should come from two measurements: how much memory one worker of your application actually uses, and how much memory is left after everything else has taken its share.
# average resident size of a live worker, in MB
ps -ylC php5-fpm --sort:rss | awk 'NR>1 {s+=$8; n++} END {print s/n/1024}'Measure it under traffic, not at rest — a worker that has served a few hundred requests is considerably larger than a freshly forked one. Divide the memory you have left by that figure and use the result. Exceeding it does not degrade gracefully: the machine starts swapping, and a swapping server is far slower than one that simply makes requests wait in the listen queue.