The block editor fetches patterns from a remote directory, which is a network request on an admin page load and a dependency on somebody else’s uptime.
// disable the remote fetch entirely
add_filter( 'should_load_remote_block_patterns', '__return_false' );
// and remove the core patterns as well, on a site with a
// strict design system
remove_theme_support( 'core-block-patterns' );
On a site behind a firewall the fetch fails and adds several seconds to every editor load while it times out, which presents as a slow admin rather than a network problem. Disabling it is also a reasonable default on any site with its own pattern library, because forty generic patterns among nine useful ones is a worse inserter than nine alone.