PHP-FPM Pool Exhaustion: Fix Your Website Performance Issues

You can have great CPU performance and ample storage capacity yet suffer from extremely slow loading speeds. The reason might be PHP-FPM pool exhaustion. New dynamic requests will have to wait once all of the PHP-FPM child processes are busy processing the requests. The bigger the queue, the longer it will take to load pages. Users might face timeout or gateway errors.

The issue mentioned above is called PHP-FPM pool exhaustion. An unexpected increase in traffic is one of the most common triggers of such problems. Nevertheless, traffic is not the only thing that may contribute to the problem.

The answer in most cases is not to increment pm.max_children. This is because pm.max_children specifies the maximum number of children PHP-FPM can have. There is no check on memory and workload. It will just end up creating more overload. The correct approach is to determine what causes the exhaustion of the workers.

What Is PHP-FPM Pool Exhaustion?

PHP-FPM (FastCGI Process Manager) is a process manager for PHP which runs pools of child processes to execute PHP code in web servers like Nginx and Apache.

Think of a PHP-FPM pool as the number of counters at a grocery store checkout point. Say you have 20 workers. They are all busy executing a PHP request and then the next request will have to wait until one of the workers is free.

pm.max_children is the value that sets the maximum number of child processes. For pm = static, it is the number of children to create. It is the maximum number of children that can be spawned for dynamic and on-demand. It is referred to as the limit of simultaneous request processing in PHP documentation.

Exhaustion of the pool happens when the demand for PHP processing exceeds the capacity of the pool for such an amount of time that requests start queuing up.

How Do You Know PHP-FPM is Exhausted?

Don’t think that a slow website necessarily means PHP-FPM is exhausted. Examine the pool’s status.

Here are the useful statistics from the status page of PHP-FPM:

• Active processes indicates the number of workers processing requests now.
• Idle processes indicates the number of workers awaiting requests.
• Total processes indicates the number of workers working right now.
• “Listen queue” refers to the number of requests awaiting the availability of a process.
• “max listen queue” refers to the maximum number of requests queued since the inception of the pool.
• The “max active processes” shows the maximum number of active processes at any given moment.
• max children reached means that the number of processes has reached the configured limit.
• slow requests indicates the number of requests exceeding the configured slowlog threshold.

1. Determine That the Pool is a Bottleneck

One needs to look into the FPM status page when the actual slowdown occurs. The limit won’t help with the current performance issues in case of an empty listening queue and idle workers.

One needs to compare the information provided by the FPM status page with the server metrics. If there is CPU overload, almost all RAM is used up, and the database is overloaded, PHP-FPM might just be a place where other bottlenecks manifest themselves.

2. Find out What Requests are Keeping the Workers Occupied

A PHP-FPM worker remains busy until either its request completes or the worker process dies. One of the most effective ways is to reduce the request time in PHP.

A dedicated log file for this is available in PHP-FPM.

You can define timeout like this:

request_slowlog_timeout = 5s

slowlog = /var/log/php/php-fpm-slow.log

Once a request takes longer than what you have defined, PHP-FPM will write a backtrace of that request to the slow log. It can help in identifying the bottleneck in that request.

Five seconds is just an example. Identify requests which are:

• Expensive database queries
• Slow plugins
• Heavy WooCommerce queries
• External API calls
• Images or files processing
• Unoptimized custom PHP code
• Heavy non-cached application queries
• Long administrative requests

Resolving a query which takes five seconds may be more beneficial than just adding more PHP workers.

3. Raise pm.max_children Only When the Server Can Take It

Wait until it is confirmed by monitorin that PHP-FPM consistently hits its child process limit and there are enough free resources left on the server. Only then can raising the pm.max_children limit can be justified.

For example:

pm.max_children = 30

30 is not a good limit for all servers. It is an illustrative number.

Every PHP-FPM child uses up memory.

There is no universal formula. Measure the real memory consumption of PHP-FPM on your server under realistic traffic conditions.

Then raise the worker limit slowly and monitor:

• RAM usage
• Swap usage
• CPU usage
• FPM queue length
• Latency
• Error rates

4. Apply request_terminate_timeout to Runaway Requests

There are certain requests which shouldn’t be allowed to consume a PHP-FPM process forever.

PHP-FPM offers:

request_terminate_timeout = 60s

This defines the maximum time in seconds PHP-FPM allows to keep running a particular request before terminating its worker process. PHP documentation describes it as an extra protection measure in case PHP’s max_execution_time isn’t working.

It depends on the nature of your application.

A site which genuinely performs some large file uploading or other operations which take a considerable amount of time may require a longer timeout. Normal page requests may require a much smaller one.

5. Address Slow Database Queries

A PHP-FPM process can stay busy as PHP awaits completion of a database query. It is a situation where you need to take into account that your server has a PHP-FPM issue when, in reality, the problem is caused by the database.

Look for:

• Slow SQL queries
• Lack of indexes or bad ones
• Plugins running many queries
• Big or inefficient queries
• DB server CPU/memory overload
• Too big a WordPress database

6. Minimize API Calls from Third Parties

Third-party services can cause the same kind of issue.

Consider a PHP request making a call to the API of some third-party service before rendering a page. The PHP-FPM process stays busy during that time if the response from the API takes eight seconds. Your pool can be quickly filled up if there are several simultaneous users initiating such requests.

Where possible:

• Cache API responses.
• Don’t call external services per page request.
• Set reasonable connection and response timeouts.
• Shift noncritical work to background processing.
• Be smart with retries for failed requests.

7. Leverage Page Caching to Offload Requests From PHP

One of the most powerful methods for reducing the PHP-FPM load is caching.

The request might not need to go through WordPress and PHP at all when it is possible to serve a public page using a page cache. It doesn’t use a PHP-FPM worker.

Use cases for page caching include:

• Blogs
• Documentation websites
• News websites
• Marketing pages
• Public WordPress pages

The idea here is simple. Do not send the request to PHP when the response can be served from cache safely.

8. Verify Proper pm Configuration

The process manager module in PHP-FPM is capable of using static, dynamic and ondemand modes. They differ in their functionality.

Static mode: the pool keeps the number of children set in pm.max_children parameter.

Dynamic mode: PHP-FPM changes the number of children according to the set parameters and uses pm.start_servers, pm.min_spare_servers and pm.max_spare_servers.

On-demand mode: new children get created on incoming requests and can later be killed if they are idle.

A website that has high traffic all the time can use its own number of workers. A website that gets only a little traffic does not have to have numerous idle PHP processes.

9. Use pm.max_requests to Recycle Workers

Here is also an additional feature of PHP-FPM:

pm.max_requests = 500

PHP in particular mentions that this is useful to counter memory leaks in external libraries. It helps in situations where the worker slowly accumulates memory throughout its lifetime. But it is important to note that pm.max_requests is not a direct solution to the problem of the exhaustion of the PHP-FPM pool. It does not speed up slow requests nor increase the capacity of the pool.

10. Monitor the Pool Even When It Is Under Load

A good twist happens when the main FPM pool is overloaded and cannot process its status request.

PHP-FPM offers pm.status_listen option in such cases. It will start a separate listener which will be capable of processing FPM status requests independently of the main pool. This is mentioned explicitly by PHP as an option when the main pool is busy handling long requests.

For example, your config may have something like this:

pm.status_listen = 127.0.0.1:9001

Note that the exact address and web-server configuration depend on your environment.

11. Keep the FPM Listener Private

The PHP-FPM FastCGI listener should not be exposed on an untrusted network. The PHP manual indicates that a client which connects to an exposed FastCGI listener may control the configuration settings for the request and may run arbitrary code.

A Unix socket can be used between the web server and PHP-FPM in case of a standard single server environment. Restrict access properly with PHP-FPM’s listen.allowed_clients where necessary in case of TCP.

Limit access to the FPM status page to internal and known client IP addresses only. 

12. Check RAM Before Adding More Workers

RAM is among the key factors to consider when tuning PHP-FPM. Consider that the server has 2GB of RAM and PHP workers use the bulk of that already. If you increase pm.max_children, it will be possible to spawn even more PHP processes, which in turn will have a chance to consume more memory.

This can result in the following issues:

• Swapping
• Latency
• Process killing
• Database instability
• Out-of-memory situations
• Worse performance in general

Always consider the amount of available RAM as the hard limit while sizing your pool.

13. Remember About CPU Saturation

Not only is memory the limit.

PHP workers consume much CPU. Increasing the number of PHP workers in that case will make more processes compete for the same CPU power.

Watch your CPU utilization along with the number of active processes in FPM. If CPU is the bottleneck, then optimize expensive PHP calls or increase CPU capacity, rather than just increasing pm.max_children.

14. Consider 502 and 504 Errors as Symptoms

PHP-FPM exhaustion may cause gateway errors. A 502 or 504 code does not necessarily indicate that the FPM pool is exhausted.

The causes may also be:

• Stopping or crashing of PHP-FPM
• Improper FastCGI configuration
• Unavailability of the Unix socket file
• Network issues
• Website timeouts in Nginx or Apache
• Application faults
• Exhaustion of other resources

Check logs of Nginx/Apache and PHP-FPM as well as system statistics. The error message must be the starting point rather than the conclusion about pm.max_children limitation.

Final Thoughts

In the case of PHP-FPM pool exhaustion, we have the capacity problem, but the problem’s root might not be the insufficient size of PHP-FPM pool. The website may use up all its workers in cases when there is an unexpected increase in traffic. In addition, the website may drain all workers due to an operation performed by some plugin, a long-running database query, a slow external API or just a hanging request. It is not right to raise pm.max_children without any other action in response to this kind of problem.

Let us consider facts. Check the FPM status page. Analyze the listen queue. Find slow requests. Check CPU usage and RAM and database performance. Then solve the problem. In some cases, this will require increasing pm.max_children. Some problems will need better caching or optimized queries. You will need to set the timeout for the hanging request in some situations. And some servers will require additional resources.

FAQs