I’d like to propose adding a new configuration directive, tentatively called pm.max_memory, to PHP-FPM. This directive would allow administrators to specify a per-child memory usage limit, after which the PHP-FPM child process would gracefully exit or restart.
Background and Rationale
In many production environments, especially those handling long-running or memory-intensive processes, memory leaks (whether in extensions, libraries, or user code) can slowly accumulate. We already have tools like pm.max_requests that recycle processes after a certain number of requests, but there are scenarios in which memory usage might skyrocket within fewer requests—or memory might slowly climb in a way that doesn’t align well with request counts alone.
A setting like pm.max_memory could:
Mitigate slow leaks: By enforcing a hard memory cap at the process level, ensuring no single worker balloons in size over time.
Provide more granular control: If certain scripts or pools are known to be memory-intensive, an admin could set a memory limit that’s appropriate for that usage pattern, rather than tying it only to request counts.
Complement system-level controls: While cgroups and container memory limits exist, a built-in FPM mechanism can be friendlier than a system OOM kill, which might be abrupt and less predictable.
Proposed Behavior- A new config directive, pm.max_memory, which, if set to a value above 0, indicates the maximum amount of RAM (in bytes) a PHP-FPM worker process is allowed to use (resident set size or a similar metric).
After handling a request, each worker would check its own memory usage. If it exceeds the threshold, it would gracefully exit or be terminated by the master process before picking up a new request.
By default, this setting could be disabled (pm.max_memory = 0), so it does not affect existing installations.
Implementation Details and Challenges
I am not proposing to implement this feature myself—I’m more of a sysadmin and don’t have the necessary C knowledge to patch php-src. However, here are some thoughts on what the implementation might involve:
Measuring memory usage: Likely via getrusage(), mallinfo() (on some platforms), or reading from /proc/self/statm on Linux.
Graceful shutdown logic: Ensuring no ongoing requests are abruptly killed, or at least minimizing the chance of partial request handling.
Platform differences: Some OSes might provide different APIs for measuring process memory usage. We’d need consistent cross-platform behavior or documented differences.
Interaction with pm.max_requests: If both are set, a worker would exit on whichever limit it hits first (memory or request count).
Alternatives- Using pm.max_requests: Currently the main workaround to mitigate leaks, but it’s less precise and can’t handle large memory spikes that happen quickly.
System-level OOM or cgroups: This approach can kill the entire pool or container, which is often more disruptive than recycling a single worker.
Request for Feedback
I’m posting this proposal to gather feedback on feasibility and interest. If there’s enough support, I’d be happy to collaborate with anyone who can handle the technical side—writing up a formal RFC on the wiki or working on a patch. If there’s a consensus that this is better handled elsewhere (system-level or container-level controls), or that pm.max_requests is sufficient, please let me know your thoughts.
Key questions:
Would a built-in memory cap be beneficial for a significant subset of PHP-FPM users?
Are there any major technical hurdles or objections to this approach?
Does anyone have suggestions on how best to measure memory usage accurately and portably across different platforms?
Thank you for reading and considering this idea. I look forward to hearing your insights and am happy to clarify or discuss any aspect of this proposal further.
On Tue, Jan 28, 2025 at 2:17 AM Arkadiy Kulev <eth@ethaniel.com> wrote:
Hello everyone,
I’d like to propose adding a new configuration directive, tentatively called pm.max_memory, to PHP-FPM. This directive would allow administrators to specify a per-child memory usage limit, after which the PHP-FPM child process would gracefully exit or restart.
I think it’s reasonable as an additional protection and a bit nicer than pm.max_requests. However, implementation might be tricky.
Implementation Details and Challenges
I am not proposing to implement this feature myself—I’m more of a sysadmin and don’t have the necessary C knowledge to patch php-src. However, here are some thoughts on what the implementation might involve:
Measuring memory usage: Likely via getrusage(), mallinfo() (on some platforms), or reading from /proc/self/statm on Linux.
This wouldn’t really work because FPM does not control the script during execution and would have to check it out after each allocation which is not really viable.
System-level OOM or cgroups: This approach can kill the entire pool or container, which is often more disruptive than recycling a single worker.
I think this is really the only way that I can see to make it work reliably. We have got already an old feature request for that: https://bugs.php.net/bug.php?id=70605 . There was even a PR https://github.com/php/php-src/pull/2440 but it was using unmaintained lib and cgroup v1. We should use something cgroup v2 based but it is not a small job to add it.
Anyway you can create a GitHub feature request for this so it doesn’t get forgotten and maybe one day there will a better PR that can be accepted.
This wouldn’t really work because FPM does not control the script during execution and would have to check it out after each allocation which is not really viable.
Thanks for the feedback! I agree that monitoring memory usage after each allocation would be infeasible. However, my suggestion was actually to check memory usage only once per request, specifically at request shutdown, when FPM regains control and before assigning another request to that worker.
We already have hooks for request startup and request shutdown in FPM. Could we simply insert a memory check at the request shutdown stage—where the worker is returning control to the FPM master process—before picking up a new request?
Just to be clear, “memory_limit” helps kill runaway scripts mid-request. By contrast, the newly proposed pm.max_memory is meant to catch processes with a slow leak across multiple requests. We only need to check at the end of each request, which is presumably when the worker returns control to FPM.
On Thu, Jan 30, 2025, at 17:41, Arkadiy Kulev wrote:
Hello!
This wouldn’t really work because FPM does not control the script during execution and would have to check it out after each allocation which is not really viable.
Thanks for the feedback! I agree that monitoring memory usage after each allocation would be infeasible. However, my suggestion was actually to check memory usage only once per request, specifically at request shutdown, when FPM regains control and before assigning another request to that worker.
We already have hooks for request startup and request shutdown in FPM. Could we simply insert a memory check at the request shutdown stage—where the worker is returning control to the FPM master process—before picking up a new request?
Just to be clear, “memory_limit” helps kill runaway scripts mid-request. By contrast, the newly proposed pm.max_memory is meant to catch processes with a slow leak across multiple requests. We only need to check at the end of each request, which is presumably when the worker returns control to FPM.
To be honest, I haven’t seen a ‘slow’ memory leak in a long time – except when using fgetcsv which has had a reported memory leak since ~2012 somewhere on the old bug tracker. (I lost the link to it and don’t even remember how to get to the tracker or if it still exists.) I haven’t checked if it has been fixed since 8.2, but I’ve seen a couple of reports of it on r/php a couple of times in the last couple of years.
To be honest, I haven’t seen a ‘slow’ memory leak in a long time – except when using fgetcsv which has had a reported memory leak since ~2012 somewhere on the old bug tracker. (I lost the link to it and don’t even remember how to get to the tracker or if it still exists.) I haven’t checked if it has been fixed since 8.2, but I’ve seen a couple of reports of it on r/php a couple of times in the last couple of years.
A significant number of production environments still run PHP 7.4. Also, beyond the standard extensions, many installations rely on third-party or custom C extensions for specialized tasks (e.g., image processing, machine learning, or connections to external systems). Even a small bug in those C bindings can cause slow leaks that accumulate over multiple requests. In these cases, a process-level memory limit (checked after each request) can help avoid excessive memory growth without waiting on a fix or implementing a complicated workaround. Though it might be rare in core PHP these days, the fact that such issues can arise (especially in non-core extensions) is one of the main reasons some users might like a pm.max_memory mechanism.
This wouldn’t really work because FPM does not control the script during execution and would have to check it out after each allocation which is not really viable.
Thanks for the feedback! I agree that monitoring memory usage after each allocation would be infeasible. However, my suggestion was actually to check memory usage only once per request, specifically at request shutdown, when FPM regains control and before assigning another request to that worker.
I think that would require a different name because it would not reflect max memory usage in any way - it would be measured at the time when the memory usage is lowest. We could maybe set some options that would measure the maximum of increase of memory between requests - I mean difference between lowest memory usage (most likely after the first request) and then compare against current usage after the latest request and set limit on this difference. Not sure about the name for that. Maybe something like pm.max_memory_increase or something like that.
We already have hooks for request startup and request shutdown in FPM. Could we simply insert a memory check at the request shutdown stage—where the worker is returning control to the FPM master process—before picking up a new request?
Yeah that would be possible.
Just to be clear, “memory_limit” helps kill runaway scripts mid-request. By contrast, the newly proposed pm.max_memory is meant to catch processes with a slow leak across multiple requests. We only need to check at the end of each request, which is presumably when the worker returns control to FPM.
There is one thing to note that memory_limit actually measure only memory allocated through the per request php memory allocator so it’s not actually limit on total usage including the standard allocator memory usage. So there would be still a use case for total limit using cgroups but I agree that the more important use is to catch slow leaks which the above should help with in a better way than pm.max_requests.
To be honest, I haven’t seen a ‘slow’ memory leak in a long time – except when using fgetcsv which has had a reported memory leak since ~2012 somewhere on the old bug tracker. (I lost the link to it and don’t even remember how to get to the tracker or if it still exists.) I haven’t checked if it has been fixed since 8.2, but I’ve seen a couple of reports of it on r/php a couple of times in the last couple of years.
I think it might still happen from time to time in external extensions. Just recently Niels fixed this one https://github.com/php/pecl-xml-xmldiff/pull/3 . In addition there can be also a bug in an external library so it’s not unlikely that there are still many such cases.
Currently many people just have pm.max_requests in configuration for that which is sometimes set to unnecessary low value. Having option that depends on memory instead could reduce significantly the number of restarts - in most cases probably to zero. In addition, it could also help to identify that there is a leak which can get unnoticed if the users don’t have alarms on memory usage or just use pm.max_requests.
This wouldn’t really work because FPM does not control the script during execution and would have to check it out after each allocation which is not really viable.
Thanks for the feedback! I agree that monitoring memory usage after each allocation would be infeasible. However, my suggestion was actually to check memory usage only once per request, specifically at request shutdown, when FPM regains control and before assigning another request to that worker.
I think that would require a different name because it would not reflect max memory usage in any way - it would be measured at the time when the memory usage is lowest. We could maybe set some options that would measure the maximum of increase of memory between requests - I mean difference between lowest memory usage (most likely after the first request) and then compare against current usage after the latest request and set limit on this difference. Not sure about the name for that. Maybe something like pm.max_memory_increase or something like that.
I see where you’re coming from, but I believe measuring memory “delta” or “increase” could be confusing for end users. In practice, admins and developers often glance at tools like top or ps to see current memory usage for each FPM worker, spot outliers, and then set a threshold accordingly.
If we start talking about “lowest usage” vs. “current usage” and a “max increase,” that becomes much harder to translate to real-world monitoring—and it’s non-intuitive compared to simply reading the value right off top and setting a limit. So, from a usability standpoint, I think a direct measurement of resident memory (as people see in common system tools) would be the most straightforward and least confusing.
Just to be clear, “memory_limit” helps kill runaway scripts mid-request. By contrast, the newly proposed pm.max_memory is meant to catch processes with a slow leak across multiple requests. We only need to check at the end of each request, which is presumably when the worker returns control to FPM.
There is one thing to note that memory_limit actually measure only memory allocated through the per request php memory allocator so it’s not actually limit on total usage including the standard allocator memory usage. So there would be still a use case for total limit using cgroups but I agree that the more important use is to catch slow leaks which the above should help with in a better way than pm.max_requests.
You’re absolutely right that cgroups handle total memory usage—including memory outside the PHP allocator—more accurately. But as I’ve mentioned before, relying on cgroups to limit memory typically means an OOM kill that can happen at any moment, often right in the middle of a request. That’s precisely what I’m trying to avoid.
The whole idea behind pm.max_memory is to allow a graceful check after each request completes, so we can recycle the worker before starting a new request. That way, no request gets abruptly killed. cgroups don’t really accommodate that scenario—they’re great for overall resource control but not for per-request, child-level recycling within FPM.
On Fri, Jan 31, 2025 at 4:06 PM Arkadiy Kulev <eth@ethaniel.com> wrote:
Hi!
This wouldn’t really work because FPM does not control the script during execution and would have to check it out after each allocation which is not really viable.
Thanks for the feedback! I agree that monitoring memory usage after each allocation would be infeasible. However, my suggestion was actually to check memory usage only once per request, specifically at request shutdown, when FPM regains control and before assigning another request to that worker.
I think that would require a different name because it would not reflect max memory usage in any way - it would be measured at the time when the memory usage is lowest. We could maybe set some options that would measure the maximum of increase of memory between requests - I mean difference between lowest memory usage (most likely after the first request) and then compare against current usage after the latest request and set limit on this difference. Not sure about the name for that. Maybe something like pm.max_memory_increase or something like that.
I see where you’re coming from, but I believe measuring memory “delta” or “increase” could be confusing for end users. In practice, admins and developers often glance at tools like top or ps to see current memory usage for each FPM worker, spot outliers, and then set a threshold accordingly.
If we start talking about “lowest usage” vs. “current usage” and a “max increase,” that becomes much harder to translate to real-world monitoring—and it’s non-intuitive compared to simply reading the value right off top and setting a limit. So, from a usability standpoint, I think a direct measurement of resident memory (as people see in common system tools) would be the most straightforward and least confusing.
It’s probably less confusing than setting pm.max_memory to some value and then see that the process allocates much more. We could potentially call it pm.max_idle_memory or something that clearly shows that it’s not a total max memory.
Just to be clear, “memory_limit” helps kill runaway scripts mid-request. By contrast, the newly proposed pm.max_memory is meant to catch processes with a slow leak across multiple requests. We only need to check at the end of each request, which is presumably when the worker returns control to FPM.
There is one thing to note that memory_limit actually measure only memory allocated through the per request php memory allocator so it’s not actually limit on total usage including the standard allocator memory usage. So there would be still a use case for total limit using cgroups but I agree that the more important use is to catch slow leaks which the above should help with in a better way than pm.max_requests.
You’re absolutely right that cgroups handle total memory usage—including memory outside the PHP allocator—more accurately. But as I’ve mentioned before, relying on cgroups to limit memory typically means an OOM kill that can happen at any moment, often right in the middle of a request. That’s precisely what I’m trying to avoid.
The whole idea behind pm.max_memory is to allow a graceful check after each request completes, so we can recycle the worker before starting a new request. That way, no request gets abruptly killed. cgroups don’t really accommodate that scenario—they’re great for overall resource control but not for per-request, child-level recycling within FPM.
Yeah it should really be the last resort. Agreed that for this particular case, the solution above would be better.