After upgrading an Amazon EC2 instance from PHP 7.2 to PHP 7.4, I ran into a problem – the server would regularly hang. Sometimes it would happen after a few days. Sometimes it would take a month. In either case, websites served by the instance became unresponsive. The only way to get things back online was through a hard reboot, which a few times had to be done at the Amazon console level. It was not a good situation.
Through lots of research and reading, I finally feel like I have things under control. If you’re struggling with a similar problem, here’s what I did to fix it. The server has went over 4 months without requiring a hard reboot since making the changes.
Before jumping into the details, I want to be clear that I am not an Amazon EC2 Linux guru. I know enough about server administration to be dangerous, mostly to myself. Therefore, this article is directed towards those who manage EC2 Linux servers on the side, either part-time or as a hobby.
About a year ago, I decided to update an Amazon EC2 server from PHP 7.2 to PHP 7.4. As part of the update, I also decided to move to PHP-FPM to optimize memory and improve performance. Moving to PHP-FPM also required changing to the event-based Apache Multi-Processing Module (mpm_event_module).
After completing the update, the server was a lot more efficient. The problem was that it would periodically exhaust memory and become unresponsive. Since I didn’t have automated alerts setup, it could sometime take a couple of days before I realized the WordPress websites hosted on the server were down.
The problem was obviously a configuration issue. However, it was extremely hard to track down. Here is the primary issues I had to fix: under heavy load, the server would spawn the maximum php-fpm instances. Since I had the Apache setting recycling web server processes aggressively to optimize memory usage, the process that spawned the php-fpm process(es) would get killed before the php-fpm process(es) completed. At this point, the php-fpm process would hang and start leaking memory.
After trying numerous tuning options, I did the following to restore stability.
- Set MaxConnectionsPerChild to unlimited (0)
Since the web server serves WordPress sites, it’s mainly processing PHP files. There is little/no risk of memory leakage in the web processes. Therefore, I set the MaxConnectionsPerChild to 0, which means unlimited.
For more information, see the section labeled ‘MaxConnectionsPerChild / MaxRequestsPerChild’ under ‘MPM Event/Worker Optimization’ in this article – https://www.liquidweb.com/kb/apache-performance-tuning-mpm-directives/.
- Determine process sizes with ps_mem.py
Instead of guessing at your Apache (httpd) and PHP (php-fpm) process sizes, use ps_mem.py to get a handle on the exact numbers. Knowing the process sizes and the limits of the server (CPU and memory) allow you to set the numbers based on objective metrics rather than feel.
For more information on installing ps_mem.py on your server, see ‘Calculate process size’ in this article – https://medium.com/@sbuckpesch/apache2-and-php-fpm-performance-optimization-step-by-step-guide-1bfecf161534.
- Determine settings using a spreadsheet
The same article referenced in step 2 links to a spreadsheet that calculates your settings based on the size of your Amazon instance. As an analytical person, I love this method. It shows how the settings are calculated and eliminates the guesswork.
To get the spreadsheet, see the link right above the heading labeled ‘Detailed Setup’ in the article reference in step 2.
- Test the settings
As a final proof point, it’s worth testing the server settings using Apache Bench Tests. For example, if you wanted to simulate 5,000 requests with 100 requests in parallel, you would use the following command: ab -n 5000 -c 100
Here are the settings that I’ve implemented for Apache and PHP-FPM. These were calculated using the spreadsheet. Bear in mind these settings are for an Amazon EC2 t4g.micro instance (2 CPUs and 1Gb RAM).
httpd.conf <Apache> StartServers 2 MinSpareThreads 25 MaxSpareThreads 75 ThreadLimit 64 ThreadsPerChild 25 MaxRequestWorkers 150 MaxConnectionsPerChild 0 ServerLimit 80
www.conf <php-fpm> pm = ondemand pm.max_children = 16 pm.process_idle_timeout = 5s pm.max_requests - 1000
Addressing the server stability was a very time-consuming and frustrating problem that I finally feel like I’ve put behind me. I sifted through a lot of material before finding the two articles above that were by far the best I found for optimizing Apache and PHP-FPM. I’d highly recommend reading through both. Note that Sebastian’s article on Medium includes links to the references he used as the basis for his article if you want to go even deeper into the reasoning behind the optimizations.