why does apache proxying to gunicorn throw 502s only during traffic spikes?

Solved502nginxreverse-proxy5 replies · 201 views · started
Accepted solution by Ivan Jensen — ok so for PHP FPM, check its log for server reached pm.max children setting 😂 when all workers are busy, new requests queue and…
  1. woke up to monitoring alerts and this lovely page on every URL:

    it's Apache proxying to Gunicorn, and it seems to happen only during traffic spikes. the error log mentions teh upstream a lot but i'm not suree if that's the cause or just a symptom. i want to fix this properly instead of adding a cron job that restarts services every hour. where do i start? i'll happily post whatever logs would help narrow this down. what am i doing wrong??

    1
  2. hmmm what exactly does teh proxy error log say at teh time of a 502? ??

    4 0 helpful
  3. yeahhh a 502 means teh proxy reacehd out to the app behind it and got a broken or no response. read the proxys error log first: sudo tail -n 100 /var/log/nginx/error.log for Nginx. connect() failed (111: Connection refused) means nothing is listening; upstream prematurely closed connection means teh app crashed or killed the request mid-way.

    0 0 helpful
  4. Accepted solution

    ok so for PHP-FPM, check its log for server reached pm.max_children setting 😂 when all workers are busy, new requests queue and eventually fail. tune pm.max_children based on available RAM divided by average worker memory, not by guesswork.

    7 3 helpful
  5. @ivan.jensen WOWWW the socket path changed after a PHP update and teh Nginx config still pointed at the old one. Updated fastcgi_pass, reloaded, done. Embarrassing, but fixed. seriously made my day 😭🙏

    9 0 helpful

Reply to this topic

Sign in or create an account to reply.