Nginx Upstream Keepalive Benchmark: Fixing 502 & 499 Spikes

Disclosure: This guide contains an affiliate link to Cloudways. If you sign up through it, GearFlow Lab may earn a commission at no extra cost to you. Read our FTC & editorial disclosure.

17 min readBy Yun Sanghun (M. Yun)

When an Nginx reverse proxy sits in front of a Python (FastAPI, Uvicorn, Gunicorn), Node.js, or PHP-FPM application container, two status codes dominate production error logs during traffic bursts: 502 Bad Gateway (upstream prematurely closed connection while reading response header from upstream) and Nginx's non-standard 499 Client Closed Request. Under default settings, Nginx opens a brand-new HTTP/1.0 TCP socket with Connection: close on every single upstream request, exhausting backend worker queues in milliseconds while mismatched client/proxy timeouts trigger cascading 499 disconnects.

To measure how upstream connection pooling and timeout alignment change failure rates under a 50-client concurrency spike, I ran 200 concurrent HTTP requests across 50 parallel threads on Docker Desktop (desktop-linux, nginx:1.27-alpine fronting a 12-worker Python backend) across three distinct reverse-proxy upstream configurations: an unpooled 12-worker upstream without keepalive queueing, a mismatched timeout pair where the client/proxy cuts off at 20 ms while the upstream worker takes 45 ms, and a tuned HTTP/1.1 keepalive 32 pool with matched timeouts.

LAB SUMMARY · 200 CONCURRENT REQUESTS AT 50-CLIENT CONCURRENCY
  • Run 1: Unpooled Upstream (No Keepalive, 12 Worker Slots): Only 12 / 200 requests succeeded (6.0%). The remaining 188 / 200 failed with HTTP 502 Bad Gateway (94.0% dropped) as the upstream rejected excess connections immediately.
  • Run 2: Timeout Mismatch (20 ms Proxy/Client Cutoff vs 45 ms Upstream Execution): 200 / 200 requests logged HTTP 499 Client Closed Request (0% completed) at a p95 of 25.3 ms, while upstream workers continued burning CPU cycles on abandoned requests.
  • Run 3: Tuned HTTP/1.1 Keepalive Pool (keepalive 32 + Matched Timeouts): All 200 / 200 requests returned HTTP 200 OK (0 errors) at 3,272.6 req/s with a p50 latency of 4.6 ms and p95 of 52.4 ms.

1. Reproducing HTTP 502 (Backlog Overflow) vs HTTP 499 (Timeout Mismatch)

By default, Nginx opens a brand-new HTTP/1.0 connection with Connection: close to your upstream backend on every incoming request. Under a 50-client burst, opening and tearing down a fresh socket on every request saturates a 12-worker pool (pm.max_children in PHP-FPM or --workers / --limit-concurrency in Uvicorn/Gunicorn) within milliseconds.

Terminal output from 50-thread reverse proxy benchmark against Nginx 1.27.3 showing 188 of 200 requests failing with HTTP 502 Bad Gateway when unpooled and 200 of 200 failing with HTTP 499 when client timeout is 20ms vs 45ms upstream
Running 200 concurrent requests across 50 parallel threads against Nginx 1.27.3: unpooled HTTP/1.0 upstream exhaustion triggered 188/200 HTTP 502 Bad Gateway drops (Run 1), while a 20 ms client cutoff against a 45 ms upstream handler triggered 200/200 HTTP 499 disconnects (Run 2).

The distinction between these two failure signatures determines which layer of your stack requires tuning. In Run 1, all 12 upstream worker slots were immediately occupied by the first wave of requests, and because no upstream keepalive pool was configured in Nginx, the remaining 38 concurrent threads attempted fresh TCP handshakes against an exhausted listen queue. The backend rejected or reset those excess sockets immediately, causing Nginx to return 188 / 200 HTTP 502 Bad Gateway responses at a fast-fail rate of 8,465.5 req/s (p50 = 0.1 ms, p95 = 22.4 ms). You can reproduce this exact 50-thread burst and inspect the status distribution using the following command inside your test environment:

# Run 200 concurrent requests across 50 parallel threads and tally HTTP status codes
docker exec gfl-bench python3 bench_nginx_upstream.py --requests 200 --concurrency 50
docker exec gfl-nginx-34 awk '{print $6, $7}' /var/log/nginx/access_upstream.log | sort | uniq -c
Split terminal comparing unpooled HTTP/1.0 Connection: close dropping 188 of 200 requests with HTTP 502 on the left against tuned HTTP/1.1 keepalive 32 completing 200 of 200 requests at 3,272.6 req/s on the right
Side-by-side comparison on Nginx 1.27.3: unpooled HTTP/1.0 drops 188 of 200 requests with HTTP 502 (94.0% loss, left), whereas enabling keepalive 32 with HTTP/1.1 and cleared Connection header completes 200/200 requests at 3,272.6 req/s (p50=4.6 ms, p95=52.4 ms, right).

When we compare Run 1 (left pane above) directly against Run 3 (right pane above), the mechanical impact of HTTP/1.1 socket pooling becomes unmistakable. In unpooled HTTP/1.0 mode, Nginx executes 200 separate TCP three-way handshakes (SYN, SYN-ACK, ACK) and 200 socket teardowns (FIN / RST) for a 200-request burst, resulting in 94.0% traffic loss (only 12 / 200 requests returning HTTP 200 OK). By contrast, enabling keepalive 32; in the upstream block and switching the proxy location to proxy_http_version 1.1; with proxy_set_header Connection ""; allows Nginx to open just 12 initial TCP connections and reuse those warm sockets across the remaining 188 requests, achieving 200 / 200 HTTP 200 OK in 0.061 seconds (3,272.6 req/s, p50 = 4.6 ms, p95 = 52.4 ms).

Nginx upstream keepalive architecture contrasting unpooled HTTP/1.0 TIME_WAIT churn and 5-second idle FIN race 502/499 errors against pooled HTTP/1.1 keepalive 64 sockets with ordered keepalive_timeout
Upstream Keepalive & Timeout Race Topology: Enabling keepalive 64 with proxy_http_version 1.1, empty Connection "", and upstream keepAliveTimeout (65 s) > keepalive_timeout (55 s) eliminates both ephemeral port churn and idle-close 502/499 spikes.
Terminal displaying /var/log/nginx/error.log and access_upstream.log separating upstream_status=502 premature close errors from status=499 upstream_status=- client cutoffs
Inspecting /var/log/nginx/error.log and access_upstream.log inside nginx:1.27-alpine: contrasting recv() failed (104: Connection reset by peer) (status=502 upstream_status=502) against premature client disconnects (status=499 upstream_status=- at 0.020s).

Before modifying nginx.conf, always verify whether your production alerts are driven by 502 upstream resets or 499 client cutoffs by adding $upstream_status, $request_time, and $upstream_response_time to your custom log_format in /etc/nginx/nginx.conf. As shown in the terminal capture above, an unpooled backlog overflow writes upstream prematurely closed connection while reading response header or recv() failed (104: Connection reset by peer) to /var/log/nginx/error.log, paired with status=502 upstream_status=502 req_time=0.001 in access_upstream.log. Conversely, when a caller disconnects at 20 ms while the backend handler needs 45 ms, Nginx logs status=499 upstream_status=- req_time=0.020 up_time=- because the client closed the downstream TCP socket before the upstream worker transmitted a single response header byte.

Code editor displaying broken default unpooled nginx.conf lacking keepalive in upstream and defaulting to proxy_http_version 1.0 and Connection: close
Why default Nginx reverse-proxy blocks collapse under concurrency: omitting keepalive in upstream and leaving proxy_http_version at its 1.0 default forces Connection: close and a full TCP handshake/teardown on every request.

Why does a clean install of Nginx fail so abruptly under a 50-thread burst? Auditing the default configuration file in /etc/nginx/conf.d/default.conf exposes four compounding defaults in Nginx's ngx_http_proxy_module. First, an upstream block without an explicit keepalive directive caches zero idle connections. Second, proxy_http_version defaults to legacy 1.0 rather than 1.1. Third, Nginx automatically rewrites the upstream Connection header to close, instructing Uvicorn, Gunicorn, Node.js, or PHP-FPM to terminate the TCP socket after every single response. Fourth, if an outer gateway or proxy sets a tight cutoff (such as 20 ms) below the backend execution time (45 ms), every single request aborts mid-flight:

# ANTI-PATTERN: Default Nginx upstream behavior (0 pooled sockets, per-request teardown)
upstream app_backend {
    server 127.0.0.1:8000;
    # Missing keepalive directive -> 0 idle sockets cached!
}
location /api/ {
    proxy_pass http://app_backend;
    # Defaults to proxy_http_version 1.0 and Connection: close!
}
Split comparison showing 200 of 200 HTTP 499 errors when client timeout is 20ms vs 45ms upstream handler on the left and 200 of 200 HTTP 200 OK when timeouts follow 70s > 65s >= 60s on the right
Why inverted timeouts trigger 100% HTTP 499 spikes: a 20 ms caller cutoff against a 45 ms upstream handler aborts all 200 requests while wasting 25 ms of backend CPU per request (left), whereas aligning 70s > 65s >= 60s completes 200/200 cleanly (right).

Run 2 (left pane above) demonstrates why HTTP 499 Client Closed Request is often more dangerous to backend stability than a fast-failing 502. When the upstream worker accepted the request and required 45 ms of CPU and database time to finish, but the outer caller (such as an aggressive frontend AbortController, webhook sender, Cloudflare proxy, or AWS ALB) gave up after 20 ms, all 200 / 200 requests logged HTTP 499 at p50 = 24.6 ms and p95 = 25.3 ms. Worse, unless your backend framework explicitly polls for client socket disconnects and cancels in-flight coroutines, all 12 upstream workers continue burning CPU cycles for the remaining 25 ms per abandoned request, wasting 5,000 ms of cumulative backend compute across 200 requests that return zero data to users.

Dual-panel Matplotlib chart comparing HTTP 200, 502, and 499 request counts and p50/p95 latency across Unpooled, Mismatched Timeout, and Tuned keepalive 32 runs
Empirical 50-client concurrency benchmark across 200 requests: comparing HTTP status breakdown (left) and p50/p95 latency (right) across Run 1 (188 HTTP 502), Run 2 (200 HTTP 499), and Run 3 (200 HTTP 200 OK at 3,272.6 req/s, p50=4.6 ms, p95=52.4 ms).

2. Full Side-by-Side Benchmark Matrix (200 Requests, 50 Concurrent Clients)

Plotting all three 50-client concurrency runs side by side illustrates how upstream socket pooling and timeout hierarchy alignment work together. Removing per-request TCP handshakes via keepalive 32 eliminates the 188 backlog resets seen in Run 1, while aligning the timeout hierarchy (Client 70s > Nginx 65s >= Upstream Worker 60s) eliminates the 200 premature client disconnects seen in Run 2 (whereas long-lived text/event-stream routes also require disabling proxy response buffering, as measured in our n8n MCP Server Trigger Nginx SSE buffering & 3600s timeout fix):

Upstream Configuration HTTP 200 OK HTTP 502 Drops HTTP 499 Cutoffs Throughput p50 Latency p95 Latency
1. Unpooled Upstream (No Keepalive) 12 / 200 (6.0%) 188 / 200 (94.0%) 0 / 200 8,465.5 req/s (fast fail) 0.1 ms 22.4 ms
2. Mismatched Timeout (Client 20 ms < App 45 ms) 0 / 200 (0.0%) 0 / 200 200 / 200 (100%) 1,995.3 req/s (aborted) 24.6 ms 25.3 ms
3. Tuned keepalive 32 + Matched Timeouts 200 / 200 (100%) 0 / 200 0 / 200 3,272.6 req/s 4.6 ms 52.4 ms
Terminal output of ss -tan inside nginx:1.27-alpine comparing 188 TIME-WAIT sockets in unpooled mode against 12 persistent ESTAB sockets and 0 TIME-WAIT in keepalive 32 mode
Kernel socket audit using ss -tan '( dport = :8000 )' inside nginx:1.27-alpine: replacing 188 torn-down TIME-WAIT ephemeral sockets (top) with 12 persistent ESTABLISHED backend connections and zero TIME-WAIT churn (bottom).

To verify that Nginx is genuinely reusing persistent upstream sockets rather than silently opening and closing connections on every request, inspect the Linux kernel socket table inside the Nginx container with ss -tan. In Run 1 (unpooled HTTP/1.0), running ss -tan '( dport = :8000 or sport = :8000 )' immediately after the burst reveals 188 TIME-WAIT sockets lingering for 60 seconds and 0 ESTAB idle connections. Under sustained traffic, this TIME-WAIT accumulation exhausts the container's ephemeral port range (net.ipv4.ip_local_port_range) and triggers 99: Cannot assign requested address errors. Once keepalive 32; is enabled in Run 3, ss -tan reports exactly 12 ESTAB warm sockets connected to 127.0.0.1:8000 and 0 TIME-WAIT sockets:

# Audit ESTABLISHED vs TIME-WAIT upstream sockets to port 8000 inside the Nginx container
docker exec gfl-nginx-34 ss -tan state established '( dport = :8000 )'
docker exec gfl-nginx-34 sh -c "ss -tan '( dport = :8000 )' | awk 'NR>1 {print \$1}' | sort | uniq -c"
Live OpenObserve v0.14.2 Logs Explorer at http://127.0.0.1:5080 displaying stream nginx_upstream_bench with 5 ingested Nginx 1.27.3 benchmark profiles
Auditing all five Nginx 1.27.3 upstream benchmark profiles in OpenObserve v0.14.2 (http://127.0.0.1:5080, stream nginx_upstream_bench): correlating err_502, err_499, ok_200, p50_ms, and p95_ms in 9 ms.

In a multi-container production environment, grepping raw log files across ephemeral Docker containers makes it easy to miss brief 5-second bursts of 502 or 499 errors. By shipping structured JSON access logs from nginx:1.27-alpine into a self-hosted public.ecr.aws/zinclabs/openobserve:v0.14.2 instance on http://127.0.0.1:5080 (stream nginx_upstream_bench, setup detailed in our OpenObserve Docker Self-Hosting Guide), we can query err_502, err_499, ok_200, p50_ms, p95_ms, and upstream_status across all five benchmark test profiles in 9 ms. The live OpenObserve Logs Explorer capture above immediately isolates the 94.0% drop rate in run_01_unpooled_502 and 100.0% cutoff rate in run_02_mismatch_499 against the 0.0% drop rate and 3,272.6 req/s throughput of run_03_keepalive_32.

Code editor showing production-keepalive.conf with keepalive 32, keepalive_requests 1000, keepalive_timeout 60s, proxy_http_version 1.1, proxy_set_header Connection empty string, and proxy_read_timeout 65s
Production drop-in /etc/nginx/conf.d/production-keepalive.conf: combining keepalive 32, keepalive_timeout 60s, proxy_http_version 1.1, proxy_set_header Connection "", and 65s proxy read/send timeouts.

3. Production Drop-In Nginx Config: Upstream Keepalive + Timeout Hierarchy

To prevent both per-request socket exhaustion (502) and premature client/proxy cutoffs (499 / 504), deploy the complete production configuration below in /etc/nginx/conf.d/production-keepalive.conf. It pairs an upstream keepalive 32; pool with explicit HTTP/1.1 header normalization and a monotonically aligned timeout ladder. You can also calculate custom worker and timeout values for your server RAM in our Interactive Nginx 502/504 Timeout & PHP-FPM Config Generator:

upstream app_backend {
    server 127.0.0.1:8000 max_fails=3 fail_timeout=10s;
    # Keep up to 32 idle persistent connections open per Nginx worker process
    keepalive 32;
    keepalive_requests 1000;
    keepalive_timeout 60s;
}

server {
    listen 8080;
    server_name api.example.com;

    location / {
        proxy_pass http://app_backend;

        # Mandatory for upstream keepalive socket reuse:
        proxy_http_version 1.1;
        proxy_set_header Connection "";

        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

        # Match timeouts to your upstream worker execution ceiling:
        proxy_connect_timeout 5s;
        proxy_send_timeout 65s;
        proxy_read_timeout 65s;
    }
}
Split comparison showing keepalive 32 without proxy_set_header Connection empty string still forwarding Connection: close and dropping 188/200 requests on the left versus full socket reuse on the right
The half-configured keepalive trap: defining keepalive 32 in upstream without setting proxy_set_header Connection "" in location causes Nginx to forward Connection: close (0% socket reuse, 188 HTTP 502s, left) versus 94% socket reuse when cleared (right).

The single most common mistake engineers make when configuring Nginx upstream pooling is adding keepalive 32; to the upstream block and setting proxy_http_version 1.1; in the location block, while forgetting to add proxy_set_header Connection "";. As proven in our fourth benchmark run (run_04_missing_connection_hdr, left pane above), Nginx still injects Connection: close into every forwarded HTTP/1.1 request header unless you explicitly clear it with an empty string! When the upstream Uvicorn or Node.js worker receives Connection: close, it obeys the HTTP/1.1 specification and closes the TCP socket immediately after sending the response, resulting in 0% socket reuse and the exact same 188 / 200 HTTP 502 collapse (94.0% dropped) as an unpooled configuration.

Matplotlib chart showing monotonically increasing active request timeout ladder (55s < 60s < 65s < 70s) on the left and idle keepalive alignment (Nginx 60s < Uvicorn 65s) on the right
Dual timeout alignment rules: active request timeouts must step up from inner DB to outer client (55s < 60s < 65s < 70s, left), while idle keepalive timeouts must be shorter on Nginx (60s) than on Uvicorn (65s) to prevent race 502 resets (right).

Fixing socket reuse is only half of the equation; you must also align both the Active Request Timeout Ladder and the Idle Keepalive Timeout Order across every hop in your infrastructure. As plotted in the left panel of the Matplotlib chart above, active request timeouts must decrease monotonically from the outside in: set your outer CDN or AWS ALB idle timeout to 70s, Nginx proxy_read_timeout and proxy_send_timeout to 65s, your application worker ceiling (Gunicorn --timeout 60) to 60s, and your database query timeout (statement_timeout = 55s) to 55s. Conversely, as shown in the right panel, idle keepalive timeouts must increase from proxy to backend: Nginx's keepalive_timeout 60s; must be shorter than Uvicorn's --timeout-keep-alive 65 so Nginx retires idle sockets before the backend unexpectedly resets them.

Terminal showing nginx -t syntax ok, ApacheBench ab -n 200 -c 50 -k completing 200 requests with 0 failed at 3,272.60 req/s, and curl /nginx_status showing 12 waiting keepalive connections
End-to-end CLI verification on desktop-linux: nginx -t syntax check, ApacheBench (ab -n 200 -c 50 -k) confirming 200 complete / 0 failed requests at 3,272.60 req/s, and stub_status reporting 12 waiting persistent connections.

After updating /etc/nginx/conf.d/production-keepalive.conf, always validate the configuration syntax with nginx -t before reloading worker processes, then run a live concurrency verification with ApacheBench (ab -n 200 -c 50 -k) and query Nginx's stub_status endpoint. In the terminal output above, nginx -t confirms clean syntax on nginx/1.27.3, and ab -n 200 -c 50 -k http://127.0.0.1:8080/api/work finishes all 200 requests in 0.061 seconds with Failed requests: 0, Keep-Alive requests: 200, and 3,272.60 [#/sec] throughput (p50 = 4.6 ms, p95 = 52.4 ms). Querying http://127.0.0.1:8080/nginx_status further confirms that 612 cumulative requests were served across just 214 TCP accepts with Waiting: 12 persistent idle sockets ready for the next burst.

Live OpenObserve v0.14.2 Streams inventory at http://127.0.0.1:5080/web/streams displaying the active nginx_upstream_bench log stream
OpenObserve v0.14.2 Streams view (http://127.0.0.1:5080/web/streams): indexing structured Nginx and Nginx Proxy Manager (NPM) upstream keepalive telemetry in the nginx_upstream_bench stream.

If you manage your Docker reverse proxies through a graphical WebUI such as Nginx Proxy Manager (NPM) (covered in our Nginx Proxy Manager Docker 502 Troubleshooting Guide and Traefik vs Caddy vs NPM Benchmark), note that NPM's standard Proxy Host form generates a direct proxy_pass without an upstream keepalive pool. To enable keepalive 32 in NPM, define your upstream app_backend { server app:8000; keepalive 32; } block in /data/nginx/custom/http_top.conf and add proxy_http_version 1.1; proxy_set_header Connection ""; inside the Proxy Host's Advanced → Custom Nginx Configuration tab. Streaming those proxy logs into OpenObserve's /web/streams inventory (nginx_upstream_bench, shown above) provides instant schema indexing and real-time SLA alerting whenever err_502 or err_499 exceeds zero.

Code editor displaying docker-compose.yml with net.core.somaxconn=2048 sysctls and uvicorn --workers 4 --backlog 2048 --timeout-keep-alive 65
Production docker-compose.yml tuning: raising net.core.somaxconn=2048 on both containers and launching Uvicorn with --backlog 2048 and --timeout-keep-alive 65 (5 seconds longer than Nginx keepalive_timeout 60s).

4. Docker Compose Kernel Backlog & Uvicorn Idle Keepalive Tuning

Even with a properly configured nginx.conf, default Linux container sysctls and Python ASGI server flags can still bottleneck cold-start bursts or cause sporadic post-idle 502 errors. By default, Docker containers inherit a restrictive kernel socket listen backlog (often 128) and Uvicorn defaults to a 5-second HTTP keepalive idle timeout (--timeout-keep-alive 5). In your production docker-compose.yml, raise net.core.somaxconn: 2048 on both containers and pass --backlog 2048 --timeout-keep-alive 65 to Uvicorn:

services:
  nginx_proxy:
    image: nginx:1.27-alpine
    ports: ["8080:8080"]
    volumes: ["./production-keepalive.conf:/etc/nginx/conf.d/default.conf:ro"]
    sysctls:
      net.core.somaxconn: 2048
      net.ipv4.ip_local_port_range: "10240 65535"
    depends_on: [app_backend]

  app_backend:
    image: python:3.12-slim-bookworm
    sysctls:
      net.core.somaxconn: 2048
    command: >
      uvicorn main:app --host 0.0.0.0 --port 8000
      --workers 4 --backlog 2048 --timeout-keep-alive 65
Split terminal comparing Uvicorn default --timeout-keep-alive 5 causing 12 sporadic HTTP 502 resets after a 5s idle pause on the left against --timeout-keep-alive 65 achieving 0 resets on the right
Diagnosing post-idle sporadic 502s: Uvicorn's default 5-second keepalive timeout closes pooled sockets while Nginx assumes they are valid for 60s (12/200 race 502s, left), whereas --timeout-keep-alive 65 lets Nginx retire idle sockets cleanly (0 errors, right).

Why is --timeout-keep-alive 65 mandatory when Nginx's keepalive_timeout is set to 60s? Both Uvicorn (--timeout-keep-alive 5) and Node.js Express/Fastify (server.keepAliveTimeout = 5000) close idle HTTP/1.1 keepalive connections after just 5 seconds of inactivity. As demonstrated in our fifth benchmark profile (run_05_uvicorn_5s_idle_race, left pane above), if 12 pooled sockets sit idle for 5.0 seconds, Uvicorn sends a TCP FIN to close them while Nginx still considers those sockets reusable until 60 seconds. When the next traffic burst arrives at t = 5.1s, Nginx writes HTTP requests onto all 12 closing sockets before processing the incoming FIN, triggering 12 / 200 HTTP 502 (104: Connection reset by peer) failures! Setting Uvicorn to 65s (right pane) guarantees Nginx always initiates idle socket closure first.

Matplotlib scaling curves from c=10 to c=200 concurrent clients comparing HTTP 502 drop rate (96.8% vs 0.0%) and kernel TCP sockets opened (4,000 TIME-WAIT vs 32 ESTAB)
Concurrency scaling from c=10 to c=200 parallel clients: unpooled HTTP/1.0 spikes to a 94.0%–96.8% HTTP 502 drop rate and churns 4,000 TIME-WAIT sockets, while HTTP/1.1 keepalive 32 holds a flat 0.0% error rate across 32 persistent sockets.

To test how this configuration scales beyond 50 threads, we swept concurrency from c=10 up to c=200 parallel clients (20 requests per client, up to 4,000 requests per burst). As shown in the left scaling curve above, unpooled HTTP/1.0 handles c=10 (0% error because 10 clients fit inside 12 worker slots) but hits a vertical wall at c=25 (52.0% HTTP 502), c=50 (94.0% HTTP 502), and c=200 (96.8% HTTP 502). Meanwhile, the tuned keepalive 32 pool paired with somaxconn=2048 holds a flat 0.0% HTTP 502 drop rate from c=10 through c=200. The right panel shows why: at c=200, unpooled mode churns 4,000 ephemeral TIME-WAIT sockets, whereas keepalive 32 caps backend TCP sockets at exactly 32 ESTABLISHED connections (99.2% reduction in socket creation).

Four-stage architecture diagram showing 50 External Clients (70s timeout) flowing through Nginx 1.27 (65s read timeout), Warm Keepalive Pool (12 ESTAB sockets, 3,272.6 req/s), and App + DB Workers (60s worker / 55s DB)
End-to-end Nginx HTTP/1.1 upstream keepalive architecture: multiplexing 50 external client connections onto 12 warm backend sockets at 3,272.6 req/s while enforcing the 70s > 65s > 60s > 55s timeout hierarchy.

5. End-to-End Architecture & Full-Stack 502 / 499 / 504 Triage Runbook

The four-stage architecture diagram above summarizes how every layer of the reverse-proxy pipeline cooperates under load. Stage 1 enforces a 70s outer timeout at the CDN, load balancer, or API caller so downstream connections never abort early with HTTP 499. Stage 2 configures Nginx 1.27 with keepalive 32;, keepalive_timeout 60s;, proxy_http_version 1.1;, proxy_set_header Connection "";, and proxy_read_timeout 65s;. Stage 3 multiplexes 50 concurrent client streams onto 12 warm ESTABLISHED backend sockets at 3,272.6 req/s (p50 = 4.6 ms, p95 = 52.4 ms). Finally, Stage 4 tunes the upstream Uvicorn/Gunicorn container with net.core.somaxconn=2048, --backlog 2048, --timeout-keep-alive 65, a 60s worker execution ceiling, and a 55s database statement timeout.

Six-row full-stack triage matrix mapping 502 (104 reset, 5s idle race, 111 connection refused, OOM 137), 499 (upstream_status=-), and 504 (110 timeout) to exact fixes
Full-stack production triage matrix: separating unpooled 502 backlog resets, post-idle 5s keepalive race 502s, Docker DNS 111 connection refused, 499 client cutoffs, 504 database lock timeouts, and Exit 137 OOM kills.

Even with keepalive 32 enabled, an upstream worker will still hang (triggering 499 or 504) or die (triggering 502 or Varnish 503) if the database or host memory underneath it saturates. Use the full-stack triage runbook above alongside our related benchmarks to isolate non-Nginx bottlenecks:

FULL-STACK BOTTLENECK DIAGNOSTIC MATRIX · TESTED BENCHMARKS

Nginx Upstream Keepalive Pooling & HTTP 499 Timeout Q&A

Why did adding keepalive 32 to Nginx upstream still reuse zero backend sockets?

Nginx defaults to HTTP/1.0 with Connection: close on upstream requests unless you set both proxy_http_version 1.1 and proxy_set_header Connection "" inside the location block. Without clearing the Connection header, the backend closes every TCP socket after one response.

What caused 200 out of 200 requests to log HTTP 499 in the Nginx benchmark?

When the caller/proxy cutoff (20 ms) was shorter than the upstream execution time (45 ms), clients disconnected before the backend finished, logging 200/200 HTTP 499 Client Closed Request events while backend workers kept burning CPU cycles on abandoned requests.

How much did a tuned HTTP/1.1 keepalive pool improve Nginx upstream throughput?

Across 200 concurrent requests (50 parallel clients), unpooled Nginx dropped 188/200 requests with HTTP 502, whereas enabling keepalive 32 with HTTP/1.1 empty Connection headers and matched timeouts completed 200/200 requests with zero errors at 3,272.6 req/s (p50 4.6 ms, p95 52.4 ms).

Related Lab Notes:

100% Client-Side Utilities · Printable A4 Runbook

Run Prefilled Sizing Math & Download the 1-Page Runbook

Generate upstream keepalive Nginx timeout configs and compare multi-proxy Caddy 2, Traefik v3, and Tunnel setups.

Paired Prefilled Calculators & Vector PDF Checklist

Yun Sanghun (M. Yun) · Linux, Docker & AI Infrastructure Engineer