Smarter by Design.Simpler for Business
TR7Enterprise Application Delivery & Security Platform
I get 502 Bad Gateway; what are the possible causes?
Short answer
A 502 generated by TR7 indicates that a valid response could not be received from the Backend Service: the server closed the connection or crashed without sending the response headers, the response violated the HTTP rules, the response headers did not fit into the Buffer Size(kB) value, or the connection was refused by the server. Typical causes are the application crashing or closing the connection early, invalid or too large response headers, and SSL Service or Proxy mismatches between TR7 and the server. Which Backend Service and which stage are responsible is read from the Backend Service value in HTTP Logs and from the Termination State in Debug Mode Logs; the application's own 502 appears as Normal in Termination state.
Possible Causes
| Cause | Termination state | How to verify | Solution |
|---|---|---|---|
| The application crashes while processing the request or closes the connection early | During response headers · Server aborted/refused | Server logs; errors concentrated on a single server or a single path | The application is fixed; the health check is deepened so that it checks the application's actual function (What is the difference between a TCP port check and an HTTP check; which should I choose?) |
| Invalid response header (e.g. a disallowed character in the header name) | During response headers · TR7 intercepted | The raw response is examined with curl -sv directly to the server | The application's response is fixed |
| The response headers do not fit into the Buffer size(kB) value (e.g. a large Set-Cookie) | During response headers · TR7 intercepted | The total size of the headers in the direct response | Buffer Size(kB) is increased (What is Buffer size used for; when should I change it?) |
| The server or a device in between refuses the connection | While connecting to backend · Server aborted/refused; Retries greater than zero | The server service and port; the client may see 502 or 503 | The server service and port are fixed (Users see "503 Service Unavailable" / "No Available Server"; what should I do?) |
| SSL service or Certificate verification does not match the server | The TLS connection cannot be established: While connecting to backend · Server aborted/refused | Whether the server listens on HTTPS or HTTP; its certificate | SSL Service and Certificate Verification are set according to the server (What does the SSL Service option do; what is needed to connect to the backend over an encrypted connection?, Can I verify the backend certificate when connecting to the backend over HTTPS?) |
| Proxy protocol mismatch | During response headers · Server aborted/refused | Whether the server expects a PROXY header | The Proxy setting is corrected according to the server (What is the Proxy Protocol setting, and when should it be enabled?) |
| The own 502 of the application or of another proxy behind it | Normal; Backend service = the server's name | 502 is also returned on a direct request | The back end is fixed |
Where Is It Monitored?
| Screen | Path | What it shows |
|---|---|---|
| HTTP Logs | Load Balancer vServices → vService → Logs & Tracking | Requests that received 502, Backend Service and Response Source |
| Debug Mode Logs | In the same tab (while the vService is in Debug mode) | Termination State, Retries, response headers (Response tab) |
| Live Stats → Backend Services | vService → Live Stats | Connection errors per Backend Service; Connect and Response are separate in the tooltip |
| Backend Response Code Distribution | vService → Dashboard | Response code classes per Backend Service |
How to Diagnose?
- Whose page is it? TR7's 502 page shows a large "502" and "Invalid Response". If a Show Content for Traffic Manager Errors or Show Content for Backend Service Errors action has been added to the vService, the page seen is custom content; the source is determined from the logs (Can I show a custom page for 503/502/504 errors generated by TR7 itself?).
- HTTP logs: Left Monitor Panel → Load Balancer vServices → vService → Logs & Tracking → HTTP Logs is opened; in the Filter panel,
502is selected from the Status Code card. It is checked whether the Backend Service value always shows the same server and whether the 502s are concentrated on a specific path or request type (HTTP logs filters). If there are no records, Log Profile is not on in the vService. - Debug mode logs: In the vService edit form, Status = Debug Mode is selected and saved; the error is reproduced, and the Termination State value of the record is matched with the table above (How do I find out from the logs why a request failed (termination state)?). When the examination is finished, Status is set back to Enabled.
- Direct response: The same request is sent to the server directly or from WEB-CLI with the route table of the Backend Service selected, and the raw response is examined; for the device itself,
Hostis written in thev-devicefield, and the route table name is taken from the Tab list (It works when the backend is accessed directly but not through TR7; why?):bashsys-cmd route-table set v-device Host route-table <route-table> sys-cmd curl -sv --resolve www.example.com:443:192.0.2.21 https://www.example.com/ - Backend Service settings: On the Load Balancer → Backend Services → Edit screen, SSL service, Certificate verification and Proxy are compared with what the server expects.
Common Misconceptions
- "502 indicates that the server is down." Failing to connect to the server at all can also produce 502, but the most common result of a down server is 503; 502 is mostly produced by an invalid or incomplete response from the server.
- "Increasing the Buffer size(kB) value solves every 502." It helps only with large response headers; it increases memory usage, and the change restarts the vService.
- "TR7 produces every 502." Another proxy behind the application can also return 502; in this case, Termination state appears as
Normal, and the server's name is written in the Backend service value. - "Termination state also exists in HTTP logs." This column is found only in Debug Mode Logs.
How it is done in the documentation
