What does a vService started in Debug Mode do; why is it only available in the HTTP vService, and what is the downside of leaving it on in production?
Short answer
An HTTP vService in Debug Mode, while continuing to process traffic, records the headers of each request and response, the first 16 KB of their bodies, and the timing and TLS information; the records are examined on the Debug Mode Logs screen. In the Debug box in the Details panel, recording can be narrowed to specific requests with a condition, and with Custom Debug Log Entries, additional Smart Variable-based fields can be written to each record. Since HTTP is the only type that can read the request as HTTP, this mode exists only in the HTTP vService. It is not left on in production: passwords, session keys and personal data can be recorded along with the bodies, an additional processing load is placed on the vService, and since the records are kept in a 128 MB space per vService, they are deleted quickly.
How Does It Work?
- Enabling: Debug Mode is selected in the Status field at the top of the vService form. The Debug box in the Details panel is the same switch: when the box is checked, Status switches to Debug Mode, and when Debug mode is selected in Status, the box is checked.
- What is recorded: The headers of the request and the response and the first 16 KB of the bodies, TLS information, timing values and the termination reason of the request. The headers TR7 adds are marked green, the ones it removes red, and the ones it changes yellow.
- Conditional debug: If a condition (Conditions) is selected in the Select Conditions field in the Debug box, only the requests that match the condition are recorded (e.g. a single test IP, a specific path). If no condition is selected, all requests are recorded.
- Custom debug log entries: When the Custom Debug Log Entries switch below the Debug box is turned on (off by default), an Entry Name and an Entry Value are entered in each row. Entry name is required and is the name of the field in the record; Entry value is a Smart Variable selector (e.g. the client address, a header or cookie value). These values are listed in the Custom Entries table in the Summary tab of the record's preview.
- Viewing: Left Monitor Panel → Load Balancer vServices → vService → Logs & Tracking → Debug Mode Logs (Viewing Debug Logs).
How Are Conditional Debug and Custom Debug Log Entries Set Up?
On the Load Balancer → Load Balancer vService List screen, the HTTP vService is selected and the Edit button is clicked.
The Details switch at the top right of the form is turned on and the Debug box is checked; Status automatically becomes Debug Mode.
In the Select Conditions field, the condition that selects the requests to be recorded is selected (e.g. a Source IP condition that selects the IP of the test client). If there is no condition, it is first created with Load Balancer → Conditions → Add (Conditions).
The Custom Debug Log Entries switch is turned on; a row is added with Add, a name (e.g. real_client) is written in the Entry Name field, and a value is selected from the Entry Value field. A new row comes with the first variable not used yet.
The Save button is clicked. When the examination is finished, the vService is edited again, Status is set to Enabled, and it is saved; Debug mode does not turn off on its own.
Why Only in HTTP?
Debug recording relies on parsing the request as the HTTP protocol and reading its headers and body. L7 TCP does not read the stream as HTTP, and Network does not terminate the connection at all; in these types, there is no HTTP request to record. In L7 TCP and Network vServices, only Enabled and Disabled appear in the Status field; in L7 TCP, connection details are followed on the TCP Logs screen.
Downsides of Leaving It On in Production
| Downside | Description |
|---|---|
| Privacy | Login forms, API keys, session cookies and personal data are recorded along with the bodies and headers. |
| Performance | The detailed record of each request increases CPU, memory and disk usage; under heavy traffic, the log volume grows quickly. |
| Short-lived records | There is no adjustable retention period for Debug records; when the 128 MB space per vService fills up, old files are deleted and the request being looked for can be lost. |
For this reason, Debug mode is turned on while looking for a problem, narrowed with a condition if possible, and turned off when the work is finished.
Common Misconceptions
- "Debug mode stops traffic." The vService continues to process traffic; only detailed recording is added.
- "Debug records are sent to the log server." Debug records are kept on TR7; with a Logs profile, only HTTP and TCP records are forwarded outside.
- "Custom debug log entries change the value of a ready-made field." The defined fields are kept in the record in a section separate from the ready-made fields; even if they carry the same name, they do not replace the ready-made field.
- "I can turn it on once and forget it." Debug mode does not turn off on its own; when the problem is solved, Status is set back to Enabled.
How it is done in the documentation
