How do I reduce log volume (silencing health check and static file logs)?

Short answer

To leave specific requests out of the lines that go to the log server, the Conditional Silent Log action is used; Path conditions are defined for monitoring paths such as /health and /status or for static files such as .css, .js, .png, and the action is added to the HTTP vService with these conditions. To reduce the total volume, Sampling is turned on in the log addresses in the Logs profile; in this case, only one of every 10 logs is sent to that address. Standard Log Format, which produces simpler lines, or TR7 WAF Log, which sends only WAF blocks, also lowers the volume. These methods target the lines that go to the log server; the HTTP logs records on the device are not affected. The health checks that TR7 performs on the Backend Services do not fall into the vService logs; their record is managed with Enable HC Logs in the Backend Service.

Problem: The Log Server Fills Up with Unnecessary Records

The /health requests that monitoring systems send continuously and the dozens of static file requests per page consume the capacity and license volume of the log server; meaningful records get lost in the noise.

Solution: Three Methods

The methods are not alternatives to each other; they can be used together.

Your situation	Suitable method
Silencing specific paths or clients	Method 1: Conditional Silent Log
Reducing the total volume going to a log server proportionally	Method 2: Sampling
A smaller line or only WAF blocks	Method 3: Format selection

Method 1: Conditional Silent Log Recommended

• How it works The line of the requests that match the condition is not sent to the log server; other requests are sent normally. The HTTP logs records on the device are not affected.
• Where it is openedLoad Balancer → Actions → Action Type = Conditional Silent Log
• Requirement LOG on in the vService and a Logs profile with a log server address selected
• vService type HTTP

Step-by-Step Configuration

1

Adım 1 Open the condition form

On the Load Balancer → Conditions screen, the Add button is clicked (how to add).

2

Adım 2 Define the Path condition

Condition Name is entered, and Criteria = Path is selected. In the matching list below Criteria, Exact Match (default) is selected and Query = /health is entered for the monitoring path; for static files, End is selected and Query = .css is entered. Each value is a separate condition; multiple extensions can also be written in a single condition with Regex as \.(css|js|png|jpg)$; if uppercase extensions such as .CSS are also to be caught, Case Insensitive is checked. To silence only the requests of the monitoring systems, a Source IP condition is also created.

3

Adım 3 Save the condition

The Add button is clicked; if multiple conditions are needed, the form stays open with Add & Repeat.

4

Adım 4 Open the action form

On the Load Balancer → Actions screen, the Add button is clicked.

5

Adım 5 Define the action

Action Name is entered, and Action Type = Conditional Silent Log and Execution Level = On request are selected. In the Conditions field, the conditions are joined with or; if the Source IP condition is also to be used, it is joined with the path conditions with and. If Conditions is left empty, all the lines of the vService are silenced. To count the silenced requests, Hit counter can be turned on.

6

Adım 6 Save the action

The Add button is clicked.

7

Adım 7 Open the vService edit form

On the Load Balancer → Load Balancer vService List screen, the HTTP vService is right-clicked and Edit is selected.

8

Adım 8 Add the action to the vService

The action is selected in the Actions field in the Details panel. The action does not work unless it is added to the vService.

9

Adım 9 Save

The Save button is clicked. The progress of applying the change is followed from the sync indicator at the bottom right of the screen. In a cluster setup, with the default auto-sync, the change is also sent to the peer device; if Manual Sync Mode is on, it is sent manually with Synchronize.

Verification

1Log serverLog server

The lines of the /health requests are searched for.

ExpectedNo line arrives

2CountTR7 side

If Hit counter is on, the number of silenced requests is seen in the Action Hits tab of the vService dashboard (Can I see how many times an action has been triggered (hit counter, Action Hits)?).

ExpectedIncreasing counter

Method 2: Sampling

• How it works Only one of every 10 logs is sent to a log address on which Sampling is on; the entered value determines which line in each set of ten is sent. It is set separately for each log address.
• Where it is openedLoad Balancer → Profiles → Logs → Log Addresses → Sampling
• Requirement A Logs profile assigned to the vService
• vService type HTTP · L7 TCP

Step-by-Step Configuration

1

Adım 1 Open the Logs profile

On the Load Balancer → Profiles → Logs screen, the Edit button is clicked for the profile that the vService uses. If there is no profile yet, it is first created with the steps on the How do I send logs to a SIEM (Syslog)? page and assigned to the vService.

2

Adım 2 Turn on Sampling

The Sampling switch in the Log Addresses row is turned on (default off; even while it is on, its label stays Sampling disabled) and a value in the 1–10 range is entered; a value greater than 10 does not produce valid sampling. In the same profile, full sending can be done to one address and sampled sending to the other (high-traffic environments).

3

Adım 3 Save the profile

The Edit button is clicked. The change is applied immediately to all the vServices that use the profile.

Verification

1Log serverLog server

The number of incoming lines is compared with the number before Sampling.

ExpectedAbout one tenth

2Profile listTR7 side

In the Logs profile list, the Log Addresses column is checked.

Expected192.0.2.50:514 (Sampling: 1)

Method 3: Format Selection

• How it works The line size varies according to the format; Standard Log Format carries the fewest fields, and TR7 WAF Log writes only the requests that the WAF blocks.
• Where it is openedLoad Balancer → Profiles → Logs → Line Format
• Requirement A Logs profile assigned to the vService; for TR7 WAF Log, an HTTP vService with WAF enabled
• vService type HTTP · L7 TCP (TR7 WAF Log only HTTP)

Step-by-Step Configuration

1

Adım 1 Open the Logs profile

On the Load Balancer → Profiles → Logs screen, the Edit button is clicked for the profile.

2

Adım 2 Change the format

Line Format = Standard Log Format or, for security events only, TR7 WAF Log is selected. TR7 WAF Log produces no lines at all on a vService on which WAF is not enabled (Which HTTP and TCP log formats are supported (Standard, Apache Combined, Apache VHost, CEF, TR7 JSON, TR7 WAF, Manual JSON, Manual CEF, Manual log line)?).

3

Adım 3 Save the profile

The Edit button is clicked.

Verification

1Log serverLog server

The lines coming from the vService are checked.

ExpectedLines arrive in the new format

Common Mistakes

Leaving Conditional Silent Log without conditions

If the Conditions field is empty, the action runs on every request and silences all the lines of the vService that go to the log server.

• Looking for the action in an L7 TCP vService. Conditional Silent Log works only in an HTTP vService.
• Binding it to a condition that forms late. In conditions that depend on information formed later, such as the bot score or the WAF result, requests that are terminated earlier may still go to the log server; conditions evaluated at the start of the request, such as path, source IP and header, are preferred.
• Writing a value such as 100 in Sampling. The ratio is fixed at one in 10; the value must be in the 1–10 range.
• Expecting the HTTP logs screen on the device to be silenced too. The methods target the lines that go to the log server; the records on the device are already limited by the 256 MB vService budget (How long are logs kept on the device?).
• Looking for the Backend Service health checks in the vService log. The record of these checks is kept with Enable HC Logs in the Backend Service.

How it is done in the documentation

• Conditional Silent Log · Scenario 1: Leaving Health-Check Requests Out of the Logs
• Conditional Silent Log · Scenario 3: Leaving Static File Requests Out of the Logs
• Logs profile · Log Addresses
• Logs profile · High-Traffic Environments

Related questions

• How do I send logs to a SIEM (Syslog)?
• How long are logs kept on the device?
• Can I see how many times an action has been triggered (hit counter, Action Hits)?