How do I see the health check history (when a server went down/up)?

Short answer

The fastest way is the Health Checks tab in the Backend Service preview: records are listed with the time, the status (OK, Error, Preparing) and the error summary; in the enlarge window, they are filtered and exported with Export. The uptime rate over time is seen in the Backend Service Dashboard's Health Check State panel, the summary of the last day in the Left Monitor Panel, and status change notifications in Notification Logs; the records are also forwarded to the Syslog server with Log Forwarding. For the records to be kept, Enable HC Logs must be on in the Backend Service (on by default).

Problem: When and Why the Server Went Down Is Unknown

After user complaints, the most frequently asked question is whether the server was removed from traffic at that moment, and why. The Health indicator in the Backend Services list shows only the current status; to see the time, duration and error type of past drops, the health records or the time series charts need to be checked.

Solution: 5 Methods

The methods complement each other: record details are in Method 1, the time series in Method 2, the aggregate summary in Method 3, notifications in Method 4, and forwarding to an external system in Method 5.

Your situation	Suitable method
I want to see and export the status transitions and error texts of a single Backend Service	Method 1: Health Checks tab
I want to see the uptime rate and the check duration over time in a chart	Method 2: Dashboard → Health Check
I want to see the health summary of all Backend Services for the last day	Method 3: Left Monitor Panel
I want to receive status changes as notifications and list them retrospectively	Method 4: Notification Logs
I want to store the records on a SIEM or Syslog server	Method 5: Log Forwarding

Method 1: Health Checks Tab Recommended

• How it works The health records of the Backend Service are listed from newest to oldest; a record is written when the status, error or result information changes, and once in every 5-minute slot. In a cluster, the records of the two devices are listed together.
• Where it is openedLoad Balancer → Backend Services (or Backend Groups) → row → preview → Health Checks
• Requirement A Health Check profile assigned to the Backend Service and Enable HC Logs being on
• vService type All

Step-by-Step Configuration

1

Adım 1 Open the preview

On the Load Balancer → Backend Services screen, the Backend Service row is clicked; the preview opens on the right. The same preview is also found in the Backend Service Dashboard opened from the Left Monitor Panel.

2

Adım 2 Open the tab

The Health Checks tab is clicked in the preview. The tab is displayed only if a profile is assigned.

3

Adım 3 Set the range

Date (default Last 24 hour(s)) and Log Limit (default 50) are selected; the switch is set to Only Change Logs (default) or All Logs (Only Change Logs).

4

Adım 4 Refresh the list

The Refresh button is clicked; Date and Log Limit changes are applied with it.

5

Adım 5 Open the enlarge window

The enlarge icon in the tab is clicked; the records open as a table in the window titled Health Check Logs For. The table has the Date, State and Summary columns; in a cluster, the Source column, and in a Backend Group with two or more servers, the Address column.

6

Adım 6 Filter and fetch

Limit (default 1000), in a cluster Source (default Both devices) and Only Change Logs are selected, and the Get Logs button is clicked.

7

Adım 7 Export

The Export button is clicked; the table is downloaded as Excel, CSV or PDF.

Verification

1Status transitionsHealth Checks tab

Records are separated by day headings; each record contains the time, the status and the summary text (check type, error, duration).

Expected14:02:11 Error · HTTP / Request timeout error: 1000ms

Method 2: Dashboard → Health Check

• How it works In the Health Check tab of the Backend Service Dashboard, the Health Check State panel shows the Up/Down status with color bands and the uptime percentage in the range, and the Health Check Time panel shows the check duration. The data is stored on the device; it is updated even while Enable HC Logs is off.
• Where it is opened Left Monitor Panel → Backend Services → Backend Service → Dashboard → Health Check
• Requirement A Health check profile assigned to the Backend Service
• vService type All

Step-by-Step Configuration

1

Adım 1 Open the Dashboard

The Left Monitor Panel is opened; Backend Services is selected in the category selector and the Backend Service is clicked (Accessing the Dashboard); servers that are members of a Backend Group are listed in the Backend Service Groups category. The Dashboard is also opened with the Dashboard button in the header of the preview in the Backend Services list.

2

Adım 2 Open the Health Check tab

Health Check is selected from the view tabs.

3

Adım 3 Select the time range

The range is selected from the time range selector at the top right (default Last 1 day(s)); a range is also selected by dragging horizontally on the panel.

Verification

1Uptime rateHealth Check State panel

Up/Down bands and the uptime percentage are displayed in the selected range; in a cluster, each device is drawn separately.

Expectede.g. 99% Up

Method 3: Left Monitor Panel

• How it works The Backend Services list in the Left Monitor Panel shows the health status of the Backend Services that are not members of a Backend Group and their uptime rate in the selected period (default the last 1 day, e.g. 100% Up); if the value is below 100% Up, there was a drop within the period. Backend Group members are listed in the Backend Service Groups category.
• Where it is opened Left Monitor Panel → Backend Services
• Requirement None
• vService type All

Step-by-Step Configuration

1

Adım 1 Open the list

The Left Monitor Panel is opened; Backend Services is selected in the category selector.

2

Adım 2 Examine the status strip

The colored strip to the left of each row shows the health status, and the Avg. Value column shows the uptime rate within the period; the status strip at the top of the list counts the Backend Services by their status. The period is changed with Display Period in the View Settings window.

Verification

1SummaryLeft Monitor Panel

The health averages of the Backend Services within the period are listed.

ExpectedBelow 100% Up = a drop within the period

Method 4: Notification Logs

• How it works Backend Service status changes are produced as notifications and listed in Notification Logs; for this, a notification profile must be selected on the vService and Backend Service Status must be on in the profile.
• Where it is openedSettings → System Logs → Notification Logs
• Requirement A notification profile on the vService (HTTP and L7 TCP vServices)
• vService type HTTP, L7 TCP

Step-by-Step Configuration

1

Adım 1 Open the vService form

If the notification is not on, the vService is right-clicked on the Load Balancer → Load Balancer vService List screen and Edit is selected.

2

Adım 2 Turn on the notification

Notification is checked in the Details panel; the built-in Default Profile comes selected and produces Backend Service status notifications. If a different threshold or channel is required, a profile created on the Load Balancer → Profiles → Notifications screen is selected (How is it added to a vService?).

3

Adım 3 Save the vService

The vService form is saved.

4

Adım 4 Fetch the records

On the Settings → System Logs → Notification Logs screen, Backend service status notifications are selected with Filter and the Get Logs button is clicked (Notification types).

Verification

1NotificationsNotification Logs

The status changes of the Backend Services are listed with timestamps.

ExpectedDown / up notifications

Method 5: Log Forwarding

• How it works Health records are forwarded with Syslog to an external server (SIEM, log collection system); while Log All Health Checks is off (default), only status changes are forwarded, and while it is on, all the health records written on TR7 (status, error or result changes and the 5-minute periodic records) are forwarded. The records of Backend Services with Enable HC Logs off are not forwarded.
• Where it is openedSettings → General Settings → Log Forwarding → Log Sources
• Requirement A log collection system that accepts Syslog; a route table that reaches this system from TR7
• vService type All

Step-by-Step Configuration

1

Adım 1 Open the Log Forwarding form

The Configuration Menu is opened with the TR7 logo at the bottom left; the Settings → General Settings → Log Forwarding path is followed and Add is clicked; for an existing profile, the profile is selected and Edit is clicked (How to add?).

2

Adım 2 Enter the destination

Name (unique) is entered; Transport Type stays Syslog, and the Route Table through which the forwarding goes out is selected. Syslog Server (required; IP or domain name), Port (default 514) and Protocol (default UDP) are entered.

3

Adım 3 Select the source

In the Log Sources field, Health Check Logs is checked (it does not come selected by default); the Log All Health Checks switch that appears is turned on only if error and result changes and the 5-minute periodic records are also to be forwarded in addition to status changes.

4

Adım 4 Save

The Add (Edit when editing) button is clicked.

Verification

1ForwardingLog collection system

When the status of a Backend Service changes, the health record coming from TR7 appears in the log collection system.

ExpectedStatus change records

Common Mistakes

• Searching for history while Enable HC Logs is off. If this setting is off, no new records are kept and forwarding with Log Forwarding stops; the Dashboard charts and the Health indicator are still updated (What does Enable HC Logs do; should I turn it off?).
• Assuming every check is recorded. A record is written when the status, error or result information changes, and on the first result of every 5-minute slot.
• Expecting history on a Backend Service with no profile assigned. Without a health check, no history is created; the Health Checks tab is not displayed either.
• Looking for the group status in the records. In a Backend Group, records are kept separately for each server; the Partially Healthy status in the preview is not written to the records.

How it is done in the documentation

• Health Check Logs · Health Checks Tab
• Health Check Logs · Health Check Logs Listing Screen
• Backend Services Dashboard · Health Check State
• Notification Logs · Notification Types
• Log Forwarding · Log Sources

Related questions

• Can I see only status changes in the health check logs (Only Change Logs / All Logs)?
• What does Enable HC Logs do; should I turn it off?
• The server is running, but TR7 shows it as down; why?
• What does "Partially Healthy" mean on a Backend Group dashboard; how do I find which server in the group has a problem?