The site became slow behind TR7; where should I start?
Short answer
First, the scope and the timing of the slowness are clarified: everyone or a specific page, at peak hours or after a change. Then, the Backend Services table in Live Stats is checked for a Backend Service that has dropped out; the remaining servers may be carrying the whole load. After that, it is measured whether the delay occurs in TR7 or in the Backend Service: The site is slow; how do I tell whether the problem is in TR7 or in the backend? If the result points to TR7, the vService Limits, Connection Limit and queue, QoS and bandwidth settings are examined; if it points to a single server, the distribution and that server are examined.
Problem: The Site Is Slow, and It Is Not Known Where to Start
Users report that pages open slowly. The slowness can come from the Backend Service, from a limit in TR7, from lost capacity or from the network; to look in the right place, the checks are done in a specific order. When the vService Limits values are reached, new connections are not rejected but held; this appears as slowness from the outside (What happens when the connection limit is reached?).
Where Is It Monitored?
| Screen | Path | What it shows |
|---|---|---|
| Live Stats → Backend Services | Load Balancer vServices → vService → Live Stats | Status strip, Current Sessions, HTTP Request/s, Avg. response time per Backend Service |
| Live Stats → Frontend | In the same tab | Queued requests and the vService-wide request and connection values |
| Dashboard | vService → Dashboard | Graphs over time such as Backend Response Time Distribution and vService CPU Usage |
| Debug Mode Logs · Request Tab | vService → Logs & Tracking (while in Debug Mode) | The Performance Info values (Tc, Tr, Tw) in the Request and Response tabs of a single request |
Solution
The checks are done in the order of scope, capacity and measurement; according to the measurement result, the related setting is moved on to.
Diagnosis Flow
- How it works The flow proceeds in the order of scope, capacity and measurement; according to the measurement result, the related setting is moved on to.
- Where it is openedLive Stats · Dashboard · Debug Mode Logs
- Requirement For the time breakdown of a single request, an HTTP vService and Debug Mode.
- vService type HTTP · L7 TCP · Network
Step-by-Step Configuration
It is determined whether the slowness started for everyone, for specific clients, on a specific path, at peak hours, or after a change. If it started after a change, first a Debug Mode left on, and QoS, vService Limits, bandwidth, SSL security profile and algorithm changes are checked; who made the change and when is seen in the Log tab of the vService preview.
In the Left Monitor Panel → Load Balancer vServices → vService → Live Stats tab, the Backend Services table is checked: the status strip, Current Sessions, HTTP Request/s (New Connection/s on L7 TCP), Avg. Response Time of each Backend Service and the weight suffix next to the name (W=) are seen. In a cluster, each device shows only its own traffic (What does the Live Stats screen show?).
If there is a Backend Service with a red (Unhealthy) or yellow (Maintenance, Draining, Not balancing) strip, the remaining servers are carrying the whole load: A backend shows as down; how do I diagnose it?
It is measured whether the delay occurs in TR7 or in the Backend Service: The site is slow; how do I tell whether the problem is in TR7 or in the backend?
| Measurement result | Where to look |
|---|---|
| The server time is dominant, and the direct measurement is also slow | The application; relief on the TR7 side: Which features does TR7 offer to speed up my application? |
| The Avg. Response Time value of a single server differs | Traffic is not distributed evenly across the servers; how do I investigate? |
| The queue time is dominant, Queued Requests is greater than zero | How does the queue work; how long do requests wait at most? |
| It increases at peak hours, the limit values are low | What is the vService Limits profile; what are the default limits? |
| The vService CPU or memory usage is high | What is QoS; how do I reserve dedicated CPU/memory resources for critical vServices? |
| Large downloads stay at a fixed speed | Can I limit bandwidth (download / upload)? |
| The servers go down and up from time to time | The health check keeps going back and forth between up/down (flapping); what should I do? |
| Long operations are cut off with a 504 at the same second | I get 504 Gateway Timeout; which timeout should I check? |
| The name resolution time is long | DNS; it is outside TR7 |
Verification
After the fix, Avg. Response Time and Queued requests are monitored in Live Stats at peak hours.
Avg. response time drops, Queued requests 0The same page is measured again: curl -o /dev/null -s -w '%{time_total}\n' https://www.example.com/
The duration returns to its usual valueCommon Mistakes
- Looking at the average and missing the single slow server. The Backend Services table is read per server.
- Looking at the screen of only one device in a cluster. Each device shows its own traffic; the values are not summed.
- Removing the limits completely. The limits are there to protect the vService; they are brought to the right level (recommended values).
- Leaving Debug Mode on. Debug Mode keeps a detailed record for each request and does not turn off on its own (What does Debug Mode record; how long should I keep it on?).
- Asking for support without collecting information if the problem persists. What to add to the support ticket: Which information should I collect before contacting TR7 support?
How it is done in the documentation
