The vService cannot be reached at all; what should I check step by step?
Short answer
First, the error the client sees is distinguished: Connection refused shows that the packet reaches TR7 but nothing is listening on that IP and port, while a timeout shows that the packet does not arrive or the response cannot return. Then the vService status and the Details messages in the preview are read: the vService may be Disabled, it may not be running (vService is down), its frontend IP or interface may be missing or off, or the VIP may be on the peer device. If these are in order, it is checked that DNS returns the correct IP, that the user rules in Firewall do not block the vService port, and whether the packets reach TR7 with tcpdump. If the packet arrives but the handshake does not complete, the problem is in the return route.
Problem: The Connection Cannot Be Established at All
The browser or client cannot connect to the vService: the page times out or the connection is refused immediately. If an HTTP response (503, 502, 504) is received, the vService is running and the problem is in the response itself; this page covers the case in which the connection cannot be established at all.
IP addresses are defined on the device independently of the vService: when the vService is Disabled or not running, the frontend IP continues to respond to ping, and only the ports of the vService are closed (Disable). For this reason, a ping response alone does not show that the vService is running.
Causes and Solutions
| Cause | How to recognize it | Solution |
|---|---|---|
| vService Disabled | Refused; vService gray, vService disabled in Details | The vService is enabled |
| The license stopped the traffic | vService gray, Stopped by the license in Details | A valid license is uploaded; the vService restarts on its own |
| vService not running | Refused; vService red, vService is down and the configuration error summary in Details | The error in the summary is fixed (What does the configuration error(s) warning in the vService preview (DNS Resolution Error, Port binding error) mean; where does Port Collision appear?) |
| Frontend IP, port or protocol mismatch | Refused or TLS error | The frontend row of the vService is corrected |
| Frontend IP or its interface missing or off | Timeout; IP and interface messages in Details | The IP is enabled or added, the interface is fixed (What do I need to do on the network side first to open a vService on an IP in a different subnet / VLAN?) |
| VIP on the peer device or not active on any device | Timeout; VIP <ip> is on slave or VIP <ip> is down for both master and slave in Details | The VIP and the cluster status are examined (What does a status circle split in two mean in the vService list in a cluster?) |
| DNS returns the wrong IP | The test with the IP works, the test with the domain name does not | The DNS record is corrected |
| User DENY rule in Firewall | Timeout; the client's SYN packet is present in tcpdump, no response | The rule is narrowed or deleted |
| Firewall in front of TR7 | The client's SYN packet is not present in tcpdump | The frontend IP and port are allowed on the external firewall |
| Return route missing | The SYN packet arrives, the handshake does not complete | A route to the client network or a default gateway is added to the route table of the vService |
Where Is It Monitored?
| Screen | Path | What it shows |
|---|---|---|
| vService Status | Load Balancer → Load Balancer vService List | The colored status circle of the vService; in a cluster, two halves, one per device |
| Details | Load Balancer vServices → vService → right panel | Status messages and the configuration error summary |
| Interface and IP statuses | Network → Interfaces | Whether the IP is Up, Disabled or Absent; the (M) or (S) suffix of the VIP in a cluster |
| Firewall | Network → Firewall | User rules, auto-generated vService permissions and apply errors |
| tcpdump | vService screen → tcpdump shortcut in the header | Whether the client packets reach TR7 |
Solution: Three Methods
The methods are selected according to the error the client sees; for a timeout, if Method 2 yields no result, Method 3 is applied.
| Your situation | Suitable method |
|---|---|
| The connection is refused | Method 1: vService status and frontend definition |
| Timeout | Method 2: IP, interface, VIP and Firewall |
| Whether the packet reaches TR7 | Method 3: tcpdump and return route |
Method 1: vService Status and Frontend Definition Recommended
- How it works A connection refusal shows that the vService is off or not running, or that the IP and port used by the client are not defined in the vService.
- Where it is opened Load Balancer vServices → vService → Details · vService form → Frontend
- Requirement No additional setting is required.
- vService type HTTP · L7 TCP · Network
Step-by-Step Configuration
A request is sent from an affected client with the frontend IP instead of the domain name:
curl -v http://192.0.2.10:80/
curl -vk https://192.0.2.10:443/Connection refused is examined with this method, Connection timed out with Methods 2 and 3. If an HTTP response arrives, the vService is running; if it does not work with the domain name, the DNS record is checked, and if 5xx is returned, the page for the related error is followed (Users see "503 Service Unavailable" / "No Available Server"; what should I do?).
In the Left Monitor Panel, the vService is selected from the Load Balancer vServices list; the Details tab is opened in the right panel.
vService disabled: the vService has been turned off manually; on the Load Balancer vService List screen, the row is right-clicked and Enable is selected (Enable).vService is down: the vService is not running; the configuration error summary in Details (Port binding error, DNS Resolution Error, SSL certificate file error…) is read (What does the configuration error(s) warning in the vService preview (DNS Resolution Error, Port binding error) mean; where does Port Collision appear?).Stopped by the license: the license has stopped the traffic; the configuration is preserved, and when a valid license is uploaded, the vService restarts on its own.Modifying vService: the configuration is being applied; a short wait is needed.
On the Load Balancer → Load Balancer vService List screen, the vService is selected and the Edit button is clicked.
It is checked whether the IP and port used by the client exist exactly in the Frontend rows, whether the client's port is within the range if a port range is defined, and whether Protocol (HTTP or HTTPS on an HTTP vService) is compatible with the client's connection; the fields are explained in the Frontend sections of the HTTP and L7 TCP vService pages of the docs, and for a Network vService in the Protocol section. On vServices that share the same IP and port, it is checked that the request goes to the correct vService by Host or SNI (Can I define more than one vService on the same IP:port?).
If a change was made, the Save button is clicked. The vService briefly appears in the Evaluating (Modifying vService) status.
Verification
The Details tab is checked again.
The strip is green (OK), no error messageA request is sent again from the client: curl -v http://192.0.2.10:80/
Connected to 192.0.2.10 and an HTTP responseMethod 2: IP, Interface, VIP and Firewall
- How it works A timeout can show that the frontend IP is not active on the device, that the VIP is on the peer device, or that a Firewall rule drops the packets.
- Where it is opened Details · Network → Interfaces · Network → Firewall
- Requirement No additional setting is required.
- vService type HTTP · L7 TCP · Network
Step-by-Step Configuration
In the Details tab of the vService, the <ip> interface <interface> is absent, <interface> interface <ip> IP is absent, <ip> interface <interface> is down or … link is down messages are looked for. If no working frontend IP is left, the vService appears red (Error).
On the Network → Interfaces screen, it is checked that the frontend IP is in the list and that its IP status is Up. If the IP is Disabled, Enable for is selected by right-clicking the IP; if the IP has been deleted, it is added again. If the interface is Down or No link, the interface and the cabling are fixed.
In a cluster setup, on the Interfaces screen, it is checked on which device the VIP appears with the (M) suffix; the VIP receives traffic only on the device where it is Master (What is the Master/Slave role on a VIP?). If there is an unexpected transition, the VIP transition mechanism is examined.
nslookup www.example.com is run from the client; it is verified that the returned IP is the frontend IP of the vService.
On the Network → Firewall screen, the route table of the vService is selected, and it is checked whether there is a broad user DENY rule that also covers the vService port. In incoming traffic, the user ALLOW and DENY rules are evaluated before the automatically generated vService listening permissions; packets that do not match are dropped (Rule evaluation order). It is checked that the "<vService-name>" TCP rule permission for the vService appears in the Auto-generated Rules strip below the list and that there is no red errors counter above the list; these rules are listed in the Auto-generated rules section of the docs.
Verification
The row of the frontend IP is checked on the Network → Interfaces screen.
IP status Up; in a cluster, the VIP on this device with the (M) suffixcurl -v http://192.0.2.10:80/ is run again from the client.
The connection is establishedMethod 3: tcpdump and Return Route
- How it works Whether the packet reaches TR7 and whether the handshake completes distinguishes whether the problem is in TR7 or in the network.
- Where it is opened vService screen → tcpdump shortcut in the header · WEB-CLI
- Requirement No additional setting is required.
- vService type HTTP · L7 TCP · Network
Step-by-Step Configuration
Left Monitor Panel → Load Balancer vServices → the vService is selected; the tcpdump shortcut in the header of the right panel is clicked (Can I capture packets on TR7 and download them as a pcap (tcpdump)?).
The IP address of the client is written in the Client IP Filter field.
The Start button is clicked and the client tries to connect.
If no packets come from the client at all, the problem is in the firewall or router in front of TR7; no change is required on the TR7 side. If the SYN packet arrives but the handshake does not complete (the client repeats the SYN), TR7's response cannot return to the client.
In WEB-CLI, first the route table of the vService is selected (Host is written in the v-device field for the device itself, and the route table name is taken from the Tab list), then a ping is sent to the client (sys-cmd route-table set, sys-cmd ping):
sys-cmd route-table set v-device Host route-table <route-table>
sys-cmd ping -c 3 203.0.113.45If there is no response, a route to the client network or a default gateway is missing in the route table (What is a Route Table; when do I need to change it for a vService or Backend Service?).
On a Network vService, if LB Mode is Direct Server Return, loopback preparation is required on the servers; if it is Route Mode, the gateway of the servers must be TR7 (What is Direct Server Return (DSR); what preparation is needed on the server side?, Why does return traffic have to flow through TR7 in Route Mode?).
Verification
The tcpdump output is checked.
SYN, SYN-ACK and ACK appearCommon Mistakes
- Assuming that the vService is running because a ping response is received. The IP stays on the device independently of the vService; ping is answered even while the vService is Disabled. To turn off the IP, the IP is disabled in Network → Interfaces.
- Examining TR7 without testing DNS. The domain name may be resolving to another IP.
- Not checking on which device the VIP is in a cluster. The VIP receives traffic only on the device where it is Master.
- Writing a broad DENY rule in Firewall that also covers the vService port. User rules are applied before the automatic vService permissions.
- Testing in WEB-CLI without selecting a route table. A destination in an isolated route table appears unreachable when it is tested from the default route table.
- Asking for support without collecting information if the problem persists. What to add to the support ticket: Which information should I collect before contacting TR7 support?
How it is done in the documentation
Related questions
- What does the configuration error(s) warning in the vService preview (DNS Resolution Error, Port binding error) mean; where does Port Collision appear?
- Can I capture packets on TR7 and download them as a pcap (tcpdump)?
- Which IP address can I use for a vService; what is the difference between an Interface IP and a VIP?
