How does the queue work; how long do requests wait at most?

Short answer

If a Connection Limit is defined on the Backend Service and all servers have reached their limits, new requests wait in the queue; when a connection is closed, the next request in line is forwarded to the server. A request waits at most as long as the Queue Timeout in the Timeouts profile (default 60 seconds); if the time is exceeded, the client receives 503. If no Connection limit is defined, no queue forms.

How Does It Work?

1. The algorithm skips the server whose Connection limit is full, and the request goes to the other servers.
2. If no server with an unfilled limit remains, the request enters the queue; a request bound to a server with persistence waits in that server's queue.
3. When a connection is closed on one of the servers, the next request in line is forwarded.
4. If its turn does not come before the Queue timeout expires, the request is terminated without being processed and 503 is returned to the client (Queue timeout behavior).

The Connection limit side of the queue mechanism: What is Connection Limit; what happens to new requests when the limit is reached?

Monitoring

Screen	Indicator
Live Stats → Frontend	Queued Requests card; the average Queue Wait in the tooltip of the Wait/Connect card in the Backend Services group
Live Stats → Backend Services	Queue Wait per Backend Service
Backend Services Dashboard	Queue Depth
vService Dashboard	Backend Queue Distribution
Debug Mode Logs	While the HTTP vService is in Debug mode, for requests whose time expires in the queue, Termination State shows the queue stage

Queue in TR7

• Setting: Queue timeout is set between 0 and 60,000 seconds on the Load Balancer → Profiles → Timeouts screen and is selected in the Details → Timeout box of the vService (how to add to a vService). The change is applied without dropping open connections. In a cluster, it is also sent to the peer device with the default auto-sync; if Manual Sync Mode is on, it is sent with Synchronize.
• vService type: The queue forms in HTTP and L7 TCP vServices; in a Network vService, the Queued requests card is always 0.
• A different wait: When Max. Total Connections in vService Limits is full, connections do not enter this queue; they are held at the entry of the vService without being accepted, and Queue timeout is not applied (What happens when the connection limit is reached?).

Common Misconceptions

• "When the queue fills up, the Backup service takes effect." The Backup service takes effect when the main servers cannot be used; the limits being full is not this situation.
• "Queue timeout is applied to every request." It applies only to the requests that enter the queue.
• "Increasing the Queue timeout solves the problem." The user waits longer; the permanent solution is to increase the capacity (the number of servers or the Connection limit).

How it is done in the documentation

• Timeouts · Queue Timeout
• Backend Services · Connection Limit
• Live Stats · Frontend

Related questions

• What is Connection Limit; what happens to new requests when the limit is reached?
• What happens when the connection limit is reached?
• What do the values in the Timeouts profile (HTTP Keepalive Timeout, HTTP Request Timeout, Connect, Server, Client, Queue, Tunnel(Websocket) Timeout, Client FIN, Server FIN) mean?