What is Connection Limit; what happens to new requests when the limit is reached?

Short answer

Connection Limit is the maximum number of connections a Backend Service can carry at the same time; an integer between 1 and 65535 is entered, it has no default value, and while it is empty no per-server limit is applied. On HTTP and L7 TCP vServices, no new connection is given to a server whose limit is reached, and requests go to servers whose limit is not reached; if all servers are full, requests wait in the queue, and if no slot opens within the Queue Timeout duration (60 seconds in the DEFAULT profile), 503 is returned to the client. There is no queue on a Network vService; a server whose limit is reached does not receive new connections (the limit is not applied while Port Range is on or while Protocol is ANY - L3).

How Does It Work?

On HTTP and L7 TCP vServices:

1. A new request arrives; the algorithm selects a server.
2. If a server's concurrent connection count is at the limit, the algorithm does not select it and selects another server whose limit is not reached.
3. If there is no server whose limit is not reached, the request enters the queue. A request pinned to a specific server with Persistence waits in that server's queue if that server is full.
4. When a connection closes, the request in the queue is given to the server.
5. If the Queue Timeout expires, the request is terminated without being processed and the client receives 503.

The Queue Timeout value comes from the Timeouts profile bound to the vService; if no profile is selected, the 60 seconds of the DEFAULT profile apply.

Difference Between Connection Limit and vService Limits

	Connection Limit	Max. Total Connections
Where	Backend Service (per server in a Backend Group)	vService Limits profile
What it limits	The concurrent connections going to a single server	The vService's total concurrent connections
Default	None (no limit is applied)	20,000
Scope	Counted separately on each vService	Per vService

On a Network vService

On a Network vService, a server whose limit is reached is considered overloaded and does not receive new connections until its connection count drops; new connections go to the other servers. There is no queue here, and Queue Timeout is not applied either.

While Port Range is on or while Protocol is ANY - L3, Connection Limit is not applied; if a per-server connection limit is needed on these vServices, the limit is applied in the Backend Service's own configuration.

In TR7

• Single Backend Service: Load Balancer → Backend Services → Add or Edit → Details switch → Connection Limit; it is saved with the form button.
• Backend Group: It is per server; it is entered separately in each row of the Backends table (Backend Groups · Connection Limit).
• For an out-of-range or non-integer value, the form gives the warning "Connection limit must be in range 1-65535".
• The same Backend Service on more than one vService: The limit is applied separately on each vService; it is not a total limit.
• Where it is monitored: Requests waiting in the queue are monitored on the Queued Requests card of the vService's Live Stats screen (on a Network vService this card always shows 0); the current sessions per server are in the Backend Services table of the same screen.
• First algorithm: Connection Limit determines the server selection; the first server in the list receives all the traffic until it reaches its limit, then the next one is used.
• While clustering is on and Manual Sync Mode is off (default), the value is transferred to the peer device automatically; the limit is applied on each device to its own traffic.

Common Misconceptions

• "When the limit is reached, the Backup Service takes effect." A full server is not considered unavailable; the backup does not take effect, and requests wait in the queue.
• "When the limit is reached, new connections are rejected immediately." On HTTP and L7 TCP vServices, they first wait in the queue; 503 is returned only when the Queue Timeout expires.
• "A Connection Limit of 0 shuts the server down." The lowest value is 1; to take the server out of traffic, Maintenance Mode is used (Maintenance Mode does not apply to a Network vService; the Backend Service is removed from the vService's Backend Services list).
• "The limit is the number of requests." The limit is the number of concurrent connections.

How it is done in the documentation

• Backend Services · Connection Limit
• Timeouts · Queue Timeout
• vService Limits · Max. Total Connections
• Live Stats · Frontend

Related questions

• How does the queue work; how long do requests wait at most?
• What happens when the connection limit is reached?
• What is the First algorithm used for?