What is a bot; how is the difference between a good bot and a bad bot handled in TR7?
Short answer
A bot is a client that sends requests through automation instead of a human. Just as there are beneficial bots such as search engine crawlers and monitoring tools, there are also bad bots that scrape content, try credentials or generate fake traffic (OWASP Automated Threats). In TR7, this distinction is made by Bot Protection on vServices with Bot/DDoS protection on: it gives every request a bot score in the 0–100 range from IP reputation, header and TLS fingerprint, protocol and behavior signals, and Action Rules show a CAPTCHA or block according to the score. Crawlers whose address is in the IP range published by the search engine itself are recognized separately as Verified Bot and are exempted from the bot rules with Trust Verified Bots (on by default). Load Balancer rules have the Bot Type condition, which looks at the User-Agent, and the IP reputation database has the Bad Web Bot category for known bad bots.
Where It Is Used in TR7
| Where | Field / label | What for |
|---|---|---|
| Left Monitor Panel → Bot/DDoS Protected vServices → vService → Bot/DDoS Management | Bot Protection card: Factors, Action Rules | Selecting the signals that produce the bot score and defining actions according to the score |
| Bot Protection → Edit | Trust Verified Bots | Exempting verified search engine crawlers from the bot rules |
| Left Monitor Panel → Bot/DDoS Protected vServices → vService → Dashboard | Bot tab | Monitoring the bot score distribution and the classes |
| Configuration Menu → Load Balancer → Conditions (Policy Match) | Bot Type | Matching the User-Agent against known bot categories (Search Engines (Googlebot, Yahoo, bingbot etc.), Malicious bots …); HTTP vService only |
| Configuration Menu → Load Balancer → Conditions (Policy Match) | Network Type Group → Verified Bot | Distinguishing real search engine crawlers by source address |
| Configuration Menu → Settings → General Settings → IP Intelligence & Reputation | IP Categories → Bad Web Bot | Addresses reported for bad bot behavior |
- Score classes: What is a bot score; what do the Clean, Suspicious, Bot and Bad Bot classes mean?
- Verified bots: What is a Verified Bot; how does TR7 treat verified search engine crawlers?
- Configuration steps: How do I detect bots; can I allow good bots (Googlebot etc.) and block bad ones?
Common Misconceptions
- "All bots should be blocked." Search engine and monitoring bots are legitimate; while Trust Verified Bots is on, verified crawlers are not caught by Bot Protection rules.
- "Looking at the User-Agent is enough to recognize a bot." The User-Agent is easily spoofed. The Bot Type condition looks only at the User-Agent; Bot Protection, by contrast, produces the score from IP reputation, TLS fingerprint, protocol and behavior signals together, alongside the User-Agent.
- "Bot Protection blocks bots as soon as it is turned on." On an HTTP vService, Bot Protection starts in Monitor mode when protection is turned on; decisions are only recorded (starting modes).
How it is done in the documentation
Related questions
