Block Bytedance and Bytespider Bots on Apache 2 with ModSecurity

Reasons to block Bytedance and Bytespider Bots:

1. They do not respect robots.txt rules.
2. They are using well-known hosting services to bypass normal blocking channels or methods. For example, they are using AmazonAWS services to crawl websites and mask (or re-route) their origin IPs.
3. Their crawling rates are extremely high.
4. Typically, the bots’ origin IPs geolocation is China. When blocking the bots’ User Agents, the origin IPs geolocation changes to Singapore (another haven for malicious bots or bad actors).

Source:
wordpress.org

ModSecurity is an open-source web-based firewall application (or WAF) supported by different web servers: Apache, Nginx and IIS.

Open file modsecurity.conf
/etc/modsecurity/modsecurity.conf

Add the following code at the end, with a blank line at the end.

SecRule REQUEST_HEADERS:User-Agent "@contains bytedance" "id:'1000000',t:none,t:lowercase,deny,nolog"
SecRule REQUEST_HEADERS:User-Agent "@contains bytespider" "id:'1000001',t:none,t:lowercase,deny,nolog"

Then run the following commands to restart Apache.

sudo apache2ctl configtest
sudo systemctl reload apache2