I set things up in two stages: htaccess files that look at the user agent and block terms and browser versions, as well as IPs/CIDRs for the scoundrels I find. Unfortunately, I have to use /24 CIDRs (a block of 256 IPs) for everything found on the European site, because they mask the last digit of the IP octet in the logs iaw EU General Data Protection Regulations. The htaccess file gets the ones that come in 'direct' by IP that the CDR doesn't handle. Then I have a similar set of blocks on the CDR, which also has some of it's own rules. What's good about this system is that *most* people update their browsers and the bad guys usually use old versions of things. For example, the Chinese stuff above was all MSIE 9.0. There's a set of bots that routinely use Chrome 51, Chrome 52, Safari 9.1.2, MSIE 9.0, or nothing at all (blank string) - all are immediate blocks. If a real person gets blocked, the 403 page gives them the date, their IP, their UA string, and tells them to upgrade their browser... if they actually read it. Some don't. If they come in with a recent version string they'll get in unless I've blocked the IP previously. 90% of the time it's a good stopgap and better than nothing until I get the IP denied.
... View more