Edit: I didn't see your comment about budget. I expect Akamai may be out of reach, not sure about Cloudflare's options. Most bot detection is going to need to finger print behavior of the interaction to the site (Captcha as well). If that data is handled correctly, (not being sold/made available to a third party/destroyed after use), I believe it can be done ethically. Obviously my ethics are not yours.
Just FYI - bot management is basically snake oil. Distil can be circumvented by automating Chrome headful. Akamai doesn't even require browser automation. Can't speak to CF, but it's probably not different.
If your script kiddie is automating browsers and using residential proxies, he's probably sophisticated enough to get around this stuff. You're not missing anything.
hey there, i work for ipinfo.io and we have a privacy detection API (https://ipinfo.io/developers/privacy) that includes vpn/proxy/tor/hosting(bot, scrapes,etc) flagging which may help your predicament? You can test out how accurate we are by appending any nefarious IPs to our URL and see if we flag it correctly. e.g. https://ipinfo.io/5.62.18.115.
Not sure if that will be your magic bullet but will be happy to have a chat and riff on this. I've talked to customers that are using our privacy detection API and I do remember a good subset using it for your sort of use case. At the very least, I can pass on some learning, strategies, etc to combat it.
that's our paid plan but we do have a free tier! if you have something in mind and an interesting use case, hit me up and I can organise access for you.
You could try Bot Fight Mode which is "if it's a bot, block" using the same enterprise tech, however this is an extremely unreliable bet since you have no tuning options if it doesn't block bots, or if it blocks some of your users that happen to be using something like a text-based browser.
https://www.cloudflare.com/products/bot-management/
https://www.akamai.com/us/en/products/security/bot-manager.j...
Edit: I didn't see your comment about budget. I expect Akamai may be out of reach, not sure about Cloudflare's options. Most bot detection is going to need to finger print behavior of the interaction to the site (Captcha as well). If that data is handled correctly, (not being sold/made available to a third party/destroyed after use), I believe it can be done ethically. Obviously my ethics are not yours.