This is a little bit of a wall of text but I hope it will be a nice sample for such a setup.

The rate limit is a bit pointless as the scrapers cycle through IPs via residential proxies and large data centers with many IPs and sadly there is an increasing number of scrapers that seems to be able to solve the Anubis challenge.

Never the less, a nice tutorial for the basic setup that does still help to some extend 👍

Unrelated to the actual anti-bot topic discussed here, but I would just like to mention this point:

PROTOCOL = http+unix: because UNIX sockets are said to be faster locally but also because this avoids the painful management of ports. Also feels safer [as] there is no way to connect from outside.

This is really underrated. I try to use this for all the server software I host, and not having to manage ports is absolutely great, along with knowing that nothing can connect to the service internally when it's not supposed to.

Except I actually go a step further and use systemd's socket activation feature, which has a few additional benefits (like seamless restarts, no need for CAP_NET_BIND_SERVICE (my webserver and reverse proxy runs as a completely unprivileged user!), only starting the service once there is a request (this one is admittedly less useful), and having a quick overview of bound ports and Unix socket paths listed in the .socket file). Unfortunately, most programs need to be patched to support this, including forgejo (outdated codeberg link here because I had to set the forgejo repository on my own code forge to private earlier today because of scrapers…).

Wait why dont more services use this?

I wouldnt say pointless but slowly getting less effective over time.