RetroFed Forum
Your home away from modern life.
how does the lemmyverse crawler get data from slrpnk.net, which has anti-bot tech?
Post Reply
AuthorMessage
activistPnk slrpnk.net activistPnk@slrpnk.net
Joined: 2023-07-30
Quote

I noticed when visiting slrpnk.net there is an anti-bot mechanism saying “making sure you’re not a bot”, which takes a few seconds. I suppose there is some javascript calculations or something.

Yet most slrpnk.net communities manage to get scraped into the lemmyverse DB anyway. Did slrpnk have to make some kind of exception for that?

1   Original
Post Reply
Back to top