RetroFed Forum
Your home away from modern life.
99% of My Website Traffic Is Bots: A year of fighting scrapers on my 1.5 million-page website. What I tried, how I failed, and what's working now.
Post Reply
AuthorMessage
alyaza [they/she] beehaw.org alyaza@beehaw.org
Joined: 2022-01-28
Quote

93   Original
Em Adespoton lemmy.ca adespoton@lemmy.ca
Joined: 2023-06-04
#2 · Quote Reply

I tried to read that. Then I tried skimming for the actual content. I found some, but eventually gave up. Claude can definitely tell a long-winded story.

16
Cris_Citrus (he/him <3) piefed.zip Cris_Citrus@piefed.zip
Joined: 2026-02-26
#3 · Quote Reply

I tried to read that. Then I tried skimming for the actual content. I found some, but eventually gave up. Claude can definitely tell a long-winded story.

Nothing about this reads as ai generated to me. Just seems like someone explaining the struggles they've had with their site being scraped, and what they've tried in response

Seems like exactly what it says on the tin if you ask me 🤷🏻‍♂️

12
XLE piefed.social XLE@piefed.social
Joined: 2025-06-23
#4 · (edited) Quote Reply

Nothing about this reads as ai generated to me. Just seems like someone explaining the struggles they've had with their site being scraped, and what they've tried in response Seems like exactly what it says on the tin if you ask me 🤷🏻‍♂️

Nothing?

I see three-armed robots.

Edit: they openly use AI elsewhere on their website.

31
Cris_Citrus (he/him <3) piefed.zip Cris_Citrus@piefed.zip
Joined: 2026-02-26
#5 · Quote Reply

Nothing? I see three-armed robots. Edit: they openly use AI elsewhere on their website.

Hmmmmm, yeah I was looking at the writing style not the art. Gross.

Thanks for pointing out the ai bio disclaimer, I can see why they might seek to automate that part at appreciate them disclaiming it but it makes sense to be more critical given that detail

Unless I'm missing something the article still feels written by a person to me, but my ability to identify ai writing may be less sharply tuned compared to others, or out of date 🤷🏻‍♂️

6
Em Adespoton lemmy.ca adespoton@lemmy.ca
Joined: 2023-06-04
#6 · Quote Reply

Hmmmmm, yeah I was looking at the writing style not the art. Gross. Thanks for pointing out the ai bio disclaimer, I can see why they might seek to automate that part at appreciate them disclaiming it but it makes sense to be more critical given that detail Unless I'm missing something the article still feels written by a person to me, but my ability to identify ai writing may be less sharply t...

I think you’re still missing something then; the way words are bolded, the way bulleted lists are placed, sentence fragment structure, using actual <h#> structure and using it correctly, the even placement of generated images; it all screams “AI”.

One of the reasons AI uses that type of structure is that it is easy to generate nested context that way in markdown, and then convert directly to HTML, and AI agents talk markdown. In fact, pretty much every long form article I’ve seen generated by AI looks exactly like this, right down to the lack of information density.

There were definitely some places that were human tweaked, but most of this was generated in whole cloth off of the logs the website generated.

8
Cris_Citrus (he/him <3) piefed.zip Cris_Citrus@piefed.zip
Joined: 2026-02-26
#7 · Quote Reply

I think you’re still missing something then; the way words are bolded, the way bulleted lists are placed, sentence fragment structure, using actual &lt;h#&gt; structure and using it correctly, the even placement of generated images; it all screams “AI”. One of the reasons AI uses that type of structure is that it is easy to generate nested context that way in markdown, and then convert directly...

Gotcha. I think I'm realizing my ability to identify ai writing might mostly be in the context of more persuasive essay writing and sometimes narratives moreso than other contexts

I kinda see what you mean about formatting, but would never have identified this as ai writing

Thanks for sharing your thoughts

3
masterofn001 lemmy.ca masterofn001@lemmy.ca
Joined: 2023-06-12
#8 · Quote Reply

I think you’re still missing something then; the way words are bolded, the way bulleted lists are placed, sentence fragment structure, using actual &lt;h#&gt; structure and using it correctly, the even placement of generated images; it all screams “AI”. One of the reasons AI uses that type of structure is that it is easy to generate nested context that way in markdown, and then convert directly...

One thing I've noticed that doesn't need to know page structure - scroll all the way down and look for the last section with the title Conclusion.

Just noticed this a few daus ago. So many sites I've never heard of about niche topics have the exact same layouts and all ended with a Conclusion.

Why is there a conclusion on a page for swapping out a circuit breaker or building a fence gate or how to properly shoot heroin? Ai slop.

8
moldy_rice piefed.keyboardvagabond.com moldy_rice@piefed.keyboardvagabond.com
Joined: 2026-08-07
#9 · (edited) Quote Reply

I'm one of your human visitors who isn't counted as a human because I don't use JavaScript btw

11
Alavi programming.dev somegeek@programming.dev
Joined: 2023-06-25
#10 · Quote Reply

What have we gotten into

1
Alph4d0g piefed.social Alph4d0g@piefed.social
Joined: 2026-07-14
#11 · Quote Reply

I'm one of your human visitors who isn't counted as a human because I don't use JavaScript btw

I tried to visit the site myself and the cloudflare human verification failed. My stuff is pretty locked down to avoid trackers and such. Oh well.

3
Post Reply
Back to top