Link: https://github.com/benjismith/ai-randomness

Comments

We asked Claude to "pick a name at random" 37,500 times across five models and dozens of prompt variations, then analyzed the results.

37,500 times

The most common male name was "Marcus", chosen 4,367 times (23.6%)

That percentage maps to 20k responses, not 37.5k. Which is it?

The other 17500 responses were Claude deleting your e-mail inbox instead.

If you ask for a long-ass list of anything, LLMs have context-length problems. Like trying to repeat the word elephant over and over. The math doesn't like it and weird shit happens.

If you ask for one name at a time, and it comes up with the same name a bunch... uh... yeah? Were you expecting a perfectly flat distribution from McLovin to Mohammad? The probabilistic word-guesser is gonna have some trends. Marcus is an odd first pick, compared to its prevalence, but if you ask for a less-than-typical name, that is a correct answer.

Similarly, if you ask an image model for a generic portrait, you're gonna get something from the middle of the probability space. It might be roughly the same vaguely familiar caucasian brunette every time. Or like this silly experiment, once every five times. It's not gonna be like hitting the Randomize button on Oblivion's character creator, because that's not how this tech works.

I've been using Cursor at work (been kind of forced to, not entirely my choice) but I'm not a developer, as I do "development-adjacent" tasks. So I sometimes ask it to read large config files and such, and extract information for me.

Seeing how it works, I suspect if I asked it to write the word "elephant" 30,000 times, I kind of expect it to write a script that will do that by writing it to a file, run the script, then display the resulting file.

ChatGPT would probably just start writing "elephant" repeatedly until it runs out of context, then start doing weird things.

But I haven't tried... To anyone reading this: feel free to try it and report back

Statistically most likely random answer

I believe the clarifier is male name. So, a male name was chosen about about half the time.