They aren't moving and don't seem to be attached, as the come off easily when I wipe the leaves with a tissue. They are on two plants that were right next to each other but not the basil that was also next to one of them

Couple more pics

Aphids when they molt and leave their old exoskeletons behind.
https://www.rhs.org.uk/biodiversity/aphids

Yeah I just saw some moving as well when I was wiping the leaves. They're all over the stems too so I'll probably need to make some kind of spray to make sure I get them all

If you don’t want to spray anything you can take a damp papergowel and just kinda … wipe em off.

You may not get em all but you can take their population largely out just by wiping em away.

I got most of them off the leaves but there seemed to be a lot on the stems that I couldn't really get to. Part of the problem is the plants are pretty young and still not very hardy so I don't want to manhandle them too much. I tried a spray with a bit of dish soap in water and if it doesn't help much I'll try to be delicate about wiping.

I keep a variety of chile pepper plants indoors over winter and deal with this constantly. Here's what I've learned about aphids. They will kill your plant if you aren't vigilant about them. A neem oil spray can help but won't solve the problem entirely. Some formulations of neem oil are harder on the plants than others so test your solution every time you get a new bottle. If you have a handheld shower wand, it is worthwhile to put your plants in the shower about once a month and blast off the little buggers. And regularly fertilizing your plants will keep them strong enough to resist the attack. The best method I have seen, is to use water from a fish tank to water the plants. Good luck!

They're pretty easy to manage. Just make a solution of dish soap in water (~1 tsp per L) and spray the plants. It dissolves their waxy coating and kills them quick. It shouldn't harm the plants but spray a test area first to be sure. It won't harm their eggs so you'll probably have to spray a few times.

Everyone is saying aphids but those look more like young thrips to me. Same way to get of them though. Physical removal of the eggs and a good plant soap should get rid of them.

Mild solution of pine oil soap and water sprayed onto the leaves will likely kill whatever they are but wont harm the plant or you.

I also saw the other pics. If you zoom in closely, you can see fat aphids.
Those white flakes are their exoskeletons they shed when maturing.

Get some neem oil and silicate, that will get rid of them and make the plant more resistant for upcoming generations of the pest.

Or befriend a bunch of ladybugs!

/j (not that it wouldn't work, just that your suggestion is better)

Have you considered using a spinosad solution? In the past, I’ve had luck with Captain Jack’s Dead Bug Brew.

Grated Parmesan cheese.

Those look a lot like white spider mites.