I'm trying to get to the screws under the spindle cap, but they're stuck under it, so I was wondering if getting rid of these screws highlighted would help me to get to those screws

Looking at your photo, rotate the spindle about 1/8th of a turn anticlockwise until the cutouts in the spindle cap align with the screws?

Nice of the engineer to think of that!!!

They’re covered up by that more reflective piece of plastic. I can align the cutouts perfectly but they’re blocked by that more glossy part that I can’t move.

Just turn it until the notch aligned with the screw, then unscrew the screw while pressing in slightly on the side of the notch.

Edit: That second picture in your original post shows the notches aligned with the screws, so it should be that way. You could take a playing card and try to slip it between the two see if there is anything binding it.

It should be able to slide/rotate on top of the rubber base. It was assembled like that, with the spindle notches lining up with the holes in the base. So something has moved in the meantime to take them out of alignment.

A moderate amount of force might be needed. Eg the amount you can apply with your fingers by holding the top piece with one hand and pushing with the fingers of the other hand on the rubber base to rotate it.

You shouldn't need to grab it with tools / pliers to move it. If you think you need tools, it's better to stop and have a bit more of a think about it before going further.

If I were you I'd if get rid of the original spindle with the ball bearing and o-ring retention system, it damages discs. I bought a broken portable CD player online with the spring loaded plastic tabs and swapped them, much gentler on the disc and since the ps1 is only a 2x drive the speeds aren't an issue.

There are these shallow "U"-shaped indentations on the spindle. You can clearly see one on the left-hand side of the photo, but there's also one in the upper right.

If you turn the spindle so that one of those indentations faces the screw, you should be able to remove the screw.