Since my Peertube instance was down for a few weeks I am not entirely sure when this happened, but I suspect something in Peertube version 8.2.x changed regarding the ffmpeg settings that broke hardware accelerated video transcoding.

It still works for VOD and livestreams work with the default CPU transcoding.

I suspect the plugin I use became somehow incompatible, but if anyone here has similar issues that might narrow it down further.

Im not sure, havent had the issue, but im a bit behind on the versions now. Maybe someone else can chime in?

You should really update as there have been some serious security issues with Peertube.

what do the logs say? you should be able to watch the container's logs when you try it and see what it says

The logs are not very helpful other than confirming that it is a ffmpeg error: exit with code 234 Conversion failed, which I can't find anything useful online about.

Edit: ah, it also gives a super long error from ffmpeg itself. Still trying to parse it.

Looks like I got the right pointers here: https://framacolibri.org/t/hardware-transcoding-with-nvenc-suddently-stopped-working-for-live-streams-only/32712