I’m wondering if I could obtain some kind of old/used android phone, remove all the apps so the home screen is completely empty, transfer via USB or ssh/scp the MP3 files (and metadata + cover art), then have some kind of music player app which would be the only app on the phone/device.

This is exactly what I did. I used an older Moto and uninstalled or disabled everything I didn’t want and only used BlackPlayer EX. It also has halfway decent external speakers so I can just play music directly off the device if I want to. The only thing that doesn’t thrill me is that it’s a full sized phone, which can be a bit awkwardly large at times. It’s been working great, but when the time comes for another device, I might go with one of the little Unihertz models.

There’s RiffTrax. They’re pretty much the only way I can stand to re-watch the Star Wars prequels at this point.