I had similar issues with the rechargeable NiCd batteries on the 4 slot MVS boards. Thankfully they did not explode or leak and the boards were spotless.

Managed to switch them to some weird NiMh same spec batteries and they work, though installing them was weird.

There was a hack to use a regular CR2032 but it involved removing a resistor to stop the charging process. I didn't want to risk it so I just placed the recommended replacement.

I presume the Dreamcast should be similar...removing something to stop it from charging and using a normal CR2032. Check online to see.