It only runs Windows CE if the game uses, and has it on the disc. Basically Windows CE is just for the game itself, and lacks even the functionality of a PDA OS when it comes to making windows and icons, etc. There is a port of Linux to the DC, however.
It would take quite a bit of effort to use a Dreamcast modem on PC, as they use a parallel bus that is incompatable with anything I'm aware of on a PC (15 lines of I/O)