Yes, both consoles are/were hackable.
PS3 was much easier with earlier firmware, not certain if it's possible at all on current firmware.
I don't know much about the homebrew scene, but both consoles have all sorts of classic console and arcade emulation available.