I have both, and as stated before it depends on what you want to play. The Xbox only supports MAME 0.84, and your hardware limits are 700 MHz processor, 64 MB RAM. You can also play Xbox games like Soul Calibur II, DOA, ports of 2D fighters. I don't play the Xbox games very much, and I'm certainly planning on ditching the Xbox and using a PC instead. More and more nowadays I'm disappointed by what the box can't do, and the visuals aren't that great; MAME uses stretching and graphics filter features, doesn't look so hot compared to an ArcadeVGA card (that card has spoiled me!) that can output most of the funky resolutions used by arcade games, in many cases the game looks just like the arcade board.
So in the long run I'd go with a PC. The picture is better (ArcadeVGA cards are worth the price!) and it's versatility and upgradability is something you'll miss after you've been playing within the limits of the Xbox for awhile.