Sega Saturn:
Easy to get into the basics. Diverse games massively variable in price, quality, rarity, genre, etc. Ranging from ~$5 to over $100. Japan has a larger cheaper library.

However a bit complicated if you want to do everything. Region, copy protection, and 50/60hz, are all handled independently. Modchips will not play legal imports. Action Replay will not play backup copies. Even together they cannot change video mode (if 50hz optimized games matter to you). AR also occupies the cart slot, preventing use of backup RAM while inserted. Which is annoying for games with large save files.

Nintendo 64:
Popular when the wrestling and Pokemon booms overlapped. WWF No Mercy + Pokemon Stadium is a combo letting you relive two huge epic eras. The Rock says you should own an N64 or know your Pikachu.
Don't forget Ocarina of Time, Super Mario 64, and other classics either. Plus console FPSs began coming into their own. Few decent RPGs though.

Neo Geo:
Expensive to get the better system models. Somewhat smaller genre range than usual due to an arcade origin. Next to no RPGs here either. If you don't love fighters it's tougher to recommend. Home carts are very costly for just playing. In the long run much more affordable to use arcade ones.

Nonetheless it serves its intended purpose better than anything else. Neo Geo fans have good reason they're so adamant about the system. No load times or emulation issues is a big part.

Sega Dreamcast:
Few top tier games were both developed exclusively for Dreamcast (often originating in arcades), AND also remain only on Dreamcast today. But it lets you play them in a single place. Also offers very respectable average game quality. Almost any game will be at least decent. On one hand you won't find major clunkers, on the other you may not be AAA blown away.

Nintendo GameCube:
Quirky. Devs took risks for this system. Lots of games unorthodox in some way. Stuff like Pikmin, Cubivore...
Multi-platform titles nearly always lost online in favor of bonus content. Which at the time seemed disappointing yet is now a great thing. Super Punch-Out and similar are permanent on disc extras. Unlike shut down MP servers PS2/Xbox versions have.