1. The controller - Everyone reading this knows the problem with the analog getting worn due easily especially with Mario Party. If you find a secondhand OEM controller with a stiff stick, immediately grab it because most of the sticks you find will be flaccid and as a result the dead zone will be much greater
2. small library - just shy of 300 games means that there are about 50-100 games that are worth playing to this day. I mean not many people are still playing Hercules (non-Disney) or the Hot Wheels game. And when you consider how many of those 300 games are sports titles, that really whittles down the amount of games that are still fun to play
3. outputs - RF, composite, and s-video without modding them. S-video is very comparable to RGB but even with the sharpness maxed out it still has that hazy look just due to the inherent nature of the game design.
4. cartridges that still relied on battery save - many of the third party titles relied on either internal batteries or the memory pak which also required coin batteries. and we all know that those batteries dont last forever. fortunately most of the first party games didnt require them thank God but this isnt an issue for systems whose games relied on solid state memory like PS2, Gamecube, or even just passwords.
5. camera angles - this was always the number one complaint with N64 games. constantly fighting with the camera was annoying and this was most evident with Super Mario 64.
6. lack of RPGs - Not really that its related to how well the console has aged but this has always been a problem for the system. besides Paper Mario there arent any good RPGs. Quest 64 and Aidyn Chronicles are mediocre at best and dumpster fires of a game at worst.
7. FPSs - I know ive said this before but after Halo, its hard for me to go back to the goofy controls of these games and the aiming system
For me the racing games and platform games have aged well in addition to the Pokemon games, wrestling games, and all the 1st party titles but everything else is very hit or miss for me.