Objectively the Nintendo DS and the DS lite in particular, no contest.

-Loooong Battery Life
-Backwards Compatible with GBA
-Huge variety of games of all genras
-Great Design that protected most of the control surfaces when closed
-Not yet saddled by constant updates
-Sold somewhere in the 146mil range over model range, basically the PS2 of portable consoles.

I personally preferred the PSP for its many exclusives and multimedia capabilities of the time, but not because it was a better handheld overall. Objectively the DS lite is the current champion thus far and is by far the best portable that I've used on a regular basis. The GBA SP comes close but I feel that the DS library more then makes up for its lack of pre-GBA backwards compatibility. The GBA SP is also more resilient I've found then the DS lites but again the library really seals the deal for me on the DS.