All-Star Slammin' D-Ball is decent. I'd much rather play Super Dodgeball on NES but, should you come across All-Star Slammin' D-Ball on the super cheap, I'd recommend it.
I found this puzzle game made by Jaleco called Super Bubble Pop for $3 at a thrift store. It was quite addicting and different as I hadn't seen many 3D puzzle games. You actually control a character that throws the bubbles. Has a very techno style soundtrack to it.
C-12: Final Resistance was released as a budget title at the very end of the PSone's life cycle. It's pretty fun. Also, I think Expendable might have been a budget release.
I really enjoyed The Italian Job, as well as both of the Shooter games from Agetec.
And while I wouldn't necessarily consider it a budget title, I found a two pack at the end of the PS1's life of Rising Zan and Shadow Tower for 10 bucks. I loved Rising Zan when I bought it for 40, so I picked up the two pack, kept Shadow Tower for myself, then gave Zan to my best friend... and that game is mine once more as he gave me his entire Playstation collection (including FFVII and the black label Castlevania: Symphony of the Night I gave him for his 20th birthday) recently.
The Star Sweep game (whatever it's called) is a pretty fun little puzzle game. I also really like the two budget RPGs that someone put out (yeah I don't remember who): Eternal Eyes and Darkstone. One is a strategy-RPG, the other is a Diablo clone.
Castlevania Chronicles was a great budget title.
Originally Posted by TheShawn
Qix Neo
Gekioh: Shooting King
Gubble
Puzznic
Turnabout
Any of the Pro Pinball games
A couple have been mentioned already but here are my picks.
Boardgame Top Shop
Puzzle Star Sweep
Power Shovel
Superstar Dance Club #1 Hits!!!
Turnabout
Games that were budget repackages in shrinkwrapped double packs...
Koudelka
RC de Go!
Rising Zan
Yes! Power Shovel ~ I love that game!
So much fun with the official controller made by Hori
...so much fun anyway, but the controller adds significantly.
I always wanted to see if it works with the original Sony Dual Analog Joystick, seems like it should. However the more compact and shorter Hori Power Shovel controller is quick and snappy with its shorter joystick throw.
Not sure if this was originally budget but the Playstation Q-Bert title is cheap, lots of fun, and a superb update to the original.
Sanvein was really good, and Space Shot was definitely interesting. It has a dash move in its otherwise Thunderforce-like gameplay. I saw them for sale on the Japanese version of PSN, btw. (600 yen, I think).
http://www.youtube.com/watch?v=trT_LJE8CIk
http://www.youtube.com/watch?v=MwzEiR9TyMA
I came across this page tonight:
http://fuji.drillspirits.net/simple/
Unfortunately it does not offer reviews, but I thought it might help you sort through the catalog.