Here's my take on all of the Atari Flashbacks:

Atari Flashback 1: Avoid. It's NES-on-a-chip simulations of old games. There are some interesting choices here (Food Fight, 7800 Asteroids, Solaris) but they aren't true to the originals and the joysticks are goofy.

Atari Flashback 2: Recommended! VCS-on-a-chip, very accurate. Good collection of old games, including some Activision titles. Mod-Friendly if you have the parts and skill to add a cartridge port.

Atari Flashback 2+: Less recommended. Like #2 but with no Activision games but some sports games.

Atari Flashback 3 (60 games)
Atari Flashback 64 (64 games including Space Invaders, wired controllers)
Atari Flashback 4 (75 games)
Atari Flashback 4 Deluxe (75 games, wired paddle controls, and paper posters)

These are OK. These are all from AtGames and have emulated ROMs on a custom chip. Not as accurate as FB2 but a few more games with each successive release. All of them but the 64 have infrared wireless controllers but can also take retro style wired controls.

Atari Flashback 5 (92 games): I would get this one. It is just like the previous AtGames releases, but it has the most games. I'm pretty sure it's running the same hardware and possibly the same software.