I've always assumed that this happens because most people who decided to spend the money on a Colecovision were people who already had an Atari 2600 to begin with and were ready for the "next generation" of gaming. That's how it was in my world. I only had 2 friends who had a Colecovision new and both of them graduated up from an Atari and ended up buying the adapter to play the old games with. Never made sense to me back then, as they still had the Atari in the closet so why not just pull it out to play those old games??
On another note, I never see ANY Colecovision hardware in the wild so I'd love to know where you haunt...