I wouldn't call most instances of people watching others play via video a replacement for actually playing games. In most cases, the people watching had no interest in playing the game in the first place. When people watch reaction videos, they're interested in the humor of the person reacting, be it their overblown fear of a horror game or their frustration with a crappy game or their lampooning of one, not because they're so fascinated with the game itself. It's more like video games are being used simply as a medium or topic to beget other forms of entertainment. Or, say, with Let's Play videos of classic games, people may be watching to get fuzzy, nostalgic feelings out of it, rather than them actually wanting the experience of replaying the game.
One of the few cases where I'd say people are genuinely using videos as a way of enjoying the games themselves is for story-focused games that have minimal gameplay, like visual novels. In that case, it's akin to pirating. You get virtually the same experience as playing the game yourself, but you don't have to spend the money to acquire it. This is why publishers of these sorts of games often have strict rules as to what they'll let fans upload to Youtube and such. Full playthroughs tend to be pulled down. Publishers who deal in games with a significant gameplay component don't care nearly as much about Let's Play videos that show a game in full because they know that the experience of playing cannot be replaced by the experience of watching, thus those who are watching aren't considered lost customers. Rather, they're considered as people who either will never be customers, because they're watching for the entertainment of the videos themselves, not the games, or they may be convinced to buy the game by watching.
How is that any different from athletes who are only good at their particular sport/field? Unless you think decathlon athletes or what have you are the only athletes deserving of respect. But they're jacks of all trades who aren't nearly as good at any individual event as those who specialize.
Also, thanks for the kind words towards visual novel fans. I see you're as much of a pleasure as always.