I'm not entirely sure how people define games as casual compared to hardcore, to me casual games are things like free flash games or computer solitaire type games. Are there games like this on Nintendo systems? Or are people confusing casual games with kids games?
I have no problem with Nintendo sticking to kids games, that's where the money is. Parents are more likely to buy stuff for their kids over their own wants, and pay more for these items.
Really there are several problems with games making money now. There are really too many games coming out on too many different platforms. In the past while there were three consoles out at any time, only two would be successful with one struggling. And people would choose just one console, or they would be PC gamers and ignore console games. Several people would miss out on certain titles on other platforms forcing them to buy exclusives on their own console even if they were mediocre compared to the competition. Now it seems all three current consoles are doing well, and people tend to own more than one console while also playing PC games. Plus people are also owning portable consoles. There's too many games available to everyone so really it's easy to see why some games get overlooked or forgotten about.
Still plenty of games sell very well, selling more copies than most games sold in the past to be successful. Now it's not enough to still be profitable, they need to sell many more games than people are willing to buy. It's getting to the same point as Atari Pac-Man, soon they'll need to sell more copies of the games than there are console owners to still be profitable.
That and peoples' wages haven't increased as much as inflation over the years yet there's plenty more to spend what disposable income they have. In the 90's you could still find people who never owned a computer or cell phone, and plenty of computer users didn't bother with the internet. Now just about everyone has a cell phone or more commonly a smart phone, as well as a tablet, and a computer/laptop, or multiple for several family members, plus upgrading them every few years while paying monthly access fees as well as upgrading televisions which people never used to do until they wore out, upgrading to BluRay or 3D equipment, etc. It should be obvious why people aren't spending as much on video games with what little free money they have.
There's a lot of factors together that's hurting this industry. It's not just one thing.