Re: Some Interesting New Stats About Gaming in the UK
This stat stuck out like a sore thumb. I'm sorry but 100% of kids 6-10 do NOT play games. I'm sure that a huge majority of them do but 100% means that I couldn't find a single kid who doesn't play games and it's just rediculous to say that. Perhaps if he only polled 20 people but then that would hardly be worth reporting* All kids play games. (100% of 6-10 year olds)
This is fairly accurate, but it doesn't fully address the problem here. Even if the survey did take a sample of 2000 randomly selected people, once it is broken down into subsections such as 6-10 year olds, then it can no longer be taken as representative.Originally Posted by lendelin
To be representative of 6-10 year olds you would need a suitably sized sub-sample of randomly selected 6-10 year olds for that. Having project managed in a quant research company, I can't begin to tell you the amount of hassle it is to get representative sub-samples for all the gender / age / socio-economic grade breakdowns ...
But further than that even, 'representative' is a loaded term. If you have a 'representative' sample it does not mean that it gives you the breakdown across the whole population. It just means that there is a specific probability that it is only incorrect by a certain amount!
i.e. it could still be completely wrong, it's just unlikely that it is. But it's best not to mention that, because once you get down to that sort of nitty gritty, clients can sometimes get pissed with you.
Vroomfunkel
P.S. Woooo! 2000 posts! Level 10 at last! I score teh geek prize!
"The last time a nation did what a bush told them to do, they ended up wandering 40 years in the wilderness ... " - Anonymous
Most of the 'girl gamers' I know play World of Warcraft and nothing else. I don't consider someone a gamer if they play one game and one game only (and only play that game because they like the guild chat and their boyfriends play it, making WoW the only way they can get in regular contact with their significant other :P). I wonder if it's similar in this survey?
It is accurate, not just "fairly," and it addresses the full problem of representative samples as a whole in a very basic way (low case numbers for specific variables and a plethora of other problems are not addressed)This is fairly accurate, but it doesn't fully address the problem here.
That's a given; if you break down EVERY variable in a data set the case number is lower than the sample. If you have in one age group twenty cases, you cannot work with it. This is easily spotted just looking at the raw data or looking at the statistical significance level. A level of 3 means that there is a 30% likelihood that the result is based on a sampling error, this means it can be just a coincidence and cannot be generalized. The variable becomes absolutely useless (unless you can make a point that there is a substantial significance despite the statistical insignificance which works at times with significance levels up to 1.5)...once it is broken down into subsections such as 6-10 year olds, then it can no longer be taken as representative.
To be representative of 6-10 year olds you would need a suitably sized sub-sample of randomly selected 6-10 year olds for that.
If you combine variables to make a case, you easily end up with two one legged, black cell phone owners in the same income bracket who happen to play games on their phones once every six months. Three people are certainly not a representative sample statistically.
However, two thousand polled people are enough to draw conclusions about gaming habits for every age group as a common rule.
I know about the hassle.Having project managed in a quant research company, I can't begin to tell you the amount of hassle it is to get representative sub-samples for all the gender / age / socio-economic grade breakdowns ...I once did a mail-poll with a very poor response rate (300), questionnaires not filled out completely (which is always the case), and then you look at an important variable for your hypothesis and end up with twenty cases. BAD, very bad, because everything was for nothing.
You are way too skeptical and mathematically incorrect. The likelihood of 5% that the results of a poll are based on sampling error might skew the reality of the "actual, all engrossing population,' but the likelihood that the results are "completely" wrong is astronomical and neglectible.But further than that even, 'representative' is a loaded term. If you have a 'representative' sample it does not mean that it gives you the breakdown across the whole population. It just means that there is a specific probability that it is only incorrect by a certain amount!
A .1 significance level just gives you that --- the likelihood that the results are based on sampling error are a measly 1%. In non-biased polls with 2000 or 3000 respondents even if certain varaibles are broken down and combined, you often end up with significance levels of .05 --- certainly a very confident level.
About this poll: we just don't know the case numbers, and other very important data, we just know the percentages. Given the conceptualization probs (broad definition of gamer, inclusion of all kinds of 'gaming,' and many others) the poll is pretty much worthless, we don't have to get into more refined problems. You can't draw any interesting conclusions just based on these conceptualization problems.
I suspect it was a quick and dirty poll, probably even a phone poll without even letting a computer dial random numbers (even the latter doesn't produce unbiased polls anymore because of answering machines).
Posting Permissions
Reply With Quote