I’ve always been told to be careful about where I get my information from. That’s probably pretty good advice. We all know not to believe everything we see on T.V. or read on the internet (or at least we all should know that, hmm?), but what about published journal articles?
Scholarly journal articles, on the surface seem pretty safe to trust. They’re peer reviewed by a panel of academics in the field of study. The studies conducted are based on previous research and (usually) are replicable. That means the information they provide is correct, right?
Wrong.
I spent a semester in my undergrad and a semester in my master’s work studying statistics and research methods in order to understand how the process works, and to become a better consumer of research. Meaning that I would need to be able to pick out the good, solid research from the bad.
You can say anything with statistics. A wise person once told me that 100% of the people who eat carrots die.
It’s an accurate statement. It’s also taken out of context.
Why is this important? We are so quick to believe things that scientists or researchers say, that we need to take a moment to look at where the information is coming from.
An example: There are SO many studies out there about video games and the harmful (or helpful) effects they may have on children. First video games were bad…then they weren’t bad…then the original researchers critiqued the second study and they were bad again. Now there’s research that says they’re good!
So many different studies say so many different things. Here’s my take on why:
1. Poor methodology. I haven’t read all the research, and can’t even begin to point you in the direction of some (my school doesn’t have online access to most of the studies I’ve seen write ups on), but I do know that many of these studies are actually correlational studies.
This means that you can show that something is linked to something else…but not show causality. The only way to show that something causes something else is to do a random controlled study, which is really hard to do with subject matter such as video games, due to outside experiences.
2. Researcher bias. Everyone has their opinions, including those who conduct research. Sometimes that can influence the way studies are created.
The way a study should be created is to design the study, create the hypothesis, and then run the study. Sometimes, the problem lies in that order being mixed up. Some researchers will come up with a hypothesis and then create the study to prove it right. This is backward science.
So if someone thinks video games are bad for kids, they’ll design a study to show exactly that. The same goes for researchers who think video games are good for kids.
If I ran a study I would worry about the validity of my results, even if I knew all of this and had designed the study properly. My wish for games to be a positive influence on youth (and adults) could bias my analysis of the results, or how I present the findings statistically.
I would make a horrible researcher anyway.
Because of these problems, there will continue to be conflicting results from both sides of the issue. The only way I see out of this is for the research to be conducted by someone who couldn’t care less about the outcome…or for a team of individuals from both sides to work together on a study.
I don’t see that happening any time soon.
And that, my friends…is the trouble with trials. You can’t trust them, but you have to.
I hate research.