Mar 03 2010
Can’t Test This: Correlations in Psychological Science
Two recent studies from the psychological sciences have perfectly illustrated the problem of variable correlation: Why we can’t jump to the conclusion of causation when we find an association between two measures.
1. Long-time cannabis use associated with psychosis
The finding of this study:
Young adults who have used cannabis or marijuana for a longer period of time appear more likely to have hallucinations or delusions or to meet criteria for psychosis, according to a report posted online today that will appear in the May print issue of Archives of General Psychiatry, one of the JAMA/Archives journals. [bold mine]
How much more likely, the inquiring mind would like to know. Fortunately, this article actually provided the information. A pleasant surprise.
[Y]oung adults who had six or more years since first use of cannabis (i.e., who commenced use when around 15 years or younger) were twice as likely to develop a non-affective psychosis and were four times as likely to have high scores on the Peters et al Delusions Inventory [a measure of delusion]. [bold mine]
Thank you for that precision in reporting.
In a study such as this, having potentially important cultural and political implications, I’d like to see not only the inclusion of a control group — to put things in an absolute perspective, so to speak — but also of a comparison group of alcohol users — to put things in a relative perspective, you could say.
That said, the point I wanted to make is that this finding was not the product of an experiment. No, researchers didn’t randomly assign youth to groups and one of these or more instructed to smoke X amount of marijuana. That type of study would generate results one could be confident about a causal relationship between the variables. But good luck getting a research grant for it. And getting it past the ethics board!
And the authors acknowledged the problem with the nature of their finding.
“The nature of the relationship between psychosis and cannabis use is by no means simple,” they write. Individuals who had experienced hallucinations early in life were more likely to have used cannabis longer and to use it more frequently…..[T]hose individuals who were vulnerable to psychosis (i.e., those who had isolated psychotic symptoms) were more likely to commence cannabis use, which could then subsequently contribute to an increased risk of conversion to a non-affective psychotic disorder.” [bold mine]
Exactly! Kudos to them for keeping that in mind and pointing it out.
Actually, do they deserve kudos for such an elementary level of critical thinking? Because it at least seems relatively rare, at least in the material provided to the more popular media, I’ll applaud it when I find it.
2. Teens with more screen time have lower-quality relationships
In a study on computer and video-game usage and quality of relationships, an association between variables was discovered.
Teens who spend more time watching television or using computers appear to have poorer relationships with their parents and peers, according to a report in the March issue of Archives of Pediatrics & Adolescent Medicine, one of the JAMA/Archives journals.
Again, the skeptical mind wants to know “how much poorer” and “how measured”?
The answer to the latter is “confidential questionnaire” as well as “an assessment of their attachment to parents and peers.” What kind of an assessment and by whom?To the former — kudos again! — the authors provide some numbers, such as:
The researchers also assessed interview responses from 976 individuals who were age 15 years in 1987 to 1988. Among these teens, more time spent viewing television was associated with lower attachment to both parents and peers. For every additional hour of television, teens had a 13 percent increased risk of low attachment to their parents and a 24 percent increased risk of low attachment to peers.
Conducting controlled experiments on lab rats and mice will only get you so far. And mice are notoriously bad video game players, so that would certainly be a confounding factor . . . . Could an experiment be conducted on this topic, with humans? I wonder.
Fortunately, as with the first study listed, these authors too, highlight the hazard of correlational findings by speculating about alternative explanations for theirs:
“However, it is also possible that adolescents with poor attachment relationships with immediate friends and family use screen-based activities to facilitate new attachment figures such as online friendships or parasocial relationships with television characters or personalities,” the authors write.
Exactly! And there are other possibilities.
So, do we conclude that the psychological sciences are worthless? No, not at all. But at the same time, we need to recognize the weak nature of much of the data generation and should refrain from making too much of non-controlled-experiment generated results.
