(It’s been a long, long day. So here’s an only-slightly-used, gently-recycled essay, but with an Brand New! hyperlink for your enjoyment. Bon appétit!)
“It has been said that man is a rational animal. All my life I have been searching for evidence which could support this.”
~ Bertrand Russell
Here’s our new word for the day: pareidolia. It comes from the Greek, para = almost and eidos = form. The word itself originates in psychology, and refers to that cognitive process that results in people seeing images (often faces) that aren’t really there: the man or rabbit in the moon, canals or face on Mars, faces of holy people in tortillas or stains in plaster … It also sometimes refers to hearing things that aren’t really there in random background noise (Electronic Voice Phenomena: EVP). Pareidolia is what makes Rorschach inkblot tests possible (attribution errors are what make Rorschach tests fairly unreliable).
The human brain is “wired” to see patterns, especially those of faces. Creating and perceiving patterns is what allows all animals to operate more efficiently in their environments. You need to be able to quickly find your food sources, your mates, your offspring, and the predators in the busy matrices of sensory inputs. Camouflage relies upon being able to become part of a pattern, and therefore less recognizable. Aposematic warning coloration, such as black and yellow wasps, does the reverse, by creating a specific kind of pattern that stands out.
Sometimes people subconsciously assign patterns and meanings to things, even though they don’t intend to do so. This is why we have double-blind studies, so the people who are collecting the data don’t unconsciously assign results to the treatment replications by increasing or suppressing or noticing effects in some trial subjects. Prometheus has a lovely blogpost about this: The Seven Most Common Thinking Errors of Highly Amusing Quacks and Pseudoscientists (Part 3). (This series of his just gets better and better!)
Seeing patterns can lead to weird cognitive biases and fallacies, like the clustering illusion, where meanings are falsely assigned to chunks of information. The fact is that clusters or strings or short repeats of things will naturally happen in random spatial or temporal collections of objects or events. A lot of people think that “random” means these won’t happen (which makes assigning correct answers for multiple choice tests an interesting process; students get suspicious if they notice too much of a pattern and then start out-guessing their correct answers to either fit or break the perceived pattern).
Sometimes the reverse can happen, where instead of seeing patterns in data, people put some of the data into patterns. This is known as the Texas Sharpshooter Fallacy: a cowboy randomly riddles the side of a barn with bullets, and then draws a target where there is a cluster of bullet holes. People will perceive a pattern of events, and then assume that there is a common causal factor to those, because of the perceived pattern. This is why statistics was invented – to suss out if there is a pattern, and how likely it is. Mathematics takes the cognitive kinks out of the data so the analysis is objective, rather than subjective.
Statistics also gives research rules about how best to proceed in experiments, to avoid various errors. One of those is deciding what kinds of analyses will be used for the type of data set that is produced by the experimental design. Note that this is decided beforehand! The reason for that is because people want to see patterns, and (even unconsciously) researchers want to see results. The purpose of testing for a null hypothesis is to try to disprove the given hypothesis, to avoid these kinds of issues.
It doesn’t matter how noble your intentions are – wrong results are still wrong results, no matter how they are achieved, or to what purpose.
To look at the data and then start picking through it for patterns, (“massaging the data” or “datamining”) is inappropriate for these very reasons. The greatest problem with doing analyses retroactively is that one can end up fitting the data to their pet theory, rather than testing the theory with the data. Mark Chu-Carroll’s post on the Geiers’ crappy and self-serving data “analysis” is an elegant dissection of how this kind of gross error is done. (Note that is MCC’s old blog address; his current blog is here at ScienceBlogs.)
Doing this intentionally is not only bad statistics, it’s bad science as well. The results come from anecdotes or data sets that are incomplete or obtained inaccurately. Correlations that may or may not exist are seen as having a common causality that also may or may not exist. It’s pick-and-choose and drawing erroneous, unsupported conclusions. People want to see patterns, and do. Even worse, they create patterns and results.
The seriously bad thing is that con artists and purveyors of various kinds of pseudoscience do this a lot. The intent is to deceive or mislead in order to sell something (ideas or objects or methods).
The people who then buy into these things then think they are seeing treatment results because they want to see them. Take this secret herbal cold medication, and your cold will be cured in just seven days! (Amazingly, one will get over a cold in a week anyway.) Give your child this treatment and they will be able to learn and develop normally! (Amazingly, children will learn and develop as they get older, for all not everyone follows the same timelines – developmental charts are population averages.)
Meanwhile, the well-intended but scientifically ignorant people who buy into these things are being duped by charlatans, sometimes with loss of life as well as with great monetary expense.
Economists will tell you that the cost of something is also what you did/could not buy, and when time and money is spent on false promises, it deprives everyone involved of the opportunity to pursue truly beneficial treatments.
Then the problem is propagated because those well-intended but scientifically ignorant people become meme agents, earnestly spreading the false gospel …