Most people get confused when faced with an article about medicine, or any kind of complex science. Because people with extremely expensive educations wrote that stuff, then other people figure (at some level below common sense) that the study’s authors must be fundamentally superior.
Education is not the same as intelligence.
Intelligence is not the same as sense.
Sense is not the same as integrity.
I come from a highly educated family. (‘Nuff said.) Growing up in the context of good education really made it clear that people are people, regardless of the letters after their names. Degrees simply mean that someone can work hard on their own behalf; they’re no guarantee of logic or brilliance. It’s never wise to subvert common sense in favor of education.
So, if you’re one of the majority who doesn’t have alphabet soup after your name, give yourself some credit as you read these things.
There are a few simple principles that can help you dissect a study with reasonable confidence:
- Question assumptions (& listen to your eyebrows.)
- When the question makes no sense, you don’t have to accept the answer.
- If it seems stupid, it probably is.
- Don’t ignore the man behind the curtain.
A Furry Example of Fuzzy Logic
The article cited below is precious … A delicious exercise in mental pretzel-ry designed to reduce the average brain to cottage cheese. It’s easy to unravel if you hang onto your common sense and don’t let go, because your brain is not average.
Ready? Here it is.
- The title of the article linked below proclaims that researchers have proven that Topic A is bogus.
- The researchers’ summary says no such thing. It states that they’ve proven that your belief in Topic A should be much greater if you do believe in it or much less if you don’t; doesn’t matter which.
- Then the researchers state that that finding, itself, doesn’t matter, because they personally don’t believe in Topic A, can’t think of anything in its favor, and that you should agree with them — regardless of their own findings — simply because they said so. (A fairly common conclusion.)
Let’s pause to regroup, since this is enough to make most people tear their hair and gnaw the furniture. That tends to kill the punchline.
Backing up the train of thought to the beginning…
Pick something that there’s some disagreement about. For the sake of clarity, choose something not too emotional, like, “Do ferrets make good pets?” Pretend that’s Topic A.
You’ve already thought of ideas that support this and ideas that don’t, and you probably already know whether you, personally, would like having a ferret as a pet.
Have you ever, in your most random moments, picked a percentage or a ratio to indicate how much you would, or wouldn’t, like to have a ferret as a pet, with nothing to compare it to? I mean, is there any value to the idea of doing so? How odd is it to assume that people would?
TIP: Question assumptions. If your brain — or the skin on your forehead — starts to squirm, it’s a good clue that there’s an unexamined assumption waiting to jump up and trip you. Stop and check.
Liking pet ferrets is simple: you either like them (a little or a lot), you don’t like them (a little or a lot), or you decide you don’t know enough to have an opinion. That last option isn’t even available here, but it’s very common.
If you have an opinion about ferrets as pets, doesn’t its extent depend on external forces — whether you’ve known pet ferrets, whether their owners were responsible, whether it was a nice ferret or a real brat?
And wouldn’t the appeal of keeping/getting rid of a pet ferret depend on whether there’s a pet store stocking ferrets and ferret supplies, what your lease says about pets, whether or not your housemate can ferret-sit while you’re hiking the Camino de Santiago, whether your veterinarian can help you surrender an unwanted ferret? And don’t these circumstances themselves change, from place to place and time to time?
So how can you assign an absolute percentage to your opinion about whether ferrets make good pets? How surprised would you be if anyone asked you to do so?
And, really… Why would you? Do you assign a percentage to how much you dis/like strawberries, the color blue, or Sarah Palin? Or don’t you use value words instead — love, like, can’t stand?
Unless most (rather than very few) of you think of your preferences in numeric terms, then the very question the researchers are trying to answer is fantastical. Pure silliness.
TIP: You don’t have to accept an answer, if the question itself makes no sense.
Moreover, the way they processed the data doesn’t change your answer; it indicates that your beliefs should be far stronger, whatever they are.
They’re saying that, if you would like a ferret as a pet, you should be on your knees at the pet shop, weeping with longing — or, if you already have one, should be emitting a constant stream of happy little noises as you snuggle your ferret at work, on the bus, everywhere, all the time.
If you would not like a pet ferret, you should be packing to move so you can stay as far away as possible from anything long and furry or even vaguely ferretlike — or just blow up all the ferret-friendly pet shops where you live.
TIP: Just because someone with a very expensive education says it, doesn’t necessarily make it so. If it seems stupid, it probably still is.
Contrary to the title of the article, the results tell you: don’t change your position, just become more extreme about it. That’s their conclusion.
Isn’t that helpful? Just what we need: debates that are even more shrill, spittle-flecked and unreasoning.
TIP: Repeat prior tip… Really stupid.
In light of the decisions that led up to this conclusion and the anti-intellectual nature of the outcome, do you think this makes sense?
And, clearly following their own advice, the scientists themselves pick a side and pronounce that they don’t like pet ferrets and that you shouldn’t like them either.
Why?
Because they don’t understand how anyone could like pet ferrets and BTW other scientists in vaguely pertinent fields don’t know enough to prove how pet ferrets can possibly be desirable.
Therefore (stretching the metaphor), given this massive ignorance on the part of so many highly-educated people, ferrets are obviously terrible pets and all of them should be gassed.
…WHAT??…
SUMMARY:
This is not a terrible study and it was not done by stupid people. They just left their mental integrity in their other jeans, and that happens a lot.
Why on earth…? Because we all have assumptions and agendas.
Science aims to clear that out, but it’s done by live humans with organs and mortgages, so their objectivity is pretty hit-and-miss.
When reading science articles, be open to hidden agendas while you look for the facts. For better or worse, they go together. You might as well notice both.
Scientists are often very obvious about using big words to say silly things, and if you can step aside from feeling intimidated, it’s surprising how obvious they are.
The problems here are common problems:
- Point 1. Article’s title misrepresents the outcome of the study.
I usually read an article at least twice before making up my mind.
I read it through, then start again at the title. How accurate is it?
If the title isn’t fairly accurate, I know someone’s got an intrusive agenda.
- Point 2. Outcome doesn’t make sense.
It says you should believe either more than you do, or less than you do, but it doesn’t matter which. How to do so is not mentioned (for good reason.)
I usually look over the details of an article three or four times, to give the facts time to sift together in my mind.
When I feel my brow wiggling at something, I stop and look again. I trust my good sense more than I trust my education.
Figuring out crappy data just requires you to assume you’re not an idiot, even if you don’t know the field.
Don’t think badly of the scientists, just assume they have their own sets of human flaws. It’s a safe assumption!
- Point 3. Conclusion goes against the findings.
In any case, DON’T believe in Topic A, because the researchers have made up their minds on the basis of their ignorance, and screw their data anyway.
Continue to assume you’re not an idiot, as you read the conclusion. It’s that simple.
Then compare it, again, to the title and to the facts.
If something doesn’t add up, you know there’s agenda going on.
These particular scientists intended to prove that their statistical method was better than existing methods. Given all the logical problems surrounding their efforts, I think they blew it, but I’m not a statistician.
CLUE:
The topic of this study was ESP.
As my relentlessly rational, very prescient Dad once said when I asked him whether his use of ESP was irrational, “It would be irrational to ignore the evidence of my own experience. It’s highly consistent for me, even though most people can’t do it, or can’t do it very well. But just because they can’t use this valuable tool, does that mean I shouldn’t? That wouldn’t be very clever!”
Dad was very clever. (…And for the record, he did foresee his own end.)
Let’s step over to another, less-emotional metaphor to think about studying this subject.
Imagine that most people are basically color blind, but a few can see some color. Anyone who can see beyond the greyscale is not going to get much credit, but there are enough of them to make the rest wonder.
However, since the colorblind are looking for proof of color with instruments that see only luminance, but cannot see color even as tone or hue, they probably won’t have much luck proving something that they don’t understand, can’t use, and don’t believe in anyway. (…But they can sure get snarky, trying.)
TIP: Don’t ignore the man behind the curtain. Think for yourself … And try to remember, especially if you’re in a position of respect, that you don’t necessarily have the right to think for others.
Today’s unbelievably fatuous truism, which everyone always forgets anyway: Other people are not you. Only you are. Honor that, and things go better.
THE POINT IS:
In the end, everyone has to pursue their own logic, account for their own experiences, and come to their own conclusions.
What science is supposed to offer is a crystalline view of measurable and provable data. It doesn’t help if the scientists pick up a hammer and smash the crystal when presenting it to public view.
As I know all too well, education is not the same as intelligence; intelligence is not the same as sense; sense is not the same as integrity.
Read studies for yourself. Practice makes perfect: the more you do it, the easier it gets, and the more accurate (and potentially shocking!) your understanding becomes.
Jump in here and comment on your own experiences. I’d love to hear from you about your adventures with this.
THE STUDY:
http://www.sciencedaily.com/releases/2011/05/110518080059.htm?utm_source=feedburner&utm_medium=email&utm_campaign=Feed%3A+sciencedaily%2Fmind_brain+%28ScienceDaily%3A+Mind+%26+Brain+News%29
