Your personal opinion does not trump scientific studies

Forum adverts like this one are shown to any user who is not logged in. Join us by filling out a tiny 3 field form and you will get your own, free, dakka user account which gives a good range of benefits to you:

No adverts like this in the forums anymore.
Times and dates in your local timezone.
Full tracking of what you have read so you can skip to your first unread post, easily see what has changed since you last logged in, and easily see what is new at a glance.
Email notifications for threads you want to watch closely.
Being a part of the oldest wargaming community on the net.

If you are already a member then feel free to login now.

**BrassScorpion**

I hear some of the ridiculous arguments shot down here all too often, so I found this article particularly amusing:

http://www.blaghag.com/2010/05/your-personal-opinion-does-not-trump.html

Your personal opinion does not trump scientific studies

As a scientist, one of my big pet peeves is when someone tries to use a personal anecdote to disprove a scientific study. "Cigarette are bad for you?! But my grandpa chain smoked until he was 96, and he was healthy as an ox!"

Great for your grandpa! ...But that's irrelevant.

The whole purpose of science is to reduce our biases. Looking at your sample size of one (Grandpa) is going to lead you to the wrong conclusion about what's going on with smoking. Your grandpa was an outlier - and while that is interesting, the vast majority of people suffer harmful effects from smoking.

But my bigger pet peeve is when someone's culture, personal opinion, or political belief stands in the way of them accepting science.

For example, during our unit on aggression in my Social Psychology class, we talked about cultural causes for aggression. One example was the Southern Culture of Honor. People who grow up in this culture see a perceived insult as a threat to their ego, which increases testosterone levels* and violent cognitions, and can lead to acts of violence. Southern cities and states have much higher White homicide rates than those populated by northerners**, and in Southern states homicides exceed suicides.

Effects of Insults on Testosterone levels in Southerner and Northerner Participants

When I mentioned this in a tweet, some of my Southern followers got angry and said it wasn't true, and tried to provide anecdotal evidence about how kind and helpful Southerners are. Your neighbors may be sweet, but that doesn't negate an overall trend. Scientific studies aren't saying that all southerners are homicidal maniacs. Though you know, getting angry at a perceived insult doesn't exactly help your cause...

Another topic within aggression that really riles people up is spanking. Numerous studies have been done showing that spanking children increases antisocial*** and aggressive**** behavior. But when people who have been spanked or spank their children hear about this, they get very defensive. I can't recall the number of times I've heard "Well I was spanked, and I turned out fine!" or "I spanked my kids and now they're little angels!"

I'm sorry, but 1) Your specific experience does not negate the average response seen in hundreds of families, and 2) Your evaluation isn't necessarily correct. You could very well have had an increase in antisocial or aggressive behavior, but you didn't have a psychologist assessing your behavior, did you? I'd really like to see a psychological study on why people like to defend spanking. Do they hate thinking that their parents did something wrong? Do they hate having to come up with a better (and possibly less easy) disciplinary action?

And last, but not least: political beliefs that get in the way of accepting science. The one that bugs me the most are feminists who are such huge supporters of female equality that they simply cannot accept that males and females do differ in certain ways. For one, you kind of can't ignore that (biologically typical) males and females differ physically - we kind of have different reproductive organs and chromosomes. We also have different secondary sex characteristics - males are going to be slightly stronger and larger on average.

And because our biology differs, it's not insane to suggest our psychology differs. Saying men are better in some areas and women are better in others does not mean one is superior to another. Saying men may have certain mating strategies and females may have different ones does not mean one is morally superior, or that either are things we should actually do - humans are not simply slaves to their biology, after all. There are differences between the sexes in almost every species where there are two different sexes - humans aren't exempt. To deny these differences because they don't jibe with your political beliefs is simply unscientific.

Now, I know I'm not perfect. There have definitely been times where I've been skeptical of a study when I personally didn't like the results - it's human nature (especially when the study is saying something delicious is bad for your health). But the thing about being a scientist is reducing our biases as much as possible. So next time you find yourself giving anecdotal evidence, remember: Your personal opinion may be an interesting new hypothesis, but until you do a study of your own, it does not trump previous scientific research.

* Cohen et al (1996) Culture of Honor: The Psychology of Violence in the South
**Myers (2008) Social Psychology
*** Strauss et al (1997)
**** Taylor (2010) in Pediatrics

**Da Boss**

Heh, good article. I like to use the quote-
"The plural of anecdote is not data."
to express the same sentiment.

That said, looking critically at scientific studies is important. The amount of bad science that gets past peer review is shocking!

**MrDwhitey**

I approve of this.

**Kilkrazy**

When I mentioned this in a tweet, some of my Southern followers got angry and said it wasn't true

The irony, the savage irony!

**dogma**

Da Boss wrote:
That said, looking critically at scientific studies is important.

The problem is that your average Joe doesn't know how think critically about scientific studies. This is particularly true of the social sciences, because very few people seem to understand statistics, as illustrated in the article.

Da Boss wrote:
The amount of bad science that gets past peer review is shocking!

No joke.

**Da Boss**

In my experience, plenty of hard scientists don't understand statistics very well, either. A side effect of the way we mass produce PhD candidates I think.
I was a poor statistician until I got some pretty intensive training, and even then my knowledge is limited to experimental design and risk analysis.

**dogma**

Da Boss wrote:In my experience, plenty of hard scientists don't understand statistics very well, either. A side effect of the way we mass produce PhD candidates I think.
I was a poor statistician until I got some pretty intensive training, and even then my knowledge is limited to experimental design and risk analysis.

It also doesn't help that statistics are exceedingly tedious.

Multivariate calculus allows you to discover beautiful truths about the Universe, and stats allows you to perform lots of addition, and maybe fill in Excel spreadsheets.

And God help the person that needs to use R.

**warpcrafter**

There is a point where millions of corroborating anecdotes gain a weight of their own. That's what us non-scientists call common sense. Perhaps if scientists possessed some of that, their science would have a better reception.

**Da Boss**

I dunno, analysing data can be fun, if the data is interesting. But yeah, the way stats was taught at undergrad did not endear the subject to me. I think epidemiology is probably one of the few fields where it gets really exciting.

The stats in most social science studies is complete bunk anyway due to various sampling biases and subjective measures though. I wish they'd stop trying to appropriate the scientific method for things it's not well suited to "proving". Some things are better investigated using other paradigms.

Edit: Warpcrafter, we're not bothered about the reception of our work. We're bothered about getting to the bottom of things. Common sense often isn't.

**Soladrin**

warpcrafter wrote:There is a point where millions of corroborating anecdotes gain a weight of their own. That's what us non-scientists call common sense. Perhaps if scientists possessed some of that, their science would have a better reception.

But if you compile all those anecdotes to form a defense or whatever, it's not anecdotal anymore, it's a statistic.

**Ahtman**

Scientific studies show that common sense doesn't actually exist as a real thing, but as a rhetorical tool for dismissing scientific studies.

**Da Boss**

Soldarin:No it isn't. Or at least, it's not a very reliable statistc. To tell anything from statistics you need to make sure your samples are a true representation of the total population (whatever that is), by ensuring a truly random sample and a sufficiently large population size. The plural of anecdote is not data because these sampling criteria will not be met.

**Soladrin**

I'd think that if theres literaly a million (as warpcraft said) of individual anecdotes that are all the same it's pretty random and of a sufficiently large population...

**Da Boss**

Not really. It MIGHT be, but I'd be extremely skeptical. It depends bigtime on how the anecdotes are collected and what they are in relation to.

**biccat**

Awwww...look at the social scientist pretending that his studies actually prove something.

How cute.

**Soladrin**

Obvious troll is obvious.

And Da Boss, I by no means claim to know what the hell we are talking about, it's just how I thought it would work.. somehow.

**biccat**

Soladrin wrote:Obvious troll is obvious.

Social 'scientists' see correlation and view it as causation. They don't conduct experiments (mainly because they'd be unethical as HELL) to reduce variables and isolate causes, they manipulate statistics to propose how people behave. You can use statistical analysis to isolate variables and form a hypothesis, but it's not something that can be tested, nor does it explain outliers.

Take the example cited in the linked article: Cigarettes cause cancer. Someone mentions "No, my grandpa smoked for 90 years and never got cancer." A scientist would look at this as a challenge to his hypothesis and either reform the hypothesis or otherwise distinguish the outlier. Social scientists just say "Well, he's an outlier."

That's not science, that's statistics.

**Soladrin**

That doesn't make your previous post less trolling

It's all mumbo jumbo to me anyway.

**Melissia**

Meh, I have no problem accepting that there's differences between men and women, I just think pop culture and societal expectations exaggerates them far more than they are naturally. We don't have a control for those kinds of studies.

**Soladrin**

Where are the hermaphrodites when you need them?!

**darkPrince010**

I think it's amazing people refuse to accept or even interpret scientific studies. While you have your full right to critically analyze and call aspects of a study into question (Participant number, objectivity of researchers, methods used, etc) it baffles me that some people stick their proverbial fingers in their ears and ignore valid studies. Feel free to disagree, but provide better reasons than "my little internal voice says no!"

@biccat: While I certainly believe hard science conclusions have more weight than social sciences, most social sciences actually do provide accurate and/or useful results. I agree studies like "General feelings of happiness in the US" are fairly subjective, but correlation of specific verbal cues to produce measured differences in internal hormone levels seems solid to me.

@Soladrin: A million might do it, but iirc, psychological studies that broad need to include disparate genders, ethnicities, social classes, etc in order to qualify. This is mainly to avoid getting a sample of "A million! (Middle class caucasian straight males)."

**dogma**

warpcrafter wrote:There is a point where millions of corroborating anecdotes gain a weight of their own. That's what us non-scientists call common sense. Perhaps if scientists possessed some of that, their science would have a better reception.

Sure, if people were told what they wanted to hear they would be more likely to believe what they're told. Funnily enough there's a phrase used to denote this phenomenon, one which is often used in scientific criticism.

In any case, common sense is just a euphemism for "what I believe, but cannot explain." and appeals to it generally mark a lack of intellectual rigor.

**Soladrin**

I thought common sense was... touching a hot stove hurts. Stuff like that.

**Ahtman**

darkPrince010 wrote:This is mainly to avoid getting a sample of "A million! (Middle class caucasian straight males)."

But they are the most important demo of them all!

**dogma**

biccat wrote:
Social 'scientists' see correlation and view it as causation.

Really? Is that why all those social science papers I read differentiate between correlation and causation?

biccat wrote:
They don't conduct experiments (mainly because they'd be unethical as HELL) to reduce variables and isolate causes, they manipulate statistics to propose how people behave. You can use statistical analysis to isolate variables and form a hypothesis, but it's not something that can be tested, nor does it explain outliers.

The isolation of variables is testing. You're confusing the general concept of "test" with the particular type of test that is "experiment".

biccat wrote:
Take the example cited in the linked article: Cigarettes cause cancer. Someone mentions "No, my grandpa smoked for 90 years and never got cancer." A scientist would look at this as a challenge to his hypothesis and either reform the hypothesis or otherwise distinguish the outlier. Social scientists just say "Well, he's an outlier."

So, you've never actually read a social science journal, have you? Because, while the behavior you describe does occur (just like it does in the hard sciences, go figure) it certainly isn't definitive of the the entire category of "social scientists."

biccat wrote:
That's not science, that's statistics.

They aren't mutually exclusive. Statistics are often used in the hard sciences.

Automatically Appended Next Post:

Soladrin wrote:I thought common sense was... touching a hot stove hurts. Stuff like that.

Unless you have a particular genetic disorder, or significant nerve damage.

And then we have to wonder if, while touching the stove hurts, hurting is bad. Some people enjoy pain.

**Soladrin**

Well played...

**dogma**

darkPrince010 wrote:
@biccat: While I certainly believe hard science conclusions have more weight than social sciences, most social sciences actually do provide accurate and/or useful results. I agree studies like "General feelings of happiness in the US" are fairly subjective, but correlation of specific verbal cues to produce measured differences in internal hormone levels seems solid to me.

It is also important to note that, while a study might be titled in a provocative manner, the conclusion of that study will rarely be so provocative.

Academics love nothing more than pointing out when their peers screw the pooch, so most of us are pretty good at not making overly bold claims. There are, of course, outliers (

) who make their careers on bold claims (John Mueller, Chomsky, Alex Wendt, Dicky Dawkins, Einstein), or defending them (Ken Waltz, Huntington).

**darkPrince010**

Plus, common sense for people varies. Some believe not sticking your hand in a fire or licking a frozen lamppost is common sense, while others believe the tenets of their religion or theory of evolution are common sense.

The biggest problem is when people try to use their "common sense" as a replacement for actual proof. I believe evolution is "common sense" (my belief, not nessesarily anyone else's), but unlike some beliefs it can and has been backed up by libraries of scientific data to support it.

EDIT: Also, proving a theory or conclusion wrong using hard, scientific evidence instead of saying "That goes against my gut feeling" is the entire basis of our modern process of scientific rational. If I prove the above study on "Southern Honor" wrong by using a similar test group but a set of different verbal cues or environments and get a drastically lower testosterone result, I can scientifically say the theory is not valid and needs to be reworked to account for the aspects I discovered in my study. Just saying "Well not all Southern People are hot-tempered" disproves nothing. IMO, this is why science and religion (and to a lesser degree, politics in some places) have mixed very badly in the past and present.

**dogma**

Da Boss wrote:I dunno, analysing data can be fun, if the data is interesting.

The analysis is fun, but the entry, that's a curse on men.

Da Boss wrote:
But yeah, the way stats was taught at undergrad did not endear the subject to me. I think epidemiology is probably one of the few fields where it gets really exciting.

Anything involving a survey is fun, because so very many people confuse responses to a particular question with a statement of general truth.

Obviously this happens most often in politics, in which we see very large variation in responses to questions differing by only a single word.

Da Boss wrote:
The stats in most social science studies is complete bunk anyway due to various sampling biases and subjective measures though. I wish they'd stop trying to appropriate the scientific method for things it's not well suited to "proving". Some things are better investigated using other paradigms.

Yeah, psychology is probably the worst for that. Something like 75-80% of the aggregate sample of psych studies is composed of American college students aged 18-23.

Politics and economics do alright though, as there is enough funding to sample a broad spectrum of populations.

**Howard A Treesong**

OP can be summed up thus - anecdote does not make for data.

BrassScorpion wrote:

That's a terrible graph, what does it even mean? There's no labels.

You can't seriously criticise people's use of statistics and then present a graph elsewhere with no labels on the axis or a unit of measurement (what does the y axis even measure), error bars, an indication of significance in the data...

I think I've had this topic before. People generally only want to accept science when it corroborates their personal opinion. That's why people deny reality itself to pretend that climate change is a fraud, or that the Earth is 6000 years old, or that vaccines will kill your kids.

On the subject of statistics, I've just put together a 10,000 cell Excel table today with all my statistical data for my PhD. Statistics? Don't talk to me about statistics.