Forum adverts like this one are shown to any user who is not logged in. Join us by filling out a tiny 3 field form and you will get your own, free, dakka user account which gives a good range of benefits to you: No adverts like this in the forums anymore. Times and dates in your local timezone. Full tracking of what you have read so you can skip to your first unread post, easily see what has changed since you last logged in, and easily see what is new at a glance. Email notifications for threads you want to watch closely. Being a part of the oldest wargaming community on the net. If you are already a member then feel free to login now.

http://www.sciam.com/article.cfm?id=how-anecdotal-evidence-can-undermine-scientific-results

Throw off your evolutionary shackles, brothers and sisters, and embrace the new way of science!

It's simple Confirmation bias which tricks people, especially those who don't think analytically.

eg. "I see that red cars have accidents more often than other colours. Therefore red paint on cars makes them more dangerous." Real reason? Maybe aggressive, reckless people have a tendancy to buy red cars, or maybe there are just more red cars on the road than of any other colour.

Problem is that people understand anecdotal evidence and receive it from trusted sources like friends. They don't trust it from people they don't know, like scientists. Regardless of the disparity in quality of information and argument presented to them in each case they plump for the anecdotal one. You get it with people who practice water dowsing, they say "It usually works for me!". No it doesn't, you happen to strike liky and forget all the failed attempts meaning you think you have a higher sucess rate than you are having in reality. Tests on water dowsning have demonstrated that people don't have a greater chance of correctly identifying a water source than someone having a guess.

It's also the reason people go in for these quack 'medicines'. You see things like homeopathy being offered, basically expensive water, and people say "It cured me". But there is no evidence to support the value of homeopathic medicine beyond the benefits of placebo effect. The demonstrable benefits come from having a long consultation with a 'doctor' (which is more than the 5 mins you get under the NHS) and pure placebo effect which should not be underestimated. But here's the catch, these homeopathic practitioners dress up their 'science' with fallacious pseudoscientific nonsense and give you a hugely disproportionate fee for their fairly minimal services, which are a basically chat and a bottle of water.

This is why whenever my friends make any statements I say to them " you, I only listen to scientists", then walk away.

It would be interesting to do a study to see how this idea and Gestalt psychology (may) connect. A study may be a bit much, maybe just a few unfounded accusations will do.

It's a shame when people take anecdotal "evidence" above science; especially in it's abused for the sake of profit in that way.

Great article. I have this trouble all the time with my folks, who will believe some bloke down the pub over me about things scientific, despite the fact that I am a scientist.

You know what? I used to be real gung-ho on this. But really, what's the point. For instance, half of these quack medical products are pushed by physicians. One of the news networks had a special not long ago where they got a dermatologist to endorse a product as safe and effective when she had seen no evidence at all of its purported effects ( They produced a fake infomercial to promote a skin care pill that in reality only contianed Nestle Quick chocolate drink - Incidentally, most of the people who tried it said it worked for them).

I can completely understand the drive to understand the world around you, to know things. I was once very motivated by that, but I have found that in the years of study in my field I have learned very little that brought me any comfort or certainty, and have learned things that can keep me up at night.

Ultimately, we verify what we consider facts by repeatability. In other words, I "know" that an object dropped will accelerate toward the earth at about 9.81 m/s^2. ( actually, that is at sea level, it can vary by altitude and, believe it or not, latitude). I know this because it has been measured many times by many experimenters. Its repeatable. So what? Basically that is sort of a democracy of reality, in other words, that is "real" because the majority of observers percieve it as being so. I believe the term used is a "consensus reality". Even accepting that, what if all the constants of the universe changed tommorrow? What if the entire universe went through a phase change, like ice melting into water?

I'm not saying I agree with all that, but it is something I sit around and think about.

Hmm, looks like a round of beat on the strawman, where we compete as to who can most vehemently condemn people who are "against science". I'll go Devil's advocate on this one, as there's a fairly legitimate case for going with your experience rather than theory.

The classic example is to ask someone. "What's the odds of rolling a 6 after rolling ten thousand ones on a d6". Virtually everyone will answer "one in six", out of fear of not looking 'rational', where the actual rational approach to the question is to realize that ten thousand ones in a roll is vanishingly unlikely with a classic d6, presume a loaded dice and answer "If you repeat that process precisely there is virtually no chance of rolling a 1, certainly less than 1%" or some equivalent. Don't underestimate the value of your experiences. Ultimately theory is just what someone tells you. We may all be having a laugh at your expense.

To be fair, that's a bit of trick question. Sure, you didn't say "fair die," but most people assume that when they hear the question. Let's also not forget that theories exist as a means of explaining experiences that has been tested. Not to quibble with your word choice, but theory has a meaning. If you meant "it's important to know when to trust in personal experience rather than follow a Rule" then I'd totally agree with you.

I think the article was interesting in that it explained why the bias is useful: false positives are almost always better than false negatives when the stakes are life or death. It was once, and could be again useful, like Wisdom teeth and our body's zeal for storing excess energy as fat.

As for science as a means for ascertaining truth... well, the question of what Truth is and how to determine will never be answered. Empirical science has a bit of a leg up in that it combines theory, testing, and experience into the pot, but it's still just one attempt to assign value and meaning to our collective experiences.

40kenthusiast wrote:Hmm, looks like a round of beat on the strawman, where we compete as to who can most vehemently condemn people who are "against science". I'll go Devil's advocate on this one, as there's a fairly legitimate case for going with your experience rather than theory.

The classic example is to ask someone. "What's the odds of rolling a 6 after rolling ten thousand ones on a d6". Virtually everyone will answer "one in six", out of fear of not looking 'rational', where the actual rational approach to the question is to realize that ten thousand ones in a roll is vanishingly unlikely with a classic d6, presume a loaded dice and answer "If you repeat that process precisely there is virtually no chance of rolling a 1, certainly less than 1%" or some equivalent. Don't underestimate the value of your experiences. Ultimately theory is just what someone tells you. We may all be having a laugh at your expense.

I think when people ask that question they are refering to hypothetical dice, not real ones. I think anyone would check for a loaded die at that point. I also don't understand the second part..if you're talking about a dice loaded to roll ones, wouldnt the probability be very high of rolling another one?

Streaks do happen. The ubiquitous boards that tell you the last set of numbers rolled on the roulette wheel have made the casinos tons of money, that is why they have them. In fact, I think they call it the gambler's fallacy

40kenthusiast wrote:Hmm, looks like a round of beat on the strawman, where we compete as to who can most vehemently condemn people who are "against science". I'll go Devil's advocate on this one, as there's a fairly legitimate case for going with your experience rather than theory.

The classic example is to ask someone. "What's the odds of rolling a 6 after rolling ten thousand ones on a d6". Virtually everyone will answer "one in six", out of fear of not looking 'rational', where the actual rational approach to the question is to realize that ten thousand ones in a roll is vanishingly unlikely with a classic d6, presume a loaded dice and answer "If you repeat that process precisely there is virtually no chance of rolling a 1, certainly less than 1%" or some equivalent. Don't underestimate the value of your experiences. Ultimately theory is just what someone tells you. We may all be having a laugh at your expense.

You haven't come out on the side of "experience" at all. You still applied logic to reach your conclusion. The only difference between the person who answers "1 in 6" and "less than 1%" (which, incidentally, doesn't actually make sense as the die would have been loaded to fall on a 1) is that their set of relevant data is different.

In any case, the point of this article is that personal experience should not trump collective experience, as represented by scientific process. Which is, afterall, simply the act of obtaining conclusions from the broadest possible set of data, or experiences.

This is pretty interesting to me because it's relevant to a field I work in: bio-ethics. There's currently a debate in bio-ethics about something called "Evidence-based Medicine".

Seems pretty hard to argue against evidence-based medicine, eh?

Well, people do and do so successfully. They argue successfully because they argue on the basis of practice, what doctors actually do and what they're able to do. Something that most practicing physicians do not do, and are not able to do, is to digest the massive amount of scientific studies out there in a way that complements their experience in the medical field.

Partly this is a problem with the way studies are written (abstracts so abstract that they misrepresent the content of the article, studies so poorly conceived that just about anything can be concluded from the data, plenty of field-specific technical jargon, and trained scientists doing their own writing rather than letting trained technical writers do it, etc, ad nauseum). And partly this is a problem of physicians having only so many hours in a day for professional development, having been trained to deal with specific patients and their histories rather than the abstracted groups used in medical studies, and generally being human.

It's basically economies of scale: while personal experience may be more prone to error, it's less costly to employ personal experience than it is to employ scientifically valid resources. There's a neat little game-theoretic model of this, where you have two jars of marbles. Each jar contains 90 marbles of one shade and 10 marbles of another shade. Now, personal experience is basically like choosing one marble and deciding that the majority is like that one marble, while a statistically valid method is basically like choosing ten marbles at random, and deciding the majority is like the majority of that selection. Both could be wrong, of course, but the statistically valid method is more likely to be correct. The problem is that picking ten marbles may be prohibitively expensive, and in fact more expensive than the relatively small increase in accuracy warrants.

The problem with (Western) science is that it doesn't actually know everything. It knows a lot, but there are a lot of things (physical phenomena) that cannot be explained very well by Western science.

Acupuncture comes to mind as a pretty good example. It's been around for a few thousand years, works very well, but doesn't fit well with Western Science.

That's a straw man. The efficacy of science (science is science, there is no special "Western" science) is predicated on several highly effective ideas:

(1) The idea that we don't know everything.

(2) The idea that we may be wrong about what we know.

(3) The application of mathematics and logic.

I'm reminded of a debate my MA thesis supervisor had with a PhD candidate chemistry student, on intelligent design. The PhD candidate gave a compelling argument that the chemistry of DNA meant that it could not have developed on its own from simpler organic acids. My adviser, stood up, agreed whole-heartedly that our current science of organic chemistry could not explain the process in question, and then cheerfully pointed out that turning to an intelligent designer as an explanation for the obvious success of the process wasn't anywhere near the sensible thing to do. The sensible thing to do, of course, was to come up with a better theory of organic chemistry.

Howard A Treesong wrote:

eg. "I see that red cars have accidents more often than other colours. Therefore red paint on cars makes them more dangerous." Real reason? Maybe aggressive, reckless people have a tendancy to buy red cars, or maybe there are just more red cars on the road than of any other colour.

no its true, red cars have more accidents because RED WUNZ GO FASTA!

JohnHwangDD wrote:The problem with (Western) science is that it doesn't actually know everything. It knows a lot, but there are a lot of things (physical phenomena) that cannot be explained very well by Western science.

Acupuncture comes to mind as a pretty good example. It's been around for a few thousand years, works very well, but doesn't fit well with Western Science.

Actually the German government health service did a major analysis of acupuncture which showed it did not work in the way claimed by its practitioners

That is, it turned out that sticking needles into people with elaborate ceremonial did as well at alleviating their symptoms as sticking the needles in the right places with elaborate ceremonial.

The scientific explanation of this is that it is the ceremonial surrounding acupuncture which causes its beneficial effect, rather than controlling the flow of chi through body meridians. In other words, acupuncture has a similar effect to a placebo.

The effects of placebos are well attested by research. So western science has explained the effect of acupuncture.

Nurglitch wrote:...

...

...

It's basically economies of scale: while personal experience may be more prone to error, it's less costly to employ personal experience than it is to employ scientifically valid resources. There's a neat little game-theoretic model of this, where you have two jars of marbles. Each jar contains 90 marbles of one shade and 10 marbles of another shade. Now, personal experience is basically like choosing one marble and deciding that the majority is like that one marble, while a statistically valid method is basically like choosing ten marbles at random, and deciding the majority is like the majority of that selection. Both could be wrong, of course, but the statistically valid method is more likely to be correct. The problem is that picking ten marbles may be prohibitively expensive, and in fact more expensive than the relatively small increase in accuracy warrants.

This is what economists call search and transaction costs.

Most people don't understand enough about statistics to draw conclusions from the kind of stuff that gets reported in newspapers.

There is a great book called "How To Lie With Statistics" which everyone should read.

http://www.amazon.co.uk/How-Lie-Statistics-Penguin-Business/dp/0140136290/ref=sr_1_1?ie=UTF8&s=books&qid=1217017169&sr=8-1

It won't teach you statistics but it will teach you ways to look critically at the kind of statistically based 'evidence' used by politicians, newspapers and advertisers.

I always just assume that all statistics are for funsies and rely on my opinion of the source to determine whether or not to believe the conclusion. For almost every study done, I do not have the expertise nor the time to re-examine the study's parameters in a qualified manner, so it's of no use to me.

Personally, I tend to view scientists the way I would view the clergy of a large religion.

They have their dogmas and they have their agendas, both large scale as well as personal. Some are excellent, while others are up for sale to the highest bidder.

I trust science but I don't trust scientists, and I think it is a mistake to take any conclusion a scientist gives at face value.

I'm sensing heresy in this thread. A signal has been sent out and an Inquisitor will arrive soon to help with this problem. Forcefully.

You know what's real fun? Studies that show stuff like "Women in 10-minute dating events are more likely to choose rich men" or "Men who are physically attractive (on a BMI scale) are more likely to have multiple sexual partners." Then there was the psych prof who told us about a study that said that people will follow the orders of a guy in a labcoat just cuz.

I would have never made these correlations without these helpful studies.

Kilkrazy wrote:
There is a great book called "How To Lie With Statistics" which everyone should read.

http://www.amazon.co.uk/How-Lie-Statistics-Penguin-Business/dp/0140136290/ref=sr_1_1?ie=UTF8&s=books&qid=1217017169&sr=8-1

To be fair though, if you're the type of person who's going to buy a book called "How to lie with statistics",, you should probably already know how to lie with statistics. I probably won't buy this book, but I could still write a review along the lines of this http://www.amazon.co.uk/review/product/0140136290/ref=cm_cr_pr_hist_3?%5Fencoding=UTF8&filterBy=addThreeStar

Sorry, I was going to write something witty but I need to sleep for some hours and sober up. Basically, what I'm trying to say is that anyone with a modicum of intelligence should know that you can make some arguments with statistics.

I thought this was quite funny

Maths eh? Relating it to real life? Yup. Does it, and quite well. And I can only do maths to a level standard.

Doctor Thunder wrote:Personally, I tend to view scientists the way I would view the clergy of a large religion.

They have their dogmas and they have their agendas, both large scale as well as personal. Some are excellent, while others are up for sale to the highest bidder.

I trust science but I don't trust scientists, and I think it is a mistake to take any conclusion a scientist gives at face value.

Still, compared to clergymen, at least the scientists sell their souls to rational, real-world entities like corporations, as opposed to imaginary, nebulous sky-tyrants.

Ultimately, we verify what we consider facts by repeatability. In other words, I "know" that an object dropped will accelerate toward the earth at about 9.81 m/s^2. ( actually, that is at sea level, it can vary by altitude and, believe it or not, latitude). I know this because it has been measured many times by many experimenters. Its repeatable. So what? Basically that is sort of a democracy of reality, in other words, that is "real" because the majority of observers percieve it as being so. I believe the term used is a "consensus reality". Even accepting that, what if all the constants of the universe changed tommorrow? What if the entire universe went through a phase change, like ice melting into water?

Funny you talk about this. Normally this is crap you spout when you're smoking some grass with some friends. However, "SEED" wrote an article about this topic in this month's magazine. Lucky for everyone, they have a full online version.

Agamemnon2 wrote:

Doctor Thunder wrote:Personally, I tend to view scientists the way I would view the clergy of a large religion.

They have their dogmas and they have their agendas, both large scale as well as personal. Some are excellent, while others are up for sale to the highest bidder.

I trust science but I don't trust scientists, and I think it is a mistake to take any conclusion a scientist gives at face value.

Still, compared to clergymen, at least the scientists sell their souls to rational, real-world entities like corporations, as opposed to imaginary, nebulous sky-tyrants.

Well, that distinction is pretty academic for me. If a source is so biased that I cannot trust the information it gives, the reason for the source's bias is tangential to the fact that I can't trust it.

I suppose it is relevant in the sense that a clergy of a real religion have a stated and open agenda, while scientists wear the pretext of being unbiased when they are not, so there is a level of hypocrisy among scientists that I find distasteful.

Still, I don't want the thread to become a discussion about religion vs. science, I was just using it as an example.

Grignard wrote: I was once very motivated by that, but I have found that in the years of study in my field I have learned very little that brought me any comfort or certainty, and have learned things that can keep me up at night.

QFT.

As for intelligent design, and the theory of organic chemistry, that's a very nice anecdotal piece of evidence. It also doesn't really help us understand more about organic chemistry (or anything for that matter) to fall back on intelligent design. It's like turning on a light bulb and claiming it is "magic". Well that's all well and good, but what happens when you want to change the lightbulb, or build another one, or explain to your friend how to use it. Science tells us "how" and people turn to religion/faith to explain "why". These things should not be mutually exclusive. If you can believe that God created the universe out of nothing, how does that interfere with the big bang theory? What if God was the Big Bang, or started it all off with his divine power? Anyone, flameshield on, because I know that some of you will probably disagree, but that's how I see it. Science = how, Faith = why or everything else you might need.

And Doc Thunder, what type of scientists are you referring to that are biased, and for that matter, which clergymen?

whitedragon wrote:Science tells us "how" and people turn to religion/faith to explain "why". These things should not be mutually exclusive.

I completely agree. As Einstein once said, science without religion is lame, and religion without science is blind.

DONT BE TAKEN IN!

I was skeptical about that article, until I read their baised and unsubstantiated attack on wheat grass. Then I knew they could no longer be trusted. Wheat grass saved my life and made me a better person. That article was obviously published by the shills for the macrobiotic diet, to undermine public confidence in wheat grass.



Did you read the comments? Three pages of people arguing about "Nuh uh, vaccines do TOO cause autism!" The best is last, though- beware the dangers of evolutionarily highly risky reasoning, and use ID safe approach which only needs to be reasonable to win approval from logical minds. Comedy gold.

JohnHwangDD wrote:The problem with (Western) science is that it doesn't actually know everything. It knows a lot, but there are a lot of things (physical phenomena) that cannot be explained very well by Western science.

Acupuncture comes to mind as a pretty good example. It's been around for a few thousand years, works very well, but doesn't fit well with Western Science.

Are you Azn?