Switch Theme:

Can we trust peer-reviewed papers?  [RSS] Share on facebook Share on Twitter Submit to Reddit
»
Author Message
Advert


Forum adverts like this one are shown to any user who is not logged in. Join us by filling out a tiny 3 field form and you will get your own, free, dakka user account which gives a good range of benefits to you:
  • No adverts like this in the forums anymore.
  • Times and dates in your local timezone.
  • Full tracking of what you have read so you can skip to your first unread post, easily see what has changed since you last logged in, and easily see what is new at a glance.
  • Email notifications for threads you want to watch closely.
  • Being a part of the oldest wargaming community on the net.
If you are already a member then feel free to login now.




Made in us
Wise Ethereal with Bodyguard




Catskills in NYS




Thought you guys might find this interesting, I certainly did.

Homosexuality is the #1 cause of gay marriage.
 kronk wrote:
Every pizza is a personal sized pizza if you try hard enough and believe in yourself.
 sebster wrote:
Yes, indeed. What a terrible piece of cultural imperialism it is for me to say that a country shouldn't murder its own citizens
 BaronIveagh wrote:
Basically they went from a carrot and stick to a smaller carrot and flanged mace.
 
   
Made in au
Thinking of Joining a Davinite Loge






This... article? is making a bogus point.

What he's saying is the equivalent of listening to the neighbourhood quack and then saying you can't trust your GP because the quack led you up the garden path.

There's a reason we have multiple statistical analysis to determine Journals power and thoroughness and any other metric you can dream up.

There are Journals that are beyond repute, some that are good, and others that need a bargepole.

And yes, there have been issues with peer review-most of these scandals are quite old, and of the thousands, if not millions, of reputable papers that have been published, there's a tiny handful that have failed to conform-we're always upgrading our standards and guidelines.

This message was edited 1 time. Last update was at 2017/07/16 18:25:34


My $0.02, which since 1992 has rounded to nothing. Take with salt.
Elysian Drop Troops, Dark Angels, 30K
Mercenaries, Retribution
Ten Thunders, Neverborn
 
   
Made in us
Wise Ethereal with Bodyguard




Catskills in NYS

 Farseer Anath'lan wrote:
This... article? is making a bogus point.

What he's saying is the equivalent of listening to the neighbourhood quack and then saying you can't trust your GP because the quack led you up the garden path.

There's a reason we have multiple statistical analysis to determine Journals power and thoroughness and any other metric you can dream up.

There are Journals that are beyond repute, some that are good, and others that need a bargepole.

And yes, there have been issues with peer review-most of these scandals are quite old, and of the thousands, if not millions, of reputable papers that have been published, there's a tiny handful that have failed to conform-we're always upgrading our standards and guidelines.

Did you watch the video? That's his point.

An entire section is dedicated to pointing out how to tell the difference between a reputable journal and a fraudulent one.

And then it talks about why these bogus journals are a bad thing.

This message was edited 1 time. Last update was at 2017/07/16 18:30:06


Homosexuality is the #1 cause of gay marriage.
 kronk wrote:
Every pizza is a personal sized pizza if you try hard enough and believe in yourself.
 sebster wrote:
Yes, indeed. What a terrible piece of cultural imperialism it is for me to say that a country shouldn't murder its own citizens
 BaronIveagh wrote:
Basically they went from a carrot and stick to a smaller carrot and flanged mace.
 
   
Made in us
Longtime Dakkanaut





Denison, Iowa

There are several times things like this have been picked up on by society. Media outlets and politicians cling to a "peer reviewed" article that has little basis in reality and then base public policy around it. Never mind the fact that the study quotes non-experts, or misquotes actual experts, or misrepresents data.

I did a thesis paper on this in college, I'll see if I can find it. One of the effects of this I found was that, for a time, an invasive species was actually protected by law.
   
Made in jp
[MOD]
Anti-piracy Officer






Somewhere in south-central England.

There have been many examples of dodgy science, even in peer reviewed papers. The main problem is when researchers falsify their data in some way to get the "right" result. The paper can look totally valid through peer review.

That doesn't invalidate the idea of peer review. The scientific method means that when people try to replicate the results, if they are honest, they will fail, and this will invalidate the original paper. That's what happened to the MMR vaccine paper by ex-Dr Wakefield.

Probably a worse problem is the non-publication of research that finds out the "wrong" results. That's why an increasing number of scientists are calling for all research proposals to be publicly registered.

Another aspect of research validation is the "Clarendon" meta-review. (I might have got the name wrong.) It looks at all the available science on a particular topic. Any real problems soon show up.

I'm writing a load of fiction. My latest story starts here... This is the index of all the stories...

We're not very big on official rules. Rules lead to people looking for loopholes. What's here is about it. 
   
Made in us
Longtime Dakkanaut





Denison, Iowa

ex-Dr Wakefield.... Man, if ever there was a fraud it was this guy.

I'm still in awe whenever I see people refer to his work as a way to validate not vaccinating their children. This movement has set the world back a decade at least and is responsible for dozens of needless deaths.
   
Made in us
[DCM]
Secret Squirrel






Leerstetten, Germany

 Kilkrazy wrote:
There have been many examples of dodgy science, even in peer reviewed papers. The main problem is when researchers falsify their data in some way to get the "right" result. The paper can look totally valid through peer review.

That doesn't invalidate the idea of peer review. The scientific method means that when people try to replicate the results, if they are honest, they will fail, and this will invalidate the original paper. That's what happened to the MMR vaccine paper by ex-Dr Wakefield.

Probably a worse problem is the non-publication of research that finds out the "wrong" results. That's why an increasing number of scientists are calling for all research proposals to be publicly registered.

Another aspect of research validation is the "Clarendon" meta-review. (I might have got the name wrong.) It looks at all the available science on a particular topic. Any real problems soon show up.


I think an extension of this is the focus on doing original research, frequently as a way of obtaining funding for existing, leading to a low number of studies being replicated and examined for validity.
   
Made in gb
Ridin' on a Snotling Pump Wagon






 cuda1179 wrote:
ex-Dr Wakefield.... Man, if ever there was a fraud it was this guy.

I'm still in awe whenever I see people refer to his work as a way to validate not vaccinating their children. This movement has set the world back a decade at least and is responsible for dozens of needless deaths.


If I'm right in thinking, the anti-vax idiots aren't even getting his bogus conclusions right.

I'm sure his claim was a specific, combined Measels, Mumps and Rubella jab may be linked to autism - but the three individual jabs that just happened, as things do, to be sold by the company sponsoring his paper were/are perfectly safe.

So it's was never 'vaccines cause autism'. Just 'this one vaccine might'

This message was edited 1 time. Last update was at 2017/07/17 21:16:30


   
Made in gb
Lord Commander in a Plush Chair





Beijing

They were starting to talk about at least one of the MMR diseases, measles or mumps I think, going the same way as Smallpox. It's no where near now, that's long set back.

Some fraud is deliberately carried out. The fraudulent evidence reported about stem cells by South Korean scientists some years ago was published, and later redacted from Nature.

Some reputable publications have compromised themselves when rushing to get an article out. National Geographic got egg all over their face when publishing a missing link story a few years ago based on a fraudulent fossil. This fraud wasn't supposed to decieve the academic world, just entice a buyer for fossils. A black market of fossils exported from abroad sometimes has people put together things that ought not to be together. But some academics saw the fossil, and seized upon its significance and rushed to publish before testing in detail, just to get the scoop.

All said though, given the sheer volume of work published every month these occurrences are very rare.

This message was edited 1 time. Last update was at 2017/07/17 21:24:21


 
   
Made in us
5th God of Chaos! (Yea'rly!)




The Great State of Texas

 Howard A Treesong wrote:
They were starting to talk about at least one of the MMR diseases, measles or mumps I think, going the same way as Smallpox. It's no where near now, that's long set back.

Some fraud is deliberately carried out. The fraudulent evidence reported about stem cells by South Korean scientists some years ago was published, and later redacted from Nature.

Some reputable publications have compromised themselves when rushing to get an article out. National Geographic got egg all over their face when publishing a missing link story a few years ago based on a fraudulent fossil. This fraud wasn't supposed to decieve the academic world, just entice a buyer for fossils. A black market of fossils exported from abroad sometimes has people put together things that ought not to be together. But some academics saw the fossil, and seized upon its significance and rushed to publish before testing in detail, just to get the scoop.

All said though, given the sheer volume of work published every month these occurrences are very rare.


Inversely we now have drug resistant tuberculosis and gonorhea appearing. One of the big killers becoming drug resistant and we are back to the middle ages.*


*Except for the rad filled muties in New Mexico. They have proven resistant to all forms of disease, but not belly rubs.

-"Wait a minute.....who is that Frazz is talking to in the gallery? Hmmm something is going on here.....Oh.... it seems there is some dispute over video taping of some sort......Frazz is really upset now..........wait a minute......whats he go there.......is it? Can it be?....Frazz has just unleashed his hidden weiner dog from his mini bag, while quoting shakespeares "Let slip the dogs the war!!" GG
-"Don't mind Frazzled. He's just Dakka's crazy old dude locked in the attic. He's harmless. Mostly."
-TBone the Magnificent 1999-2014, Long Live the King!
 
   
Made in au
Grizzled Space Wolves Great Wolf





26 minutes? Can we get a brief summary? I don't have that kind of time on my hands

As someone who works in research, yeah, journals should be considered a building of knowledge rather than an infallible set of data and conclusions.

One of the big risks at times is ignorant fools (sometimes the regular uneducated media) taking things out of context. You'll read an article that said science said something and try and go back to the source article and realise the actual scientists were much more conservative in their conclusions than the people who actually feed that information to the public.
   
Made in us
Wise Ethereal with Bodyguard




Catskills in NYS

AllSeeingSkink wrote:
26 minutes? Can we get a brief summary? I don't have that kind of time on my hands

As someone who works in research, yeah, journals should be considered a building of knowledge rather than an infallible set of data and conclusions.

One of the big risks at times is ignorant fools (sometimes the regular uneducated media) taking things out of context. You'll read an article that said science said something and try and go back to the source article and realise the actual scientists were much more conservative in their conclusions than the people who actually feed that information to the public.

Ummm. In short pay to publish online predatory journals have led to a lot of bogus articles being published (and often used by politicians to attack legitimate science). And it points out the ways to tell bogus journals and papers from real ones.

Homosexuality is the #1 cause of gay marriage.
 kronk wrote:
Every pizza is a personal sized pizza if you try hard enough and believe in yourself.
 sebster wrote:
Yes, indeed. What a terrible piece of cultural imperialism it is for me to say that a country shouldn't murder its own citizens
 BaronIveagh wrote:
Basically they went from a carrot and stick to a smaller carrot and flanged mace.
 
   
Made in au
Grizzled Space Wolves Great Wolf





 Co'tor Shas wrote:
AllSeeingSkink wrote:
26 minutes? Can we get a brief summary? I don't have that kind of time on my hands

As someone who works in research, yeah, journals should be considered a building of knowledge rather than an infallible set of data and conclusions.

One of the big risks at times is ignorant fools (sometimes the regular uneducated media) taking things out of context. You'll read an article that said science said something and try and go back to the source article and realise the actual scientists were much more conservative in their conclusions than the people who actually feed that information to the public.

Ummm. In short pay to publish online predatory journals have led to a lot of bogus articles being published (and often used by politicians to attack legitimate science). And it points out the ways to tell bogus journals and papers from real ones.
Ah ok.

Most people don't actually have access to quality modern journal articles anyway so rely on 3rd party sources for their scientific information. Unless you're enrolled in a University and that University has paid for journal access, most high level journals can't be accessed. I didn't pay to have my articles published, but to access my published articles (beyond the abstract) you have to pay or have access through an education/research facility.

That and most people don't have the patience to actually go back to the source material anyway If they do, there's a good chance they're already well educated enough to know the difference between good journals and dodgy journals.

I think even good science and good research is misused though. When you publish an article you 1. present data, 2. make some conclusions from it and usually 3. you push a bit beyond your data to make some guesses about what that might mean in a wider context. Researchers take care to frame their discussion in a way that's it's clear what are direct conclusions and what are guesses, but people who aren't themselves researchers will often take point 3 and act like it's point 2.

This message was edited 2 times. Last update was at 2017/07/18 13:53:18


 
   
Made in gb
[SWAP SHOP MOD]
Killer Klaivex







AllSeeingSkink wrote:

Most people don't actually have access to quality modern journal articles anyway so rely on 3rd party sources for their scientific information. Unless you're enrolled in a University and that University has paid for journal access, most high level journals can't be accessed. I didn't pay to have my articles published, but to access my published articles (beyond the abstract) you have to pay or have access through an education/research facility.

Or use Scihub.

In all seriousness, none of this is news to anyone in the academic world. You see a newspaper doing a 'shocking reveal' of pay to publish once a year in the mainstream press, but nobody in the first world who seriously wants to build academic kudos publishes in 'Peoples Front of Judea Weekly' or suchlike. And no other academic tends to pay any attention to it.

This message was edited 1 time. Last update was at 2017/07/18 20:08:39



 
   
Made in ca
Longtime Dakkanaut




Building a blood in water scent

 Ketara wrote:


In all seriousness, none of this is news to anyone in the academic world. You see a newspaper doing a 'shocking reveal' of pay to publish once a year in the mainstream press, but nobody in the first world who seriously wants to build academic kudos publishes in 'Peoples Front of Judea Weekly' or suchlike. And no other academic tends to pay any attention to it.


Splitters! It's the Judean People's Front!

We were once so close to heaven, St. Peter came out and gave us medals; declaring us "The nicest of the damned".

“Anti-intellectualism has been a constant thread winding its way through our political and cultural life, nurtured by the false notion that democracy means that 'my ignorance is just as good as your knowledge.'” 
   
Made in us
Wise Ethereal with Bodyguard




Catskills in NYS

 Ketara wrote:
AllSeeingSkink wrote:

Most people don't actually have access to quality modern journal articles anyway so rely on 3rd party sources for their scientific information. Unless you're enrolled in a University and that University has paid for journal access, most high level journals can't be accessed. I didn't pay to have my articles published, but to access my published articles (beyond the abstract) you have to pay or have access through an education/research facility.

Or use Scihub.

In all seriousness, none of this is news to anyone in the academic world. You see a newspaper doing a 'shocking reveal' of pay to publish once a year in the mainstream press, but nobody in the first world who seriously wants to build academic kudos publishes in 'Peoples Front of Judea Weekly' or suchlike. And no other academic tends to pay any attention to it.

The issue is, of course, when our leaders use these studies.

Homosexuality is the #1 cause of gay marriage.
 kronk wrote:
Every pizza is a personal sized pizza if you try hard enough and believe in yourself.
 sebster wrote:
Yes, indeed. What a terrible piece of cultural imperialism it is for me to say that a country shouldn't murder its own citizens
 BaronIveagh wrote:
Basically they went from a carrot and stick to a smaller carrot and flanged mace.
 
   
Made in gb
Lord Commander in a Plush Chair





Beijing

It's doesn't help that a lot of 'science' reported in newspapers is just rubbish. Anything where someone has come up with a 'formula' for the 'somethingest day of the year' is paid for and promoted by a marketing company somewhere.

Often stories where 'scientists have shown that eating buttered toast every day is good for you' is research paid for by some spreadable fat company or bread manufacturer, bunging some scientist money to be a shill for them. This sort of 'research' goes into the popular press but rarely makes the grade for an academic journal, or was a mere footnote in a much wider piece of research that marketing companies want to sieze upon out of context.

This message was edited 1 time. Last update was at 2017/07/19 06:37:50


 
   
Made in au
The Dread Evil Lord Varlak





AllSeeingSkink wrote:
As someone who works in research, yeah, journals should be considered a building of knowledge rather than an infallible set of data and conclusions.


Yeah, this is a major element. Peer review is a process, a process that allows science to fail forward, improving steadily over time. It isn't a means of creating absolute truth.

This means, as you say, that journals are part of a process of creating a depth and breadth of knowledge. Unfortunately most people read a summary of a summary on one study in a whole field, and carry that with them as the absolute truth for years afterwards.

“We may observe that the government in a civilized country is much more expensive than in a barbarous one; and when we say that one government is more expensive than another, it is the same as if we said that that one country is farther advanced in improvement than another. To say that the government is expensive and the people not oppressed is to say that the people are rich.”

Adam Smith, who must have been some kind of leftie or something. 
   
Made in gb
Longtime Dakkanaut




Read up about the 'eating chocolate is good for you' debacle, that was basically some scientists showing just how shocking the state of current journalistic integrity is

i.e. non-existent.
   
 
Forum Index » Off-Topic Forum
Go to: