Most Published Scientific Research Is Wrong

-SWS wrote:Interesting that you should mention that, Jeff.

I was just thinking how science, religion, and philosophy are attempts to know the same truths... and how often those three have historically been at odds with one another. Those three very often embrace the perception of knowing versus proving differently. As a result their conclusions of truth frequently differ as well.

One day the whole world will be as harmonious as cpaptalk.com.

And the snoredog shall also dwell with the science.

I feel the need to reply to this topic as a published medical researcher. I certainly agree that there are studies with poor methodology, poor data management and shoddy statistics. However, that doesn't mean that there is no valid research being published. There are many very careful researchers who do the best they possibly can to ensure that we have reliable research studies. The process of obtaining funding for a grant proposal is grueling and requires much careful thought, pilot studies and evaluation of the size and other particulars of the study to show the size of the expected effect. There are standards for conducting studies and major journals in each field usually are extremely careful in their review of manuscripts.

That said, are there biases? Sure! Are there mistakes? Of course! But I think it's an over-generalization to say that "most" published research is wrong. Often one study builds on top of another so initial conclusions are disproven by further research that is based on information gleaned form these studies. As part of our training we have "journal club" sessions where we have a discussion group to go over one or more published papers to evaluate the hypothesis, methods and results. We try to teach newer researchers what to look for to indicate if a paper is a potentially reliable source of information.

So, yes, we need to be very discriminating about what we read and how we evaluate it. But is most of it wrong? I doubt it. There are many studies that are too preliminary to draw working conclusions but they contribute to the general body of knowledge to eventually lead to useful results.

Sorry - I had to get that off my chest!

Mindy

Mindy, I personally don't believe that most published research is false either. Still, I give John P. A. Ioannidis great credit for a tenable essay that inspired debate in the scientific community and, more importantly, a closer look at how inconclusive and misleading science can be at times----based largely on the selection and execution of methodology. I also give loannidis credit for endeavoring to characterize relative likelihoods based on essential factors such as study size. I think he deserves credit for a notable contribution to science.

His position will continue to receive debate in the scientific community. I personally think that is so very healthy of science! But I also happen to believe much of what is published is guaranteed to be wrong.

And the snoredog shall also dwell with the science.

This thread is an amazing coincidence. I have been pursuing truth for decades. As luck would have it I finally discovered truth not more than an hour ago. It was on eBay of all places: HERE'S TRUTH .

I guess my gut response to the statement 'most published research is wrong' is to say: 'Well, yeah, duh! Of course it is! And we like it that way!' Being wrong is simply part of the process.

I would say that the productive search for scientific truth is all about being unafraid of being wrong 99% of the time. The statistics of the process will always stink, since no one is going to study what no one has any questions about.

Throwing a statement, or line of reasoning, or body of research, out for peer review in order for others to comment on it or try to prove it wrong (or right) is what I expect a good scientist to do. So I don't care if what I read turns out to be wrong, as long as it is good science.

After all, who wants to read a scientific journal filled with nothing controversial or cutting edge? It's hard enough to read that stuff as it is!

PS- How disturbing that there are TWO versions of truth for sale on E-bay! Shouldn't there only be one?! Hope you buy the right one -SWS.

Jeff, I think I resonate with your comment on page one---that most published research are essentially submissions in search of replication and collective validation. I agree that the scientific method is a highly collaborative process.

jnk wrote:PS- How disturbing that their are TWO versions of truth for sale on E-bay! Shouldn't there only be one?! Hope you buy the right one -SWS.

Hey, we're gonna need two versions of the truth if a photon particle can really be in two places at the same time.

-SWS wrote: . . .the scientific method is a highly collaborative process. . . .

When it is done "right" it is.

And if you don't mind my saying so, you, sir, do it right! If I needed to show someone an example of that principle in progress, I could confidently point to any one of your threads, at random, and say, "See, that's how it's done." Please take that for the honest compliment that it is. You do good work.

But I'm not sure photons exist until you try to figure out where they are. Slippery little suckers.

Thank you sir!

jnk wrote:But I'm not sure photons exist until you try to figure out where they are. Slippery little suckers.

I'm thinking they can be found on eBay... Got 'em! PHOTONS HERE

Yup... they can be in two places at once. Some of those same listings can be found on amazon.com as well.

jnk wrote:But I'm not sure photons exist until you try to figure out where they are. Slippery little suckers.

Just a guess here, Jeff... Do you also think that if a tree falls in the forest but there's no one there to hear it, the falling tree makes no sound?

Marsha

Just a quick comment that the fallen tree is typically auctioned off as firewood on eBay... maybe even amazon.com as well.

-SWS wrote:Interesting that you should mention that, Jeff.

I was just thinking how science, religion, and philosophy are attempts to know the same truths... and how often those three have historically been at odds with one another. Those three very often embrace the perception of knowing versus proving differently. As a result their conclusions of truth frequently differ as well.

It is not correct that science and Christianity (I have narrowed your reference to religion) have historically been at odds or are at odds today.

Science as an organized, sustained enterprise arose only once in human history. And where did it rise? In Europe, in the civilization then called Christendom. Why did modern science develop here and nowhere else? ... it was due to Christianity's emphasis on the importance of reason. ... modern science is an invention of medieval Christianity, and .. the greatest breakthroughs in scientific reason have largely been the work of Christians.

There is a convincing discussion with evidence presented in chapter 8 - 10 of What's So Great About Christianity by Dinesh D'Souza.

Songbird wrote:

jnk wrote:But I'm not sure photons exist until you try to figure out where they are. Slippery little suckers.

Just a guess here, Jeff... Do you also think that if a tree falls in the forest but there's no one there to hear it, the falling tree makes no sound?

Marsha

I will publish my research when I've come to a conclusion about whether you have to have an ear before you can call it a "sound." Maybe it depends on whether the forest is half full or half empty, and whether you can even find that forest for the trees. I do know that when I fall over I usually make a sound, but we won't get into that.

Rooster, let's not be at odds about whether anyone has ever been at odds. I won't burn you at the stake if you won't burn me. Deal? You do start some awesome threads, man!

jnk wrote: .......
I will publish my research when I've come to a conclusion about whether you have to have an ear before you can call it a "sound." Maybe it depends on whether the forest is half full or half empty, and whether you can even find that forest for the trees. I do know that when I fall over I usually make a sound, but we won't get into that.

.........

How far can a dog run into the woods?

rooster wrote:

-SWS wrote:Interesting that you should mention that, Jeff.

I was just thinking how science, religion, and philosophy are attempts to know the same truths... and how often those three have historically been at odds with one another. Those three very often embrace the perception of knowing versus proving differently. As a result their conclusions of truth frequently differ as well.

It is not correct that science and Christianity (I have narrowed your reference to religion) have historically been at odds or are at odds today.

Science as an organized, sustained enterprise arose only once in human history. And where did it rise? In Europe, in the civilization then called Christendom. Why did modern science develop here and nowhere else? ... it was due to Christianity's emphasis on the importance of reason. ... modern science is an invention of medieval Christianity, and .. the greatest breakthroughs in scientific reason have largely been the work of Christians.

There is a convincing discussion with evidence presented in chapter 8 - 10 of What's So Great About Christianity by Dinesh D'Souza.

Rooster, my statement doesn't preclude the fact that subscribers of most religions have and continue to contribute to science. Rather, my statement simply alluded to the fact that philosophy, science, and religion have historically diverged on a regular basis. As an example, the notion of anything other than a geocentric model of the universe was once considered religious heresy among Christians. The accepted "Truth" is but a forever-changing perception of a singular body of truth in all cases and of all historical eras. Today science, religion, and philosophy all tend to reject that same geocentric model that was formerly prescribed by the religious interpretations of yesteryear. Today issues such as the age of the universe or the role of evolution enjoy heated scholarly debates very similar to yesterday's geocentric debates.

Rooster, I never meant to say that science, religion, and philosophy disagree about everything---only that they have often been at odds with one another about very central truth-related issues throughout history---despite there being a singular body of truth.

rooster wrote:It is not correct that science and Christianity (I have narrowed your reference to religion) have historically been at odds or are at odds today.

Science as an organized, sustained enterprise arose only once in human history. And where did it rise? In Europe, in the civilization then called Christendom. Why did modern science develop here and nowhere else? ... it was due to Christianity's emphasis on the importance of reason. ... modern science is an invention of medieval Christianity, and .. the greatest breakthroughs in scientific reason have largely been the work of Christians.

That's an interesting point, Rooster, and the book sounds interesting, too. I find it particularly fascinating that the publisher of Skeptic magazine wrote one of the best Amazon reviews there. I'd never heard of D'Souza, but I'm adding him to my reading list. Years ago, someone suggested that I keep three things in mind when trying to sort out questions of faith and reason, and it's helped me a lot: Truth is true, humans were given brains for a reason, and the Bible never claimed to be a science book.

Marsha