Announcement

Collapse
No announcement yet.

Somewhat O/T - Scientific integrity and validity of research

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Somewhat O/T - Scientific integrity and validity of research

    Some of our discussions on this forum have dealt with the issue of scientific integrity and the validity of scoliosis research. Sharon has often (<cough> perhaps some would say too often) noted the Greek paper where the author made a compelling case that Most Published Research is False.

    Last week, someone released a large amount of internal emails, fortran code, and data from one of the top research institutions involved with climate science. Some have characterized this as “hacked” or “stolen” files. Others (myself included) would characterize the documents as “leaked” from a “whistleblower”. Time may clarify the issue of the source/provenance of the data but the institution involved admits that they appear to be authentic.

    This is a topic I’ve been interested in for some time and I recognize that the people involved (correspondence in the emails) are at the forefront of the research. They are not a small group on the periphery; they are the MOST cited individuals and the MOST influential in the field.

    Perhaps a lesson related to scoliosis research that we can take from this is to be constantly questioning (<cough> perhaps more so in our minds than on the forum) the validity of all research. The paper Sharon cites about most research being false (maybe someone can link to it) is well worth reading. (Although I should note that the paper doesn’t list malfeasance (as we may be seeing here) as one of the reasons research is often false)

    Now, regarding the “leaked” information, there are over 60 MB of information and folks are pouring over the information now. Some of the revelations may be misinterpreted or taken out of context (such as the “Trick” issue) but others seem a quite a bit more damning.

    Here is a link discussing/summarizing some of the initial revelations uncovered. The data and fortran code has been often requested by scientists wishing to investigate the claims arising from the research. They have been the subject of multiple Freedom of Information (FOI) requests. Indeed, the name of the zipped file containing the data is called FOI2009.zip. In any event, the topic makes for interesting reading and it will be interesting to see how the scientific community (and media) deal with it.

    Another way in which this topic may tie in with scoliosis is the potential mixing of policy and science. I know some have suggested that the BrAIST trial may be a ploy to come up with data eliminating bracing as an (insurance covered) option or eliminate school screening as a mandated thing. (A “conspiracy” theory I don’t subscribe to but in light of what is going on with Climate Science, well, you never know).

    Anyway, I thought it better to discuss it here than on Dingo’s genetic thread. It gets so hard to follow stuff here sometimes.

  • #2
    Hi CD ~

    I have been wondering what this is all about, thank you for the posting. I've read through the links and plan to go back and read in more detail after discussing this with my brother who understands such things far better than myself. I think I see, at first glance - that we have been lied to and that there are some shedding light on the subject? Lying with statistics for the sake of profit has been around for a long time. Where do we find the truth? I suspect as Ballet Mom states: Follow the Money. Relative to research everywhere, including scoliosis? Definitely food for thought. Pass the nitro?

    Comment


    • #3
      Originally posted by concerned dad View Post
      Last week, someone released a large amount of internal emails, fortran code, and data from one of the top research institutions involved with climate science. Some have characterized this as “hacked” or “stolen” files. Others (myself included) would characterize the documents as “leaked” from a “whistleblower”. Time may clarify the issue of the source/provenance of the data but the institution involved admits that they appear to be authentic.
      Another possibility:

      So this is my theory is and this is only my theory:

      A few people inside CRU possessed the archive of documents being held in reserve in case the FOI appeal decision was made in favor of Steve McIntyre. They shared it with others by putting it in an FTP directory which was on the same CPU as the external webserver, or even worse, was an on a shared drive somewhere to which the webserver had permissions to access. In other words, if you knew where to look, it was publicly available. Then, along comes our “hackers” who happened to find it, download it, and the rest is history unfolding before our eyes. So much for the cries of sophisticated hacking and victimization noted above.

      If I had to bet money, I would guess that David Palmer, Information Policy & Compliance Manager, University of East Anglia, has an even chance of being the guilty party, but it would only be a guess.

      To repeat the basic premise of this theory.

      There’s an old adage, never assume malice when stupidity or incopetence will explain it.
      You can read more here....

      http://wattsupwiththat.com/2009/11/2...on/#more-13003

      Comment


      • #4
        Originally posted by concerned dad View Post

        Now, regarding the “leaked” information, there are over 60 MB of information and folks are pouring over the information now. Some of the revelations may be misinterpreted or taken out of context (such as the “Trick” issue) but others seem a quite a bit more damning.
        I'm not so sure they're being misinterpreted:

        You can claim an email you wrote years ago isn’t accurate saying it was “taken out of context”, but a programmer making notes in the code does so that he/she can document what the code is actually doing at that stage, so that anyone who looks at it later can figure out why this function doesn’t plot past 1960. In this case, it is not allowing all of the temperature data to be plotted. Growing season data (summer months when the new tree rings are formed) past 1960 is thrown out because “these will be artificially adjusted to look closer to the real temperatures”, which implies some post processing routine.

        Spin that, spin it to the moon if you want. I’ll believe programmer notes over the word of somebody who stands to gain from suggesting there’s nothing “untowards” about it.

        Either the data tells the story of nature or it does not. Data that has been “artificially adjusted to look closer to the real temperatures” is false data, yielding a false result.
        http://wattsupwiththat.com/2009/11/2...ry/#more-13065


        Another:


        Back in December 2004 John Finn asked about “the divergence” in Myth vs. Fact Regarding the “Hockey Stick” -thread of RealClimate.org.

        Whatever the reason for the divergence, it would seem to suggest that the practice of grafting the thermometer record onto a proxy temperature record – as I believe was done in the case of the ‘hockey stick’ – is dubious to say the least.

        mike’s response speaks for itself.

        No researchers in this field have ever, to our knowledge, “grafted the thermometer record onto” any reconstrution. It is somewhat disappointing to find this specious claim (which we usually find originating from industry-funded climate disinformation websites) appearing in this forum.

        But there is an interesting twist here: grafting the thermometer onto a reconstruction is not actually the original “Mike’s Nature trick”! Mann did not fully graft the thermometer on a reconstruction, but he stopped the smoothed series in their end years. The trick is more sophisticated, and was uncovered by UC over here. (Note: Try not to click this link now, CA is overloaded. Can’t even get to it myself to mirror it. -A)

        When smoothing these time series, the Team had a problem: actual reconstructions “diverge” from the instrumental series in the last part of 20th century. For instance, in the original hockey stick (ending 1980) the last 30-40 years of data points slightly downwards. In order to smooth those time series one needs to “pad” the series beyond the end time, and no matter what method one uses, this leads to a smoothed graph pointing downwards in the end whereas the smoothed instrumental series is pointing upwards — a divergence. So Mann’s solution was to use the instrumental record for padding, which changes the smoothed series to point upwards as clearly seen in UC’s figure (violet original, green without “Mike’s Nature trick”.
        http://wattsupwiththat.com/2009/11/2...-nature-trick/

        Look at second chart at link to see the difference in the two series. I believe the "instrumental record" they're talking about is simply the thermometer based records versus a few magic trees' rings in Russia.

        Comment


        • #5
          Originally posted by concerned dad View Post

          Another way in which this topic may tie in with scoliosis is the potential mixing of policy and science. I know some have suggested that the BrAIST trial may be a ploy to come up with data eliminating bracing as an (insurance covered) option or eliminate school screening as a mandated thing. (A “conspiracy” theory I don’t subscribe to but in light of what is going on with Climate Science, well, you never know).
          In regards to the Braist study, I recently found out that most insurance companies' utilization management functions are staffed and performed by nurses. These are the people who decided that a recommended treatment is appropriate or not.

          I also notice that Lori Dolan, who is the study director for Braist, is part of the adjunct/secondary faculty of the School of Nursing at the University of Iowa.

          This suggests to me that the insurance companies are starting the process of removing bracing as a paid benefit due to "equipoise". They are simply trying to create a study to back them up to give them some protection in court when they finally decide to start defunding bracing.

          Comment


          • #6
            Originally posted by Ballet Mom View Post
            They are simply trying to create a study to back them up to give them some protection in court when they finally decide to start defunding bracing.
            "Creating" or conducting a study is one thing.

            Only the results will determine whether they continue funding bracing or not, yes? The data will be what they will be and will be collected by orthopedic surgeons participating in the study.

            Are you predicting the study results will not support bracing?

            And I assume you aren't suggesting that Dolan et al. will be manufacturing data to fit a preconceived notion are you? Dolan et al. follow the data unlike the great run of lay untrained parents.
            Sharon, mother of identical twin girls with scoliosis

            No island of sanity.

            Question: What do you call alternative medicine that works?
            Answer: Medicine


            "We are all African."

            Comment


            • #7
              Originally posted by Ballet Mom View Post
              In regards to the Braist study, I recently found out that most insurance companies' utilization management functions are staffed and performed by nurses. These are the people who decided that a recommended treatment is appropriate or not.
              But they aren't making these essentially research decisions by themselves!!!!!!!!!!! Nursing isn't a research degree!!!!!!!!!

              They are working from the standards GIVEN them and at present the standard for certain curves includes bracing.
              Sharon, mother of identical twin girls with scoliosis

              No island of sanity.

              Question: What do you call alternative medicine that works?
              Answer: Medicine


              "We are all African."

              Comment


              • #8
                Continued from Dingo's "Risk of Scoliosis Among 1st Degree Relatives" thread:



                Originally posted by Pooka1:

                No the money thing is a red herring and you never see this mentioned by folks who know how monies are awarded in the case of merit-based science.
                Of course they wouldn't mention it...but don't be naive. Of course money is a primary driver for much of what goes on in the world. Perhaps the researchers don't individually get more money....even that is debatable...but they certainly get more prestige from their peers, their universities, their funders....they even get to KEEP THEIR JOBS! If they aren't funded, they can be denied tenure and moved out. Not a very good place to be in a research university without funding. And let's face it, I see lots of scientists who would be quite happy if they could just make all the policy in the world according to their biases.

                Even the study you quote that questions research findings states:

                A research finding is less likely to be true ......when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance. Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias.

                http://www.ncbi.nlm.nih.gov/pubmed/16060722

                I actually think this author is applying this to all research not just medical but I don't have the actual paper to read, just the summary.

                Comment


                • #9
                  Originally posted by Pooka1 View Post
                  But they aren't making these essentially research decisions by themselves!!!!!!!!!!! Nursing isn't a research degree!!!!!!!!!

                  They are working from the standards GIVEN them and at present the standard for certain curves includes bracing.

                  Of course, that's why the research is being done at a University. You're wrong though, lots of the nursing degrees at universities are research oriented...I suspect that Iowa is one of them. And you'll notice, obviously, that Weinstein is the principal investigator of the study. It's pretty clear they're using medical and research-oriented nursing people to perform studies nowadays. Obviously, the people in the field will use the research results from a university. I suspect that the insurance companies have managed to find a willing participant at the Univ of Iowa to hopefully help them find a way to defund bracing. It is absolutely the way it would work here in the U.S. This is my opinion, of course. I've had too many years to learn to not be naive.

                  Comment


                  • #10
                    Originally posted by Pooka1 View Post
                    "Creating" or conducting a study is one thing.

                    Only the results will determine whether they continue funding bracing or not, yes? The data will be what they will be and will be collected by orthopedic surgeons participating in the study.

                    Are you predicting the study results will not support bracing?

                    And I assume you aren't suggesting that Dolan et al. will be manufacturing data to fit a preconceived notion are you? Dolan et al. follow the data unlike the great run of lay untrained parents.
                    Actually, the study could only apply to the braces used in the study, couldn't it? However, I suspect that it will be used as a proxy to not pay for any bracing or school screening, no matter what kind of brace is used.

                    I think even with their Braist study design, they will not have the numbers, nor the attributes identified to accurately decide who can benefit from bracing or not. And therefore, they will be doing a disservice to the many kids that do benefit from bracing, because quite obviously lots of kids do benefit from bracing, at least it's obvious to those without closed minds.

                    It would certainly be nice if they managed to figure out who does benefit from bracing and continue to brace those kids. Maybe that would mean barring overweight kids from bracing, or perhaps tall kids, or maybe kids without flexible spines. Would that be okay with you?

                    How do you know "Dolan et al. follow the data unlike the great run of lay untrained parents?" We do know that research people are pure as the driven snow.... By the way, you do get awfully testy whenever I bring up the Braist study....any particular reason?
                    Last edited by Ballet Mom; 11-23-2009, 01:25 PM.

                    Comment


                    • #11
                      Originally posted by Ballet Mom View Post
                      [...] at least it's obvious to those without closed minds.
                      Look I know you aren't doing this on purpose but this is one of MANY examples in your writing where you are using the EXACT "arguments" that creationists use. Now they are FLAT OUT counterfactual but your arguments are not. So I fail to see why you continually resort to this ignorant paranoia when you are not advocating clearly counterfactual material (for the most part).

                      I think there are better arguments for you to use. At least I assume so. The stuff you have been bringing up is just paranoid.

                      How do you know "Dolan et al. follow the data unlike the great run of lay untrained parents?" We do know that research people are pure as the driven snow.... By the way, you do get awfully testy whenever I bring up the Braist study....any particular reason?
                      I get testy when people who don't know how scientists operate claim they are being nefarious. It certainly isn't limited to the Braist study and in fact animates most of my comments to certain lay, untrained parents in this sandbox. There is nothing special about the Braist study in this regard. Why do you think it is different than all the other stuff that gets ignorantly pilloried?

                      The heart of science is not mathematical models. It is intellectual honesty. The great run of scientists are honest. Anyone who implies that is not the case for most scientists is simply being ignorant and paranoid.
                      Last edited by Pooka1; 11-23-2009, 08:24 PM.
                      Sharon, mother of identical twin girls with scoliosis

                      No island of sanity.

                      Question: What do you call alternative medicine that works?
                      Answer: Medicine


                      "We are all African."

                      Comment


                      • #12
                        Continued from Dingo's "Risk of Scoliosis Among 1st Degree Relatives" thread:

                        Originally posted by Pooka1:

                        See this topic almost can't be discussed because of the huge divide between scientist jargon and the general public.
                        That's what the priests used to say also before those pesky Protestants published the bible so everyone could read it.


                        No the money thing is never advanced by people who understand this topic as far as I know.
                        Don't be naive.

                        Consider the following:

                        1. That is the money he and his colleagues got in grants over ~19 YEARS I assume. That would not be unusual but it's pretty good. (ETA: The salaries of these researchers are fixed. It doesn't go up as the grant money goes up. I assume you knew that, yes?)
                        I suspect their salaries have indeed gone up and so have their perks of the job, including their rank and stature at their place of work.

                        2. Federal NSF monies are awarded on the basis of merit via peer review, not political expedience. Now the call for proposals guides the research but those never fund specific conclusions, only specific fields of study. The research falls out whichever way it falls out.
                        I think we're finding out the truth or untruth of that right now. Especially interesting is the way these scientists managed to quash any other opinions from peer review journals to silence the opposition.

                        3. You can bet you bottom dollar several of those millions that researcher was awarded were awarded during the 8-year Bush administration, the SAME Bush administration which was publicly accused of suppressing results consistent with AGW. How does that square with your claim that politics determines Federal research funding?
                        Well, seeing as this was a British research unit, hopefully the US wasn't involved with funding it at all. However, since the EU is quite set on global government, they certainly have been funding it. Gee, I wonder why, if it's true, that the Bush Administration wouldn't have wanted to pay for funding this political position "research". Perhaps, they had good reason to believe it wasn't science, but politics dressed up as pseudo-science.
                        Last edited by Ballet Mom; 11-23-2009, 02:18 PM.

                        Comment


                        • #13
                          Originally posted by Pooka1 View Post
                          Look I know you aren't doing this on purpose but this is one of MANY examples in your writing where you are using the EXACT "arguments" that creationists use. Now they are FLAT OUT counterfactual but your arguments are not. So I fail to see why you continually resort to this ignorant paranoia when you are not advocating clearly counterfactual material (for the most part).

                          I think there are better arguments for you to use. At least I assume so. The stuff you have been bringing up is just paranoid.
                          Get out of here. My arguments are not creationist arguments, you really need to get rid of that ugly mindset. I can see with my own eyes that bracing can work. I guess that's the difference between engineers and "scientists"...engineers can work with what we observe...not ponder the statistical studies that may or may not accurately reflect what actually occurred.

                          Comment


                          • #14
                            I dont know about global warming..... but it tends to get pretty hot here

                            Ballet Mom, I dont agree with you about the nusrses deciding what is covered. My insurance (Cigna) updates a position paper every year (written by a physician, I believe) discussing scoliosis; what's covered, whats not, why, the state of the science, etc. It is pretty interesting reading

                            regarding the "braist conspiracy"; the only good case I saw for that was made by the poster on the other forum where she noted that they included curves of only 25 degrees and up. Braist changed their protocol (in response to this?)

                            Comment


                            • #15
                              Originally posted by concerned dad View Post
                              Ballet Mom, I don't agree with you about the nurses deciding what is covered.
                              I can't believe she meant that as written. If she did then we can all go home.
                              Sharon, mother of identical twin girls with scoliosis

                              No island of sanity.

                              Question: What do you call alternative medicine that works?
                              Answer: Medicine


                              "We are all African."

                              Comment

                              Working...
                              X