J.C. Moore Online
Current events from a science perspective.

Posts Tagged ‘peer-review’

Gaming the Peer Review System, Part III: A Hostile Takeover

Mon ,26/03/2012

A group of Skeptics once managed to take over an editorship at a peer-reviewed journal  and publish articles hostile to mainstream climate science. With the help of politicians and large funding sources, the hostilities have continued to this day.

Skeptics: Science values its skeptics as they make science strong and they sometimes make valuable contributions by opening new fields for investigation. True skeptics follow the methodologies and the ethics of science, which requires they subject their work to review by their peers and divulge conflicts of interest. There are some skeptics, particularly in the areas of climate science, who violate the ethical principles of science for money and power. To separate those from true skeptics, they will be designated here as “Skeptics”. They are usually just ignored by scientists, but there are problems when a Skeptic becomes a journal editor. 

Journal editors are almost completely responsible for seeing that articles are properly reviewed and scientifically sound before they are published. Some journals, such as  Energy and Environment, cater to Skeptics such as Sallie Baliunas, Patrick Michaels, Ross McKitrick, Stephen McIntyre, Roger Pielke Jr., Willie Soon, and Steve McIntyre; who publish articles there that would not be accepted by legitimate journals. The editor, Sonja Boehmer-Christiansen, once said “the journal I edit has tried to keep this debate [climate scepticism] alive”.  Articles published in Energy and Environment are not taken seriously, but Skeptics hostile to climate science once managed a takeover of a reputable journal. An analysis by John Mashey showed the Skeptics managed to publish fourteen articles in Climate Research before they were caught gaming the peer review system.

Takeover: The takeover began in 1997, when Chris de Freitas became an editor at the reputable journal, Climate Research. There were 10 editors for the Journal and each worked independently, so it was possible for one editor to shepherd papers through the peer review process and see that they were published. The first paper  from a Skeptic, edited by de Freitas was by Patrick Michaels. The paper seemed to agree with the scientific findings of the IPCC reports, but it cast doubt at the end by concluding “this finding, instead adds further support to the emerging hypothesis that the Earth’s climate is not necessarily changing in a deleterious fashion”. Over the next six years, Chris de Freitas edited and published a series of fourteen papers by Skeptics who were interested in developing Dr. Michael’s “emerging hypothesis”. The articles caused so many complaints from scientists that some of the other editors questioned Dr. de Freitas about the quality of the papers he edited. He replied that they were on a “witch hunt”.

Restoring Order: The hostile takeover was uncovered after the fallout over a paper written by Sally Baliunas and Willie Soon. The paper reviewed the literature on the climate science of the last 1000 years, and concluded that the global warming in the 20th century was not unusual and that natural forces, rather than man’s activities were the cause. An important piece of their evidence was the Medieval Warm Period, which they claimed was warmer worldwide than the latter 20th century. But there was obviously something wrong with the paper. There were no accurate temperature records in Medieval Times, the Americas had not yet been discovered, and much of the Southern hemisphere was unknown. Proxy records from multiple sources show that the Medieval Warm Period amounted to only a small hump in the Earth’s temperature record. Shortly after its publication, the American Geophysical Union (AGU) issued a press release from thirteen of the scientists whose work was used in Baliunas and Soon’s paper, saying Soon and Baliunas seriously misinterpreted their research. The thirteen scientists then coauthored a paper explaining exactly why the Baliunas and Soon paper was in error.

 All this caused quite a furor at Climate Research. Five members the editorial board eventually resigned in protest and the newly hired chief editor, Hans von Storch stated the paper had serious errors and should never have been published. Tom Wigley, who often reviewed papers for Climate Research, wrote, “I have had papers that I refereed (and soundly rejected), under De Freitas’s editorship, appear later in the journal—without me seeing any response from the authors.” All this was followed by an unusual public statement from the publisher, acknowledging flaws in the journal’s editorial process. Under pressure, Chris de Freitas resigned shortly thereafter, and papers from the Skeptics stopped appearing in Climate Research.

Extended Hostilities: That should have ended the matter, except that some politicians found the conclusions of Baliunas and Soon’s paper to be advantageous to the fossil fuel industry to whom they owed allegiance. Political pressure was put on regulatory agencies to accept the results of the paper, in spite of its obvious flaws and distortions. The EPA was unwilling to include the paper in its assessment of climate science, so Sen. James Inhofe (R – OK) scheduled a meeting of the Senate Environment and Public Works (EPW) Committee to examine the paper.

At the EPW hearing, Michael Mann represented the scientific viewpoint, presenting evidence from multiple sources showing that the Medieval Warm period was not uniformly worldwide and resulted only in a small hump in the Earth’s temperature record. Dr. Soon stood behind his work and, in response to a direct question about his funding sources, testified that he had not received any funds that might have biased his objectivity. However, the paper lists the American Petroleum Institute as a major source of funding. Documents received later from the Smithsonian Institution in response to FOIA requests, revealed that since 2001  Dr. Soon has received over $1 million in funding from oil and coal interests.

Sen. Inhofe was upset by the turn of events and tried to get him fired – Michael Mann that is. At Sen. Inhofe’s insistence, the University of Pennsylvania, a Quaker University, conducted two investigations into Dr. Mann’s research and found no misconduct. A 2010 Science article reviewed the investigations, declaring “Michael Mann is cleared, again. “ Dissatisfied with the ruling, Sen. Inhofe has tried to get the attorney general to charge Michael Mann with fraud. It doesn’t get much more hostile than that. Sadly, for the first time in history, scientists are collecting a legal defense fund to defend scientists against political attacks. And even worse, the scientific opinion of the senior member of our Environmental and Public Works Committee is based on a paper that would not have passed freshman English.

 (c) 2012 J.C. Moore

Gaming the Peer Review System: Part 2. Exploiting Loopholes

Fri ,03/02/2012

There is evidence that the authors of a recent paper may have gamed the peer review system to publish a biased climate science paper.

The Review Process: When a paper is submitted to a journal for publication, the editor removes the name of the author and sends the manuscript to several experts in the area, usually three, for review. The editor keeps the names of the reviewers confidential. If an error is found, the reviewer’s comments are returned to the author with suggestions for corrections. It is a good system for ensuring the quality of research publications, but even then papers are sometimes published that contains errors. The reviewers may miss an error, a biased editor may publish the paper in spite of flaws, or authors may exploit loopholes in a journal’s rules to get a paper published. Some journals allow the author to suggest names of reviewers and the editor often picks reviewers from the list. Most scientists submit names of reliable reviewers as it is an embarrassment to have errors found in their paper after publication. However, even if the papers are properly reviewed, the practice can bring accusations of “pal” review. Since reviewer’s names are kept confidential by the editor, it is difficult to know for sure whether that may have happened. However, there is evidence that the authors of a recent paper may have gamed the system by suggesting a set of reviewers that shared their bias. See what you think.

The paper: Last July 25th, Roy Spencer and Danny Braswell authored a paper in the rather specialized technical journal, Remote Sensing, titled “On the Misdiagnosis Of Surface Temperature Feedbacks From Variations In Earth’s Radiant Energy Balance“.  The paper claimed “The sensitivity of the climate system to an imposed radiative imbalance remains the largest source of uncertainty in projections of future anthropogenic climate change. Here we present further evidence that this uncertainty from an observational perspective is largely due to the masking of the radiative feedback signal by internal radiative forcing, probably due to natural cloud variations.”  It seems that only an expert in climatology would know what that means or what its implications were, but in three days a sensationalized version of the paper appeared on internet sites, in major business magazines, and in news articles in major newspapers. Millions of people likely read about the paper.

The Publicity: The renewed public interest in science should have made climate scientists pleased; however, they were not. Beneath the technical language is a claim that the climate sensitivity to CO2 has been misinterpreted by climate scientists because of natural cloud variations. Were it true, it would mean that natural forces, not man, were responsible for much of the observed global warming. That idea had been examined before and found to be inconsistent with the evidence, but the idea is one that some climate skeptics have been promoting. And, they are part of a well-funded pipeline that carries misinformation about climate science to major news outlets before all the facts can be known.

Forbes: One main branch of the misinformation pipeline runs through the Heartland Institute, where James Taylor is listed as a senior fellow. James Taylor once wrote articles for the tobacco industry suggesting that secondhand smoke was not harmful, and he has now turned his talents to denying the ties between rising CO2 levels and global warming. Inexplicably, James Taylor has been hired by Forbes magazine to write on energy and environmental topics. James Taylor picked up on Spencer’s paper and wrote an article for Forbes titled, New NASA Data Blows Gaping Hole In Global Warming Alarmism. Not only was the title inaccurate and misleading, but the article was clearly an opinion article, miscategorized as news.  The editors of Forbes might not have known that Spencer’s “NASA Data” was the same data that climate scientists use to reach a very different conclusion, but perhaps they should have noticed that no reasonable news story would describe climate scientists as “alarmists” 15 times. The business community considers legislation that would reduce our carbon emissions to be anti-business, and business newspapers such as Investors Business Daily, the Wall Street Journal, and Forbes often are biased toward the skeptic’s position. The bias shows up in story selection, opinions miscategorized as news, a disproportionate number of skeptics articles on opinion pages, and  in sensationalized headlines. From Forbes, the article was picked up as a news story by other business magazines, Yahoo! News, MSNBC, and skeptic’s blog sites, which had a field day with the article. It is sad that millions will have read the distorted article, but few will ever read the climate scientist’s rebuttal. The article will soon sink into obscurity,  but it will have accomplished it’s purpose, which was to spread doubt about climate change.

Reproducibility: Publication in a peer-reviewed journal is not the only requirement for a paper to become accepted as part of the science literature. The research must stand up to the scrutiny of other experts in the field and it must be reproducible by other scientists with comparable knowledge and skill. Spencer’s paper reached the news media before climate scientists had a chance to respond, but they soon found a number of obvious errors in the paper. Trenberth and Fasullo summed it up:”The model has no realistic ocean, no El Niño, and no hydrological cycle, and it was tuned to give the result it gave. The bottom line is that there is NO merit whatsoever in this paper.”  Given time, A.E. Dessler analyzed Spencer’s paper in detail and published a rebuttal. The abstract in Geophysical Review Letters reports the key points of his paper:

  • Clouds are not causing climate change;
  • Observations are not in disagreement with models on this point;
  • Previous work on this is flawed;  ( referring specifically to Spencer’s paper in Remote Sensing).

Clearly, Spencer’s paper had serious methodological flaws and was not reproducible. How did the paper get through Remote Sensing’s peer review process? The answer would likely not have been found, except for the publicity.

The Catastrophe: The editor of Remote Sensing, who had been trying to build the reputation of the Journal, considered the publicity a catastrophe. The instructions in Remote Sensing asks authors to suggest five reviewers, and it is possible that Spencer could choose five skeptics.  The editor would not have to pick from those, but apparently in this case he did.  In the next issue of Remote Sensing, the editor, Dr. Wolfgang Wagner, resigned and issued a public apology for this article saying, “With this step I would also like to personally protest against how the authors and like-minded climate skeptics have much exaggerated the paper’s conclusions in public statements.” “The problem is that comparable studies published by other authors have already been refuted in open discussions and to some extent also in the literature, a fact which was ignored by Spencer and Braswell in their paper and, unfortunately, not picked up by the reviewers. “ And he concluded, “But, as the case presents itself now, the editorial team unintentionally selected three reviewers who probably share some climate sceptic notions of the authors.”

© 2012 J.C. Moore

 

Gaming the Peer Review System : Part 1. Biased Editors

Sun ,22/01/2012

If I have seen further it is only by standing on the shoulders of giants. … Sir Isaac Newton

 Peer review:  Every scientist’s work depends upon the evidence and reasoning of all the scientists who preceeded him. To ensure that previous work is reliable, scientific journals have established a peer review process to ensure that published papers are free of errors in reasoning and methodology. Normally,when a paper is submitted for publication, the editor of the journal removes the name of the author and sends the manuscript to a number of experts in the area for review. The editor keeps the names of the reviewers confidential. In case an error is found or corrections suggested, the reviews are returned to the author with suggestions for improvement. When the reviewer’s concerns are addressed to the editor’s satisfaction, the article is published. It is a good system for ensuring the quality of research publications, but in a few cases ways have been found to game the system.           

Biased Editors: The editors of journals published by major science organizations are chosen for their expertise in the area and for their fairness. However, any organization may publish a journal and claim their articles are peer-reviewed.  For example, a recent op Ed article in the Tulsa World claimed “Climate predictions must be science-backed“. That’s certainly true, but the author claimed his opinion was backed by a “a peer-reviewed article based on NOAA  data which proves that CO2 may not be the cause of global warming.”  However, no peer reviewed article reaching that conclusion could be found. When I contacted the author for his source, he referred me to an article by Ferenc Miskolczi published in Energy and Environment. Though Miskolczi’s article is based on NOAA’s data, it finds that adding CO2 to the atmosphere does not change its spectroscopic properties – a conclusion violating the laws of physics. Miskolczi’s article was criticized by van Dorland and Forster, who wrote: “Miskolczi (2010) theorizes that atmospheric CO2 increases cannot be a cause of global warming. We show his theory to be incorrect both in its application of radiation theory and from direct atmospheric observations.” How did such a paper get published?

The editor of a journal is almost completely responsible for seeing that articles are properly reviewed and for deciding if they should be published. Sourcewatch says that Energy and Environment is a peer-reviewed social science journal published by Multi-Science and the editor is Sonja Boehmer-Christiansen, who is described as a reader in geography. Many climate change skeptics such as Sallie Baliunas, Patrick Michaels, Ross McKitrick, Stephen McIntyre, Roger Pielke Jr., Willie Soon, and  Steve McIntyre publish articles there that would not be accepted in major journals. Sourcewatch says the editor admits in an article published online that “the journal I edit has tried to keep this debate [climate scepticism] alive”. She also states “I’m following my political agenda — a bit, anyway … But isn’t that the right of the editor?”

Not really, if you want to claim to be a peer reviewed science journal.

Bits and Pieces 1: Do Scientists Keep Secrets?

Tue ,20/07/2010

Complaints about  “scientific secrecy” are disingenuous: There is very little secrecy in science. Scientific papers are presented and openly debated at meetings where anyone can attend. The peer reviewed papers include the data, the results, and the reasoning and are available at public libraries and many are now online. Also:

Researchers are required to keep records of their research so that any other scientist with comparable training and skills could reproduce the research. The “reproducibility” of the research is an important factor in the reviewer’s evaluation of the research. The public has a right to information produced by publicly funded research and that may be requested through the Freedom of Information Act (FOIA). Usually a “Gatekeeper”, such as the project’s director, is designated to handle FOIA requests. That Gatekeeper has a responsibility to see not only that the public’s rights are upheld, but also to see that the FOIA process is not abused and that the scientists are protected. (1)

Only a few things are kept confidential to preserve the integrity of the peer review process.  The main barriers preventing a better understanding of science by the public is not “secrecy”, but poor science education, the lack of responsible and informative reporting by the media, and an ongoing campaign to spread misinformation by those who find the conclusions of science inconvenient to their ideological or financial interests.

Bits and Pieces

Fri ,16/07/2010

This article contains bits and pieces, usually short comments on recent science  articles and issues. Other bits and pieces will be added with the newest at the top.

The High Cost of Doing Nothing: A  report by the National Academy of Sciences details the high economic cost of inaction on environmental legislation (2). It’s relatively easy to figure the cost of regulations to protect the environment, but relatively hard to keep from inflating the cost for political purposes.  As a Republican, I am a little ashamed that Republicans have adopted the grossly inflated annual figure of $3200 per  household. That is useful for sticker shock and propaganda, but totally inaccurate. The CBO has estimated that it would cost around $300 and that there would be added savings that would reduce the deficit.

The cost of regulations  should  be compared to the cost of doing nothing. Estimates by the World’s top economists such as Britain’s Nicholas Stern or the US’s Paul Krugman are that right now it would cost about 2% of the worlds GDP to mitigate environmental damage – but if delayed, that amount could rise to 20% or more. That also doesn’t take into account intangibles such as clean air,  clean water, and a more sustainable economy.

Ocean Acidification is Serious: Since preindustrial times, the concentration of CO2 in the air has risen from 280 ppm to 385 ppm, a 38% increase.   As the amount of CO2 in the air increases, the amount that  dissolves in the ocean increases proportionately.  When the CO2 dissolves in seawater, it makes it more acidic, just as adding CO2 to soda makes it acidic. The pH of sea water has  been measured to be  more acidic by 0.1 pH unit than a century ago. Since the  pH scale  is logarithmic, the decrease of 0.1 unit means the oceans are now over 20% more acidic than a century ago and the cause is most certainly CO2.

To put that in perspective, human blood has a  carbonate buffer system similar to that of the oceans.  Normal blood pH is from 7.45 to 7.35 , and a blood pH less than 7.1 would require emergency treatment. Increasing the carbon dioxide in the blood by 38% will decreased the blood pH to about 7.25, not critical, but surely a sign that something is wrong. If the oceans get much more acidic, the coral, the fisheries, the shellfish, and the oxygen-producing plankton that give life to the oceans are threatened.

Complaints about the “scientific secrecy” are disingenuous: There is very little secrecy in science. Scientific papers are presented and openly debated at meetings where anyone can attend. The peer reviewed papers include the data, the results, and the reasoning and are available at public libraries and many are now online. Also:

Researchers are required to keep records of their research so that any other scientist with comparable training and skills could reproduce the research. The “reproducibility” of the research is an important factor in the reviewer’s evaluation of the research. The public has a right to information produced by publicly funded research and that may be requested through the Freedom of Information Act (FOIA). Usually a “Gatekeeper”, such as the project’s director, is designated to handle FOIA requests. That Gatekeeper has a responsibility to see not only that the public’s rights are upheld, but also to see that the FOIA process is not abused and that the scientists are protected. (1)

Only a few things are kept confidential to preserve the integrity of the peer review process.  The main barriers preventing a better understanding of science by the public is not “secrecy”, but poor science education, the lack of responsible and informative reporting by the media, and an ongoing campaign to spread misinformation by those who find the conclusions of science inconvenient to their ideological or financial interests.

Peer Review, Science Data, and the Public’s Right to Know

Tue ,13/07/2010

Does the public’s “right to know” extend to the peer review process and to the scientist’s data?

Peer review: Reputable scientific journals have a peer review process to ensure that published papers are free of errors in reasoning and methodology and that they report only the best research. Upon submission of a paper, the editor of the journal removes the name of the authors and sends it to expert researchers to be reviewed. The names of the authors are kept confidential by the editor to ensure that the author’s reputation, past personal differences, or factors other than the quality of the work cannot affect the review. The editor of the journal considers the reports of the reviewers and decides whether the paper should be published or returned to the author for corrections. Few papers receive outright rejection and the papers returned for correction are usually returned with reviewers comments.

The names of the reviewers are kept confidential by the editor to ensure that the author does not directly contact the reviewer to argue or does not retaliate against a reviewer. In a recent case, John Christy was able to discover through the stolen CRU e-mails who reviewed one of his papers and why the editor published it as he did. The paper was controversial in nature and contained opinions not held by most other climate scientists. The editor, in an attempt to present both sides of the issue, published Christy’s paper alongside a paper that presented the opposite view. Using information to which he should not have been entitled, Christy publicly attacked the reviewers, the editor, the peer review process, and climate science in general. His actions violated the integrity of the process and also the professional ethics required of scientists as he released his opinions to the public before the matter could be impartially investigated.

The Public’s Rights: The claims that the names of the reviewers and the editor’s reasons should be made public are invalid. Scientific journals are funded by subscriptions and dues of members and not publicly funded. The review process is set up as it is to ensure the integrity of published science papers and “peer reviewed” is the gold standard of quality in science information. The editor of the journal has the right to choose the reviewers and decide what is published just as the editor of a newspaper has the right to publish or reject articles without divulging the reasons.

Scientific Data: The public’s right to the data of researchers is another matter. Researchers are required to keep records of their research so that any other scientist with comparable training and skills could reproduce the research. The “reproducibility” of the research is an important factor in the reviewer’s evaluation of the research. The public has a right to information produced by publicly funded research and that may be requested through the Freedom of Information Act (FOIA). Usually a “Gatekeeper”, such as the project’s director, is designated to handle FOIA requests. That Gatekeeper has a responsibility to see not only that the public’s rights are upheld, but that the FOIA process is not abused and that the scientists are protected.

Scientists are understandably reluctant to release their data – as some who did release it later came to feel as if gremlins had seized their work and their lives. Some researchers have been harassed by numerous and frivolous  requests for information  meant only to impede their work. That is particularly true in climate science where there are apparently well-funded gremlins, some of them ex-scientists*, at work. Worse, scientists have been criticized publicly for reasonable practices that can be misconstrued. For example, good research requires the calibration of equipment, yet that has been led to accusations  of “adjusting the data”. And, a math ‘trick” used to simplify a computation, was mischaracterized as “tricking the public”.

Even worse, when Phil Jones, the CRU director, released his raw data for a 1990 research paper to a former London financial trader, Douglas J. Keenan, Keenan combed through the data and then tried to have the FBI arrest Jones’ co-author for fraud. An investigation cleared the researchers of any wrongdoing but it took a toll on their time and work. Incidents like that have  a chilling effect on the willingness of scientists to release their data. Some scientists who released their raw data, have seen it “recalculated” in such a way as to reach conclusions contrary to their findings, yet attributable to them. Reputable journals will not publish the erroneous conclusions of “recalculated” data , but some newspaper articles, blog sites, and even Congressional hearings will use them to promote a controversy manufactured by someone who actually did no research. And, once the fallacy is “out there”, it is hard to correct.

Certainly, the public has a right to openness in public funded research. Much of the scientific debate take place at scientific meetings and those wishing to hear the research debated may attend . The FOIA Gatekeeper has an important role to see that the FOIA requests are valid, that scientists are not harassed, and that those who wish to use the data for unscientific, or even malevolent purposes, do not have easy access to the data. The next step for those who do not like the Gatekeeper’s decision is to seek redress in the courts – not by illegally hacking the researcher’s computers.

* The author considers those scientist who abandon the methodology, ethics, and objectivity of science; especially for money, notoriety, or political purposes, to be “ex-scientists”.