Research ethics: 3 ways to blow the whistle

Are more people doing wrong or are more people speaking up? Retractions of scientific papers have increased about tenfold during the past decade, with many studies crumbling in cases of high-profile research misconduct that ranges from plagiarism to image manipulation…

Are more people doing wrong or are more people speaking up? Retractions of scientific papers have increased about tenfold during the past decade, with many studies crumbling in cases of high-profile research misconduct that ranges from plagiarism to image manipulation to outright data fabrication. When worries about somebody’s work reach a critical point, it falls to a peer, supervisor, junior partner or uninvolved bystander to decide whether to keep mum or step up and blow the whistle. Doing the latter comes at significant risk, and the path is rarely simple. Some make their case and move on; others never give up. And in what seems to be a growing trend, anonymous watchdogs are airing their concerns through e-mail and public forums. Here, Nature profiles three markedly different stories of individuals who acted on their suspicions. Successful or otherwise, each case offers lessons for would-be tipsters.

The analytical Uri Simonsohn sees himself as more of a data-whisperer than a whistle-blower. His day job as a social scientist at the University of Pennsylvania in Philadelphia involves scouring archival data — from house prices to auction records to college admissions — as part of his research into judgement and decision-making. He suspects that this background has predisposed him to catching spurious patterns in other psychologists’ results. “With an experiment, you do a t-test and move on,” he says. “But people who work with archival data are used to looking at data very carefully.” It was this intuition that stirred when he first came across papers by Dirk Smeesters at Erasmus University Rotterdam in the Netherlands and Lawrence Sanna at the University of Michigan in Ann Arbor in the summer of 2011. In both cases, the data seemed too good to be true, containing an overabundance of large effects and statistically significant results. In one of Sanna’s papers, Simonsohn noticed that one experiment — in which volunteers were supposedly split into different groups — produced results with uncannily similar standard deviations. In the results of Smeesters’ studies, he saw a suspiciously low frequency of round numbers and an unusual similarity between many of the averages. “If there’s too little noise, and the data are too reliable again and again, they cannot be real,” he says. “Real data are supposed to have error.”

Simonsohn checked his suspicions by simulating experiments thousands of times to show how unlikely the reported results actually were. He replicated his analyses on other papers by the same authors and found the same patterns, and he carried out negative controls, showing no suspicious patterns in the work of other psychologists who used the same set-ups. Simonsohn contacted both authors and spent months systematically ruling out alternative explanations for the discrepancies he found. Eventually, according to Simonsohn, only one remained — that they had manipulated their data. He still refrained from accusing anyone, liaising privately with Smeesters, Sanna and their co-authors, asking for raw data, outlining his concerns and asking if another party, such as a student or research assistant, could have tampered with the data. “I was extremely open-minded,” he says. “My working hypothesis was that it’s not in your interest to fake if you’re a researcher.” Towards the end of 2011, Simonsohn learned that Erasmus University, which he had contacted, had begun an investigation. He also found out that because of his inquiries, the University of North Carolina at Chapel Hill, where Sanna had performed his work, had also started to investigate. By the summer of 2012, both Smeesters and Sanna had resigned from their posts, and several of their papers have since been retracted. In previous statements, Smeesters has said that he never fabricated data and that the practices he used are common in his field; he chose not to provide a further comment when contacted by Nature. Neither Sanna nor his former institution have publicly addressed questions about his resignation and Sanna could not be reached for comment.

“If the data are too reliable again and again, they cannot be real.”

When asked about the two careers that have been broken by his investigations, Simonsohn pauses. “I don’t feel bad about it,” he concludes. “If I’m going to the same conferences as these people, and publishing in these journals, I can’t just look the other way.” Joe Simmons, a psychologist at the University of Pennsylvania, says that he admires his colleague’s integrity and sense of obligation. “He couldn’t not do something,” he says. Simonsohn hopes that his actions will spur psychologists to instigate reforms to stem fraud — one option would be to require researchers to post raw data, thereby making them more open to checks by watchful data-sleuths. He also wants researchers to disclose more details of their work at the outset of an experiment, such as the variables to analyse or their planned sample sizes. That would discourage subtler forms of data-tampering — such as continuing experiments only until results meet significance — which, in his opinion, flood the psychological literature with false positives (see Nature 485, 298–300; 2012). Simonsohn’s whistle-blowing attracted its share of attention. He has received around a dozen offers to look into suspected cases of dodgy data, typically from people outside science who have personal concerns about, say, the US election. He rarely replies. He has little interest in being drawn into unnecessary disputes and bristles at any suggestion that he has led a witch-hunt — a term that he associates with the wanton use of poor diagnostic tests, not his own careful review. “Some people think he does it for the fame, but he finds the fame annoying,” says Simmons. Simonsohn, for his part, says he hopes that his new-found identity as a whistle-blower will morph into a different label, as “a person who looks carefully at data. I would be very happy with that reputation,” he says.

The quixotic Helene Hill thought she was nearing retirement in 1999 when, one day, she decided to take a peek at a lab mate’s culture dishes. A radiation biologist at the University of Medicine and Dentistry of New Jersey in Newark, Hill was collaborating with a junior colleague on a project to study the ‘bystander effect’, a phenomenon whereby cells exposed to an agent — in this case radiation — influence the behaviour of unexposed neighbours. Hill had trained the postdoc, Anupam Bishayee, on the technique and wanted to see how he had fared. The plates, she says, were empty, yet Bishayee later reported cell counts from them. Hill would spend the next 14 years trying to expose what she believes to be a case of scientific misconduct. University panels, the US Office of Research Integrity (ORI), and two courts of law have evaluated and dismissed her concerns. Her journey has cost her thousands of dollars in legal fees and countless hours trawling through more than 30,000 documents. And it could cost her her job. Yet Hill, now 84, has no intention of backing off. “A person has an obligation to do the right thing if they can,” she says. After the first observation, Hill and another postdoc decided to covertly shadow Bishayee’s experiments, snapping photos of his cultures in the incubator. When Bishayee reported data from an experiment that they thought was contaminated with mould, Hill and her colleague accused him of fabricating the results and took their concerns to the university’s committee on research integrity. But their case soon frayed. Under questioning, her colleague acknowledged that he had moved Bishayee’s culture tubes before taking photos of them, which the committee viewed as potentially tampering with the evidence. And Hill explained that she had used a microscope that she was unfamiliar with when checking Bishayee’s cultures. The committee determined that Hill did not have enough evidence to prove her case.

“A person has an obligation to do the right thing if they can.”

Hill would not let the matter lie. Bishayee had published his results in a paper that lists Hill as a co-author (A. Bishayee et al. Radiat. Res. 155, 335–344; 2001) and his adviser, Roger Howell, used Bishayee’s data to support a grant application to the National Institutes of Health (NIH) in 1999. Hill took the case to federal investigators at the ORI, who conducted a small statistical analysis of Bishayee’s data. Hill says that in her opinion the patterns therein suggested fabrication, and one ORI investigator, Kay Fields, thought the case had merit. But Fields was overruled by a superior, in part because he believed that the control data for the analysis — Hill’s own — were also statistically questionable. The ORI determined that there was insufficient evidence to prove misconduct. Hill continued to petition her university and the ORI to review the data. Fields, meanwhile, says that she felt obliged to tell Hill about another option: a ‘qui tam’ lawsuit. Such lawsuits, allowed under the US False Claims Act, can be brought by any citizen to aid the government in recouping taxpayers’ funds allocated under false pretences. Hill’s case could be eligible because of the NIH grant. Qui tam can be a risky strategy, says David Lewis, director of the research misconduct project at the nonprofit National Whistleblower Center in Washington DC. He has filed two qui tam lawsuits in the past, unrelated to Hill’s (see Nature 453, 262–263; 2008). Both were unsuccessful, and Lewis generally doesn’t recommend the strategy. In Hill’s case, the process dragged on for years and cost her US$200,000 in legal fees. “I don’t think my children are too happy with my having lost that much money,” she says, “but I just felt I had an obligation to see it through.” New Jersey District Court judge Dennis Cavanaugh ruled in favour of Bishayee and Howell in October 2010, and referred to Hill’s battle as “a quest of Quixotic proportions that ultimately must be put to rest”. Hill lost her final appeal in October 2011. Still, she says that her investment paid off: the discovery phase of the lawsuit allowed her access to ten years’ worth of the Howell lab’s notebooks. With those data in hand, she teamed up with statistician Joel Pitt of Georgian Court University in Lakewood Township, New Jersey. Together, they pored over data that Bishayee had hand-recorded from a machine that counts cells. The duo also gathered larger control data sets from others who had used the same machine. Pitt looked at the frequency of the numbers appearing as the least significant digit of each recorded count. These should have a random distribution, but Bishayee’s data seemed to favour certain numbers. Pitt calculated the odds of those frequencies arising by chance as less than 1 in 100 billion. In Hill’s view, the implication is clear: Bishayee made the numbers up. Along with Pitt, Hill has been trying, so far unsuccessfully, to publish these statistical analyses and further publicize her allegations, actions that Robert Johnson, the dean of her institution — now part of Rutgers University — warned in a strongly worded letter in July could lead to “additional disciplinary action, up to and including termination”. Howell, in a written statement to Nature, expressed frustration at the time spent revisiting the issue despite no finding of wrongdoing. Bishayee did not respond to Nature’s request for comment. Fields says: “I admire Dr Hill for the courage of her convictions, but it is difficult to say that she was prudent to pursue the case for so long and at such expense.” Hill, for her part, remains undeterred. “I want to finish,” she says. “It becomes almost an obsession.”

The anonymous Anonymous tipsters are nothing new. But since 2010, someone going by the pseudonym ‘Clare Francis’ has seriously upped the ante. She or he (or they; many suspect it is a group of people) has sent hundreds of e-mails to life-science journal editors, flagging up suspected cases of plagiarism or instances in which figures appear to be manipulated or duplicated. Her terse, sometimes cryptic complaints have resulted in a handful of retractions and corrections, but editors have felt bombarded by her voluminous notices — many of which, they say, lead nowhere. Like her or not, Francis has sparked a debate about how editors deal with anonymous tips, which are now poised to grow thanks to the proliferation of websites that allow anyone to publicly air grievances about research papers. Sabine Kleinert, a senior executive editor at The Lancet and former vice-chair of the UK-based Committee on Publication Ethics (COPE), calls the recent surge in anonymous comments “the Clare Francis phenomenon”. Phenomenon is an apt descriptor. Francis estimates that she has e-mailed “about 100” different editors. And those publishers who agreed to talk to Nature said that their editors generally receive multiple messages from her. Diane Sullenberger, executive editor of the Proceedings of the National Academy of Sciences, says that as many as 80% of the allegations they receive come from Francis. And the scientific publisher Wiley says that in 2011 Francis’s name was on more than half of its investigation requests. Anonymity generally makes people uncomfortable, says Ulrich Brandt, editor-in-chief of Biochimica et Biophysica Acta. “One has to wonder about the motivation of the whistle-blower,” he says. “Ill-founded allegations of scientific misconduct can do harm and may constitute a form of scientific misconduct themselves.”

“I do not have a ‘tone’. I try to describe what I can see.”

By 2011, editors were growing increasingly frustrated by Francis because — quite apart from her anonymity — many of her claims did not check out. “I have no problem taking time to look at an allegation — but I don’t like people wasting my time,” says Eric Murphy, editor-in-chief of Lipids. Moreover, many of Francis’s complaints are oblique and hard to follow, says Sullenberger. “It is helpful to know specific details about the concerns from a scientific standpoint, not just, ‘The bands in the 10- and 60-minute lanes are geometrical and superimposable’ or ‘Background is silvery smooth’,” she says, referring to some of Francis’s e-mails. Some journal editors have warned Francis that they are less likely to follow up on her requests than other complaints. In September 2011, Wiley’s then legal director, Roy Kaufman, sent her an e-mail saying that the company could “not guarantee that all anonymous allegations sent to us will be investigated”. Francis made the note public, sparking debate over how such allegations should be handled. Two years on, the attitudes of editors have changed to some degree. In February this year, COPE put out guidelines on “responding to anonymous whistle blowers”. Francis was not mentioned by name, but was the main driving force behind the work, says Virginia Barbour, COPE’s current chair. “Editors were feeling guilty, and upset, and didn’t understand how they should approach it.” COPE reminded them that, no matter where they came from, “all allegations … that have specific, detailed evidence to support the claim should be investigated”. But Anna Trudgett, editorial director at the journal Blood, says that the journal still addresses Francis’s e-mails only selectively. “Not all anonymous correspondence is treated the same way,” she says. Wiley has adjusted its practice to investigate all complaints, says spokesperson Helen Bray. Fundamentally, editors are not just reacting to Clare Francis’s pseudonymity. They are also irritated by the way she works. “For some, it’s not that Clare Francis is a pseudonym; it’s that the pseudonym is Clare Francis,” says Tom Reller, a spokesperson for Elsevier. Some editors bring up what they say is Francis’s aggressive tone and pursuit of lost causes. “When we determine that the allegation is not founded, it is not uncommon for Clare Francis not to accept the result,” says Véronique Kiermer, Nature Publishing Group’s executive editor. In Barbour’s view, Francis’s tactics are not a good model for other anonymous tipsters to emulate. To make up for the inevitable loss of trust that comes from being anonymous, tip-offs gain credence if they are precise, detailed and polite. Francis sometimes meets these standards but often does not. To Francis, such critiques miss the point. Asked about her tone, she wrote back: “I do not have a ‘tone’. I try to describe what I can see.” She adds that editors often focus narrowly on their journal when she sends what she says are connected patterns of image manipulation across many journals. “They will not look at the whole picture, but remain in purdah,” she writes. As for alleged false leads, she says: “The hit rate would be higher if they paid attention to what is on the page rather than their fantasy world.” One thing that editors and Francis might agree on is that anonymous whistle-blowing is likely to increase, given the increased access to papers by people all around the world and the availability of online tools for spotting potential plagiarism and image manipulation. One site, called PubPeer, is already becoming a venue for anonymous comments — including postings in a similar vein to Francis’s style. The growth here is a sign that whistle-blowers are not being protected enough within the academic environment, says Kleinert. “This is where we have to do much more. Somebody should feel comfortable to be able to raise issues without fearing retaliation or damage to their own career”.