In September 2011, a team of physicists announced they had measured neutrinos moving faster than light. By the following March, that astonishing claim was disproved. Was the first announcement “bad science?”(More)

Publish or Perish, Part II: Faster Than Light, Or Not

This week Morning Feature looks at proposals to improve academic journals. Yesterday we asked why many journals reject studies that attempt to replicate prior work, and studies that yield negative results. Today we see why online publishing of tentative findings can improve a study, but may also yield embarrassing headlines. Tomorrow we’ll look at a ‘promise-to-publish’ model that reviews proposals before the results are known, and whether that will improve academic publishing.

OPERA-worthy

On September 11, 2011, a team of almost 200 physicists working on the OPERA project made a truly stunning announcement:

The OPERA neutrino experiment at the underground Gran Sasso Laboratory has measured the velocity of neutrinos from the CERN CNGS beam over a baseline of about 730 km. The measurement is based on data taken by OPERA in the years 2009, 2010 and 2011. Dedicated upgrades of the CNGS timing system and of the OPERA detector, as well as a high precision geodesy campaign for the measurement of the neutrino baseline, allowed reaching comparable systematic and statistical accuracies. An arrival time of CNGS muon neutrinos with respect to the one computed assuming the speed of light in vacuum of (6.5 +/- 7.4(stat.)((+8.3)(-8.0)sys.))ns was measured corresponding to a relative difference of the muon neutrino velocity with respect to the speed of light (v-c)/c =(2.7 +/-3.1(stat.)((+3.4)(-3.3)(sys.))x10^(-6). The above result, obtained by comparing the time distributions of neutrino interactions and of protons hitting the CNGS target in 10.5 microseconds long extractions, was confirmed by a test performed at the end of 2011 using a short bunch beam allowing to measure the neutrino time of flight at the single interaction level.

In plain English, the OPERA team fired a beam of neutrinos at a target 730km away, and their data showed the neutrinos reached the target 0.00000006 seconds faster than a beam of light. That was truly astonishing because Albert Einstein’s theories of relativity predict that nothing can travel through space faster than light.

That “through space” bit matters. Space itself can expand faster than light, and one explanation of relativity is that everything else moves at exactly the speed of light through spacetime, so the faster you move through space, the slower you move through time.

Einstein’s theories of relativity have been tested hundreds of times, and they’ve always fit the evidence. And technologies like GPS rely in part on calculations from relativity theory. Thus, the OPERA results – finding neutrinos moving faster than light – would have forced scientists to rethink one of the most fundamental principles of modern physics.

“Our results are in agreement with what Einstein would like to have”

On March 15, 2012, a team of 65 physicists working on the ICARUS project reported on their attempt to replicate the OPERA results:

The CERN-SPS accelerator has been briefly operated in a new, lower intensity neutrino mode with ~10^12 p.o.t. /pulse and with a beam structure made of four LHC-like extractions, each with a narrow width of 3 ns, separated by 524 ns. This very tightly bunched beam structure represents a substantial progress with respect to the ordinary operation of the CNGS beam, since it allows a very accurate time-of-flight measurement of neutrinos from CERN to LNGS on an event-to-event basis. The ICARUS T600 detector has collected 7 beam-associated events, consistent with the CNGS delivered neutrino flux of 2.2 10^16 p.o.t. and in agreement with the well known characteristics of neutrino events in the LAr-TPC. The time of flight difference between the speed of light and the arriving neutrino LAr-TPC events has been analysed. The result is compatible with the simultaneous arrival of all events with equal speed, the one of light. This is in a striking difference with the reported result of OPERA that claimed that high energy neutrinos from CERN should arrive at LNGS about 60 ns earlier than expected from luminal speed.

Again in plain English:

Now another experiment located just a few metres from OPERA has clocked neutrinos travelling at roughly the speed of light, and no faster. Known as ICARUS, the rival monitored a beam of neutrinos sent from CERN in late October and early November of last year. The neutrinos were packed into pulses just 3 nanoseconds long. That meant that the timing could be measured far more accurately than the original OPERA measurement, which used 10-microsecond pulses.

“Our results are in agreement with what Einstein would like to have,” says Carlo Rubbia, the spokesperson for ICARUS and a Nobel prizewinning physicist at CERN. Neutrinos measured by the experiment arrived within just 4 nanoseconds of the time that light travelling through a vacuum would take to cover the distance, well within the experimental margin of error.

“Maybe we should have been more cautious”

Both the OPERA and ICARUS findings were first released at arXiv.org, an online ‘pre-journal’ for physicists. I call it a ‘pre-journal’ because physicists routinely post new findings at arXiv before submitting them to peer-reviewed journals, or post them online while peer review is underway. And that can leave egg on their faces:

“I find it embarrassing,” says Luca Stanco of the National Institute of Nuclear Physics in Padova, Italy, an OPERA member who initially refused to sign a paper about the result. “Maybe we should have been more cautious and done more checks.”

Dr. Stanco was quoted in a Nature article that revealed timing glitches in the OPERA experiment, and most physicists now agree those glitches account for the seeming faster-than-light measurement.

“Claiming the team has lost credibility is ridiculous”

A similar pre-journal mistake happened this March when the BICEP2 team reported having found gravity waves from the Big Bang. Within a month, other teams had compared the BICEP2 data to data from the Planck satellite and found the gravity waves were probably emissions from space dust. And last month over 200 physicists looked at Planck data and found the space dust emissions fully explained what at first looked like gravity waves.

This prompted one reader to comment:

Seriously though: the BICEP people have lost all credibility with me. YMMV, but it’s clear that the whole thing was a huge gamble on their part. The thing is, though, that when you gamble and lose, you should expect to pay up….

But you have to click to see that comment. It’s been hidden because so many other readers disagreed, as one explained:

Claiming the team has lost credibility is ridiculous. This is what science is all about – a messy process of finding what’s really out there. Mistakes happen all the time and science has given us tools to detect these mistakes and to learn from them. What happens next will be very interesting.

A scientist from the Planck team also defended his colleagues at BICEP2:

As a member of the Planck Science Team, I would urge caution concerning the interpretation. What we are saying is that polarised dust emission in the BICEP2 field is high. But it may be that there is something left in the BICEP2 signal that can be attributed to gravitational waves. We need to cross-correlated the Planck maps with the BICEP2 maps. This analysis is underway.

“There’s no shame here”

And science writer Philip Ball argues this has always been the essence of the scientific method:

Some will regard this as a huge embarrassment, not only for the Bicep2 team but for science itself. Already some researchers have criticised the team for making a premature announcement to the press before their work had been properly peer reviewed.

But there’s no shame here. On the contrary, this episode is good for science. This sequence of excitement followed by deflation, debate and controversy is perfectly normal – it’s just that in the past it would have happened out of the public gaze. Only when the dust had settled would a sober and sanitised version of events have been reported, if indeed there was anything left to report.

Yet while that is “good for science” as a matter of method, it may undermine the reputation of science among the general public. But other polls show distrust of science is especially strong among Tea Party Republicans.

That may help explain the challenge for ‘pre-journals.’ Science requires – and most scientists have – a higher-than-average tolerance for ambiguity. The scientific method requires that even widely-accepted and reliable theories be seen as subject to refinement or reversal by future evidence. But most people do not have a higher-than-average tolerance for ambiguity, and research shows that political conservatives tend to have a lower-than-average tolerance for ambiguity. Simply, some people prefer enduring dogma over ongoing discovery:

When a social science, made up entirely of observations and hypotheses, tells us first that men are polygamous and women homebodies, and then that men are monogamous and women gallivanters – and, what’s more builds far-fetched protocols of dating and courtship and marriage and divorce around these notions – maybe it’s time to retire the whole approach.

All the while, the first books of the Bible are still hanging around.

Online pre-journals give such critics more ammunition to use against science. We can’t abandon pre-journals – they help scientists do science better – but we need to recognize their risks.

+++++

Happy Friday!