A scientific controversy has been brewing concerning the results of BICEP2, the experiment that measured polarized microwaves coming from a patch of the sky, and whose measurement has been widely interpreted as a discovery of gravitational waves, probably from cosmic inflation. (Here’s my post about the discovery, here’s some background so you can understand it more easily. Here are some of my articles about the early universe.) On the day of the announcement, some elements of the media hailed it as a great discovery without reminding readers of something very important: it’s provisional!
From the very beginning of the BICEP2 story, I’ve been reminding you (here and here) that it is very common for claims of great scientific discoveries to disappear after further scrutiny, and that a declaration of victory by the scientific community comes much more slowly and deliberately than it often does in the press. Every scientist knows that while science, as a collective process viewed over time, very rarely makes mistakes, individual experiments and experimenters are often wrong. (To its credit, the New York Times article contained some cautionary statements in its prose, and also quoted scientists making cautionary statements. Other media outlets forgot.)
Doing forefront science is extremely difficult, because it requires near-perfection. A single unfortunate mistake in a very complex experiment can create an effect that appears similar to what the experimenters were looking for, but is a fake. Scientists are all well-aware of this; we’ve all seen examples, some of which took years to diagnose. And so, as with any claim of a big discovery, you should view the BICEP2 result as provisional, until checked thoroughly by outside experts, and until confirmed by other experiments.
What could go wrong with BICEP2? On purely logical grounds, the BICEP2 result, interpreted as evidence for cosmic inflation, could be problematic if any one of the following four things is true:
1) The experiment itself has a technical problem, and the polarized microwaves they observe actually don’t exist.
2) The polarized microwaves are real, but they aren’t coming from ancient gravitational waves; they are instead coming from dust (very small grains of material) that is distributed around the galaxy between the stars, and that can radiate polarized microwaves.
3) The polarization really is coming from the cosmic microwave background (the leftover glow from the Big Bang), but it is not coming from gravitational waves; instead it comes from some other unknown source.
4) The polarization is really coming from gravitational waves, but these waves are not due to cosmic inflation but to some other source in the early universe.
The current controversy concerns point 2.
Dust in the Sky
One of the challenges for the BICEP2 measurement (and any others like it) is that dust grains in the galaxy, spinning in the presence of magnetic fields, absorbing light, and re-radiating it as somewhat-polarized microwaves, can give a signal that would mimic or obscure a signal of polarized microwaves that are generated by ancient gravitational waves. Consequently, one of the things that the BICEP2 people had to show, to convince first themselves and then others that they were really seeing effects of gravitational waves, was to obtain and present evidence that the effect of polarized dust, in the part of the sky in which they were looking, is too small to affect their measurement. It is this evidence that is now coming into question.
If you had a perfect experiment and all the money and time you could want, it would be straightforward to figure out whether you were seeing polarized microwaves from dust or polarized microwaves caused by gravitational waves. Here’s how: the amount of polarized light from dust grows rapidly as you look at higher and higher frequency microwaves, peaking at about 400 GHz (GHz = billions [G] of cycles per second [Hz]) while the amount from gravitational waves falls slowly at higher frequencies. So if you could measure the polarization at various frequencies, then by looking at whether your polarization signal does or does not increase rapidly with frequency between 100 and 400 GHz, you could distinguish dust-generated polarization easily from gravitational-wave-generated polarization.
But BICEP2’s high-precision measurements of microwaves have only been made at one frequency (150 GHz — measurements at 100 GHz will also be available in future). So to assure their polarization signal wasn’t from dust, the BICEP2 experimenters had to obtain information from another source. In their paper, they gave an argument that polarized dust was far too small — at least four and perhaps ten times too small — to affect their measurement. They did this in two types of ways, indicated in the figure below, which is taken from the BICEP2 paper and annotated by me.
One argument relied on various people’s models of how dust is distributed around the galaxy. These models all show that the amount of dust-generated polarization should be extremely small: most of the lower curves in the figure are from these models, and they lie very close to zero compared to the BICEP2 data. But this argument is only slightly reassuring, because these models are really highly-educated guesses. A claim of such an important discovery shouldn’t rest on that kind of foundation.
So the really strong argument BICEP2 presented was based on reinterpreting some data from the Planck experiment. This data had only partly been published; some of it had only been made public in graphical form, in plots that were presented during talks at conferences. It mainly involves polarization from dust, measured by Planck across the sky, mainly at 353 GHz (where dust has the biggest effect). The BICEP people said to themselves: suppose that the polarization Planck sees comes entirely from dust; well, we know how the polarization effect depends on frequency, so we can infer how much polarization from dust we would find at the BICEP2 frequency. This argument showed that the effect of dust in the region of the sky they were looking at was far too small for them to worry about, even if they viewed it very conservatively; there’s no way it could explain the signal that they see. More precisely, of the two estimates I’ve marked with an arrow, even the worst case scenario, DDM2 (the dashed blue curve), was far below the BICEP2 data. So there was no way, in the view of the BICEP2 team, that their signal could be coming from dust.
But some experts believe that there’s something problematic with the way BICEP2 did this, and that when done right, it is possible that the dust polarization will be a far bigger effect than BICEP2 estimated. [Raphael Flauger is one person with serious concerns: here is a talk that he gave this past week, but I'm afraid that only experts have a hope of understanding it, and I myself have a number of crucial unanswered questions about his presentation.] I emphasize that I cannot confirm their belief is correct, as I cannot yet construct a completely coherent story from public information — so I’m not going to speculate about it. I advise you to emulate the universe, and be patient; the truth will come out eventually, almost certainly within a year.
What Planck Didn’t Say
Another part of this story is that a couple of weeks ago Planck put out a paper (not published yet, but submitted for publication in a journal) showing their measurements of polarized dust across the sky… or rather, across much of the sky. They’ve done measurements at various frequencies and can, as I described already, figure out how much polarization is coming from dust. You would think this measurement would have settled the issue; either BICEP’s argument agrees with or doesn’t agree with what Planck now says. But the whole point here is that BICEP2 makes more precise polarization measurements than Planck does. And in their recent paper, Planck only presents measurements in parts of the sky where they have a strong enough signal that they are confident that they know what they’re seeing, at high precision. Unfortunately, the region where BICEP2 is looking — a region where the amount of dust is very low, which is exactly why BICEP2 wants to do measurements there — is a region where Planck can’t see a strong enough signal to be sure how much polarized dust is present.
So it may well be that currently no one — not Planck, not BICEP2, not anyone — really knows for sure how much polarized dust there is in the BICEP2 region of the sky. [Or if the Planck folks already basically know, one can imagine, given how much rides on the answer, that they're not telling until they're absolutely sure.] Whether the real effect of dust is as small as BICEP2 claimed, or whether it is as big as their signal — whether their huge discovery of gravitational waves stands, or whether they’ve merely discovered that dust in the galaxy is far more polarized than expected — is simply not something that we can know for sure yet.
But all of this confusion probably does add some uncertainty — both the colloquial uncertainty (did BICEP2 really make a big discovery?) and the technical uncertainty (how statistically significant is BICEP2’s observation?). It’s up to BICEP2 to go back through their arguments and determine how much change, if any, they have to make… and whether, after any appropriate adjustments, their discovery has become notably more tentative than it was before.
What you are getting a glimpse of, if you are following this story, is the scientific process in action. In physics — I can’t speak for other sciences, and I know there are some where it is not true — the assumption by the experts is that every claim of a scientific result, especially a major discovery, is wrong until proven right. Every result, especially one of particular significance, is poked and prodded, scrutinized and questioned, and subject to a battery of stress tests. Of course the scientists doing the measurement do this first, as best they can, knowing that it’s better to discover mistakes in private than in public. Then their colleagues do the same, checking the details of the measurement, repeating it (more or less), and trying to do even better measurements of the same effect. Anywhere along the way during this process, an experiment can fail to pass muster. But those that do pass muster — such as the recent discovery of a Higgs particle, the discovery that the universe’s expansion is accelerating, and the indirect discovery of gravitational waves back in the 1980s — will be around for the ages.
BICEP2’s result, along with its widespread interpretation as evidence of inflation, may yet survive the gauntlet of tests to which it is being subjected. But it’s far too soon to tell.