© Matt Strassler [February 6, 2015]
Unfortunately, though to no one’s surprise after seeing the data from the Planck satellite in the last few months, the BICEP2 experiment’s claim of a discovery of gravitational waves from cosmic inflation has blown away in the interstellar wind. [For my previous posts on BICEP2, including a great deal of background information, click here.] The BICEP2 scientists and the Planck satellite scientists have worked together to come to this conclusion, and written a joint paper on the subject. Their conclusion is that the potentially exciting effect that BICEP2 observed (“B-mode polarization of the cosmic microwave background on large scales”; these terms are explained here) was due, completely or in large part, to polarized dust in our galaxy (the Milky Way). The story of how they came to this conclusion is interesting, and my goal here is to explain it to non-experts.
What Happened, and Why?
How did it happen that the BICEP2 scientists, after having done a spectacular experimental measurement, came to the wrong conclusion about what it meant? The issue of polarized dust within our galaxy [more accurately, “spinning dust which gives rise to B-mode-polarized microwaves”] was something that everyone, including BICEP2, knew had to be accounted for, because its presence could mimic the same effect in their experiment as gravitational waves from cosmic inflation. Unfortunately the amount of polarized dust in the region of the sky where BICEP2 was looking was quite a bit larger than BICEP2’s original analysis suggested. What was their mistake?
It’s quite simple really. BICEP2 didn’t, and couldn’t, actually know how much dust was in the region of the sky that they were studying, or how it was distributed. It wasn’t something they could measure directly. They did know, from other people’s measurements, that they were looking in a patch of sky where the amount of dust (polarized or not) was very small. They had some estimates from past studies of how much polarization was typical. They also got ahold of some unpublished data from the Planck satellite, which had been shown in public. They put this information together in various ways, and somehow managed to convince themselves that any effect from dust on their measurement had to be very small indeed. This made them confident that their discovery of B-mode polarization meant they’d seen gravitational waves from cosmic inflation. And that’s why they made a big deal when they made their announcement last spring.
But their confidence was not justified. All of their techniques for estimating the dust were problematic, and gave them overly small estimates.
Over the last eight months, results from the Planck satellite have been gradually released, indicating that in regions of the sky with very little dust, the degree of polarization of that dust is quite a bit larger than BICEP2 (and many previous experts) assumed. And now, finally, after combining their efforts, the BICEP2 and Planck teams can conclude that although the amount of dust in the BICEP2 portion of the sky is really, really small — which is why it was hard for the Planck team to make believable measurements in that region until now — it’s big enough, and polarized enough, to have generated the BICEP2 signal.
So now the questions: Why did BICEP2 and Planck have to combine their efforts to settle the issue? And how did they do it?
Why Planck and BICEP2 Joined Forces
The story has everything to do with
- the strengths and weaknesses of BICEP2 (and its successor Keck, built by the same team) and of Planck, and
- the details of the polarized microwaves (which is what these experiments measure) that are generated by dust and gravitational waves.
[Microwaves are electromagnetic waves like visible light, X-rays, and radio waves; they have a longer wavelength, and lower frequency, than visible light, but have shorter wavelength and higher frequency than radio waves. Electromagnetic waves, as they move through space, are waving perpendicular to their motion; if such a wave is heading toward you, it could be wiggling left-to-right, or up-and-down, or somewhere in between. If the wiggling of waves from a certain patch of sky has a random orientation, we call the waves “unpolarized”, while if the wiggling tends to be in a specific direction, we call the waves “polarized”.]
For the first point, it is a question of breadth versus depth. Planck is highly versatile, and can do many different types of measurements. BICEP2 and Keck are much more restricted, but for their specialized measurement, they are extremely powerful. BICEP2 and Keck only can detect microwaves in a narrow frequency band, while Planck can detect a wide range of frequencies. But BICEP2 and Keck are much more sensitive detectors than Planck, so they can detect much smaller signals. (It’s also worth knowing, though it isn’t critical in today’s story, that Planck can look across the whole sky, while BICEP2 and Keck just focus on a small patch of sky which is known to have a small amount of dust.)
BICEP2 and Keck are looking at microwaves that are vibrating with a relatively low frequency (150 GHz, i.e. 150 billion cycles per second) while Planck looks at many frequencies between 30 and 353 GHz. For today’s purposes, the only one that matters is 353 GHz, a relatively high frequency. For the rest of this post, I’ll refer to 150 GHz and 353 GHz as the “low” and “high” frequencies.
And here’s the scientific bottom line. Dust radiates microwaves much more readily at the high frequency than at the low frequency. Conversely, gravitational waves would give microwaves just as readily at the low frequency as at the high frequency. BICEP2 measured a tiny but significant amount of B-mode polarization of microwaves at the low frequency. Therefore, if BICEP2’s signal is due to gravitational waves, we expect only a tiny amount of B-mode-polarized microwaves at the high frequency, while if the signal is due to dust, we expect a much larger amount (though still quite small, and hard for Planck to measure!)
An Analogy: Is the Sky Clear or Overcast?
Here’s an analogy to help make this clear. Suppose you were to look at the sky through rose-colored glasses — glasses that filter out blue light and let red light through — and you measure how much light you are seeing. Fine, you can tell how bright the sky is in red light. But could you tell if the sky is clear (and blue) or overcast (and white?) It might be quite hard, because either way the sky looks red when seen through your glasses.
However, imagine your friend has blue-colored glasses — glasses that filter out red light and let blue light through. Your friend also can’t tell if the sky is clear or overcast, because the sky will look blue just because his glasses are blue. He can only tell how much blue light is coming in.
But now compare your measurements. If the sky is white, you and your friend will see a similar amount of light coming through your glasses, because white light contains a moderate amount of both blue light and red light. But if the sky is blue, your friend will see much, much more light through his glasses than you see coming through your red glasses. Thus, by combining your experiences (Figure 1), you and your friend can determine whether the sky is clear or overcast.
Back to Planck and BICEP2
What Planck and BICEP2/Keck are doing is similar, though with a few crucial differences (they don’t measure the total amount of microwave radiation, but rather the tiny degree to which it is polarized in B-modes), and with an additional, crucial twist (to be described a little later). And for a similar reason, the Planck and BICEP2 measurements need each other; neither can draw a firm conclusion alone. If the BICEP2 measurement at the low frequency had been due to gravitational waves, the amount of B-mode polarization at the high frequency would have been too small for Planck (which isn’t as sharp-eyed as BICEP2) to measure. But in fact Planck did observe B-mode polarization at the high frequency, with a power hundreds of times higher than the power found at the low frequency by BICEP2.
This is shown in Figure 2, taken from the BICEP2/Keck/Planck paper and annotated by me. The BICEP2/Keck data, taken at the low frequency, is on the left. Note the circled points, which describe effects on large patches of the sky. They clearly lie above the red curve (which is the prediction of how much B-mode polarization comes from gravitational lensing of E-mode polarization from other sources — an effect I described here.) This means that there is a new source of B-mode polarization, perhaps from gravitational radiation, perhaps from dust.
The data on the right is from Planck, measured at the high frequency. The circled points lie above the horizontal line at zero — a rough signal of B-mode polarization at the high frequency. Note that this signal is big (compare the vertical axes on the left and right plot to see how much bigger the Planck signal is than the BICEP2 signal). The effect BICEP2 observed at the low frequency is observed by Planck, and is apparently much brighter, at the high frequency!
Thus, as surely as Earth’s sky is blue, we learn from this that the sky above BICEP2 shows the effect of polarized dust: the effect is dim (but measurable by BICEP2) at the low frequency, and very much brighter (just enough to be seen by Planck) at the high frequency.
Or do we? The problem with this conclusion is that it is too quick. As you can see from the right-hand plot in Figure 1, the Planck data by itself actually isn’t very convincing. Notice each data point [each dot] has a big vertical “uncertainty bar”, and the data points for Planck are kind of all over the place, unlike those from BICEP2, which are nice and clean and quite far from zero (the horizontal line.) So is Planck seeing a real effect from dust? Is it possible that actually there’s no effect there at all, and this data is just due to random garbage in the electronics of the Planck detector being misinterpreted as a detection of B-mode polarization?
Clinching the Case
So here’s the final, convincing bit of evidence. Suppose, first, that BICEP2, Keck and Planck are really, truly seeing effects due to dust. In that case, in the patches of sky where BICEP2 sees a signal of dust, Planck should see a signal also! The pattern of the effect of dust on the sky should have the same shape for BICEP2 as it does for Planck.
On the other hand, suppose that Planck is misinterpreting random statistical fluctuations as due to dust, and BICEP2 and Keck are really seeing effects from gravitational waves. In that case, there is no reason that Planck should see effects in the same location on the sky where BICEP2 sees them; BICEP2 is seeing a real signal, and Planck is seeing something random. Therefore the spatial patterns they see should look different.
And so, the two experiments can check: does Planck see the same spatial patterns in the signal as BICEP2 and Keck, or not?
Yes, they do. That’s the clincher, shown in Figure 3, also taken from their paper; the circled points lie above the horizontal line, which means that BICEP2/Keck data and Planck data are correlated. You see that this combination of data is much more persuasive than if we simply look at Planck’s data alone; the uncertainty bars are much smaller and the data points are closer together. And so it is in combining the two experiments in this second manner that the case for dust is actually closed.
You could push my earlier analogy further if you want, by imagining that although your glasses are nice and clear, your friend’s blue glasses are so dirty and opaque that he can barely see any light coming through at all. He says to you “I think I see some light!” but you’re not sure you believe him. However, suppose the sky is a little hazy with high clouds, so the brightness of the sky is a little less in some places than others. Well, if you see bright and dark patches in your red glasses, and he sees bright and dark patches in his blue glasses, and the patches that he sees and that you see are in the same locations, then he must really be seeing the sky, not just imagining things. And (since his glasses are so opaque) that in turn implies that the sky must be much, much brighter in blue light than in red, meaning that the sky is clear, not overcast.
For interested readers: a couple of details I have left out. Keck sees less of an effect than BICEP2, so the combined result from the BICEP2/Keck team is smaller than it was in BICEP2’s original announcement. This may just be due to a statistical effect, or perhaps Keck, a newer experiment, is slightly better than BICEP2. Also, although I didn’t emphasize it, the size of the effect seen in Planck, the size of the effect seen in BICEP2/Keck, and the size of the correlation between Planck and BICEP2/Keck, as illustrated in Figures 2 and 3, are all quite consistent with what one would predict from a dust signal. One other conclusion of the paper is that B-mode polarization seen in small patches of sky — the points not circled in Figures 2 and 3 — is definitely from gravitational lensing of E-mode polarization.
Now it is time for the scientists to make this all quantitative; vague statements need to be made more precise. The signal these experiments observe is, in principle, due to a mixture of dust and gravitational waves. Let’s call the amount of gravitational waves “r” (a measure of how strong the effect from cosmic inflation would be) and the amount of dust “A_d” (basically short for “amount of dust” or “amplitude of dust effect”). Now we ask, what values of r and A_d are the Planck and BICEP2 (and Keck) data consistent with?
The answer is shown in Figure 4, which is taken from the paper and annotated by me. Data are always somewhat uncertain, so we can’t use the BICEP2/Keck/Planck data to get an absolutely definite knowledge of r and A_d. [Indeed no knowledge is ever absolutely definite; there is always a (possibly extremely small) amount of uncertainty.] Instead, the inner and outer ellipses (focus your attention on the black ones) show the most likely and somewhat likely values of r and A_d. Anything outside the outer ellipse is quite unlikely indeed. If the BICEP2 (and Keck) data were from gravitational waves, you would expect the ellipses to surround the point at the end of the violet arrow, with r between 0.1 and 0.2 and with A_d = 0. But if they were from dust, you would expect the ellipses to surround the point at the end of the red arrow, with r quite small and A_d significantly different from zero. (Incidentally, if all three experiments had detected just statistical noise, you would expect the ellipses to surround the point with r = 0 and A_d = 0.)
So what we learn from this figure is
- BICEP2 and Keck have detected a real signal, through a spectacular measurement.
- The signal definitely has a significant contribution from dust.
- The signal may have no contribution from gravitational waves and cannot be purely gravitational waves.
Tada! But too bad for BICEP2, and for our knowledge of the universe’s early history.
One caution. Does this result mean BICEP2 definitely did not detect gravitational waves? Strictly speaking we can’t say that — and in science, it is essential to say what you mean, and be clear about what you do and do not know. We can only say that they have no clear evidence for gravitational waves.
However, when you have a choice between interpreting an experiment as having made a mundane discovery and interpreting it as having made a stunning, Nobel-Prize-worthy discovery, consider that history favors the mundane. We know that cosmic inflation need not lead to gravitational waves that BICEP2 can detect; the gravitational wave signal can easily be ten or more times smaller. It’s therefore perfectly possible that the BICEP2 signal is entirely due to dust even if cosmic inflation did occur in the universe’s past. My personal best guess is that this is all dust. Of course, the universe doesn’t care about my best guess, and in any case, measurements looking for even smaller effects will continue into the coming decades. Each scientist will decide for him or herself whether to be hopeful or not about the future; this is a matter of opinion and hunch, not of knowledge.
It is all quite disappointing for scientists involved, not just for those on BICEP2, for whom it is a bitter end to a great measurement, but for the whole community. If BICEP2’s original interpretation had been correct, it would have had a great slew of fascinating implications and opened up a whole new field of scientific inquiry. Without it, we’re back doing what we were already doing before: still looking for evidence either for or against cosmic inflation, and more generally for clues as to why this universe of ours has so many very odd features.
Epilogue and Homily
Let me add, for those readers who might be inclined to jump to wrong conclusions, or who know others that are so inclined, that it is essential not to throw out the baby with the bathwater. The reason scientific inquiry eventually leads to correct answers, despite the fact that scientific experiments are commonly wrong, is because of the insistence on subjecting experiments to extreme scrutiny and demanding they be reproduced before taking them too seriously. It is this rigorous attention to detail that distinguishes scientific knowledge from other forms of human knowledge.
Whenever you read a claim of a new scientific discovery in the newspaper, always ask: has it been confirmed, and how? A Higgs boson of some type has been discovered; two experiments at the Large Hadron Collider, each making several separate measurements, have given strong evidence for this. The universe’s expansion is accelerating; this is now confirmed by a bevy of experiments. By contrast, no other experiment ever confirmed the OPERA experiment’s claim of faster-than-light neutrinos, and indeed an error, and a detailed study confirming the error, explained why their measurement came out as it did. Similarly, no other experiment confirmed BICEP2; in this case there is no (as yet known) error in the measurement itself, but there was certainly an error and a lack of caution in interpreting the experiment.
And so it goes in science. You should not trust scientific results — such as claims of connections between vaccines and autism, and claims about deflated footballs in cold weather — that have not been subject to heavy scrutiny and have not been confirmed by independent investigations. Conversely, do trust results that have been scrutinized and confirmed… including claims that vaccines work and save lives, that the Earth’s average temperature is gradually increasing, and that radioactivity leaked from Fukushima in 2011 is not a danger to the U.S. west coast.
Scientists, like all humans, are fallible. But the scientific process, armed with its powerful methods for identifying and reversing human mistakes, is self-correcting. The scientific method works — our technologically-based economy is clear evidence of this — and it deserves more respect than it currently receives from politicians and populist pundits in my home country.