While I was on my way to Johns Hopkins University Monday evening, I wrote the following post, for publication today (Wednesday). Tuesday morning, CERN stole some of my thunder by putting out a press release consistent with the conclusion I drew below. Not that I mind. A quote from the press release follows this post.
The rumor mills on various blogs have been going berserk, with claims that a Higgs particle of Standard Model type (the simplest possible version of the Higgs particle) has been found by ATLAS and CMS (the two large general purpose experiments at the Large Hadron Collider.) The claims are that the Higgs signal is at a mass-energy [E=mc2] of 125 GeV; that CMS, in its search for a Higgs particle decaying to two photons, sees a small excess (that’s two or so standard deviations, or 2 σ, away from zero signal); and that ATLAS sees a larger excess (perhaps 3 σ) in their similar analysis. (You may find it useful to read my recent article about a lightweight Standard Model Higgs particle, and why searching for it through its decays to two photons is the best way to find it but takes frustratingly long — or you might like my recent guest post on Cosmic Variance about the Higgs search.)
Well, rumors are sometimes true, and this one might be, more or less.
- More precisely, it might be true that ATLAS and CMS see excesses of the claimed type and size. We’ll find out on December 13th.
- And also, it might even be true that these excesses are signs of the Standard Model Higgs particle. We will not find that out on December 13th.
Why not? There’s just not enough data yet. We might see some evidence in favor of some type of Higgs particle being at some particular mass, and the evidence might be intriguing, even moderately compelling, but it certainly won’t be convincing. We need to remember a few things about statistics, before we get overly excited. It’s very easy to misinterpret claims about statistics, if you don’t ask exactly the right question.
The classic examples: the probability that you will win the lottery is very different from the probability that someone will win the lottery. If you and twenty other people are in a room together, the probability that you have the same birthday as someone else in the room is small, but the probability that some pair of people in the room have the same birthday is large. And the probability that, in a plot of an experiment’s data, any particular bin will have a 3 σ excess by pure accident is very small, but the probability that one of 25 bins will have a 3 σ excess is much larger.
To think about this within the Higgs context, let’s do a very rough estimate by looking at one bin at a time. Let’s say ATLAS and CMS bin their data on two photon events in 1 GeV-wide bins. Background from non-Higgs processes will contribute about 400 events per bin, for putative Higgs masses near 120 GeV. Statistical fluctuations in the background by 1 σ upward would mean 20 extra events; a 2 σ fluctuation would mean 40 extra events. Meanwhile, a Higgs signal might mean perhaps 40 events total, which, with a favorable fluctuation upward, might turn out to be 50 or 60. Putting the favorable signal on top of an upward fluctuation on the background could mean a 2 or even a 3 σ excess. That’s for one bin. But just because you have a 3 σ excess in one bin does not mean you have a 3 σ excess in your measurement… because there are many bins. The question is: if you see a 3 σ uptick in one of the bins, is that a signal of a Higgs, or perhaps something that happened just by chance? The probability that any one bin will fluctuate upward, by chance, by 3 standard deviations is much less than a percent. But the probability that one among 25 bins will do so — or that two or three bins together will fluctuate up by something less than 3 standard deviations each — is much more than a percent. And the probability that you’ll get a 2 standard deviation upward fluctuation somewhere within 25 bins is closing in on 50 percent! (This reduction in the statistical significance due to multiple bins is often called the “look-elsewhere” effect, which I discussed previously here in the context of this summer’s nearly coincident earthquake and hurricane in the northeastern USA.)
So I can believe that, if there is a Standard Model Higgs at 125 GeV, CMS might have now an excess of 2 σ in one bin, or collectively in two or three neighboring bins. I can even believe that ATLAS might have a 3 σ excess of a similar sort; they might have gotten lucky, and a small signal might be sitting on an upward fluctuation of the background. But I cannot believe that the correct statistical significance of the results, once the look-elsewhere effect from the multiple bins is accounted for, will be as large as the raw rumors have suggested.
In fact, we saw this same issue this summer at the Grenoble conference, when ATLAS reported an excess in their Higgs search that reached 2.8 σ. Any expert knows that you can’t just take a number like that at face value, without understanding it better, and that’s why, immediately following Kyle Cranmer’s talk on the subject, I went to ask him about it. And indeed I learned (and reported to you) that one had to be very careful interpreting that “2.8”, because the look-elsewhere effect was not included in it. Instead the probability of the excess was a few percent, not the fraction of a percent that 2.8 σ would naively imply.
The really important question that December 13th will answer, aside from whether these excesses exist at all, is whether the rumors are true that the ATLAS and CMS excesses are located at the same value of the Higgs mass. They’d better be almost exactly in the same place; if one is at 126 GeV and the other is at 123 GeV, that won’t count for much, because the mass measurement from collisions that make two photons is precise to the level of 1 to 2 GeV. If both experiments have an excess at the same location, that will be favorable evidence in favor of a new particle, but given the current amount of data it will not be statistically enough for any claim of a discovery, much less confirmation that the new particle is the Standard Model Higgs particle. Fluctuations of this size do happen occasionally.
[For instance, earlier this year, at that same Grenoble conference, the CDF experiment at the Tevatron reported four events with two charged leptons and two charged anti-leptons that were roughly consistent with a non-Standard Model Higgs particle at 327 GeV. And in this case, there was essentially no background to worry about; these four events were all alone, off by themselves. It certainly looked significant! Yet the observation appears to have been a fluke; CDF could not confirm the result when they looked for other related processes that they should have been able to observe, and neither DZero (CDF’s competition at the Tevatron) nor ATLAS nor CMS has seen comparable events, which would have happened by now].
Assume for a moment the rumors are all correct; then, to be sure that the current Higgs evidence isn’t a statistical fluctuation that will disappear with more data, we’ll need to see what things look like when there’s at least twice as much data available and analyzed as there is now. That is unlikely to happen before June 2012, since the LHC is off for the winter and won’t restart til March. [And really, we’d probably want four times as much data to reach the coveted level of 5 σ evidence, which might mean late summer or fall.]
So. Everyone should just relax. Even if there is evidence for the Higgs at 125 GeV, and even if, down the line, the evidence turns out to be correct, the Higgs search is very unlikely to deliver a clear message on December 13th…
… unless the Higgs particle which is the subject of current rumor is not a Standard Model Higgs particle, and is being produced and/or is decaying to two photons at a larger rate than would be the case for a Standard Model Higgs. In that case my estimate of the size of the signal could be too small, and the evidence in favor of the Higgs particle could be much stronger than I expect. That would be fantastic! I wouldn’t bet on it though.
After this post was written but before it appeared, CERN produced a press release, dated December 6, in which it is stated:
“A seminar will be held at CERN on 13 December at which the ATLAS and CMS experiments will present the status of their searches for the Standard Model Higgs boson. These results will be based on the analysis of considerably more data than those presented at the summer conferences, sufficient to make significant progress in the search for the Higgs boson, but not enough to make any conclusive statement on the existence or non-existence of the Higgs.”
Remember the CERN management has actually seen the data… So you may or may not trust me, but you should trust them.