Matt Strassler 12/19/11
Particle physics sure had an exciting week, with the latest update on Phase 1 of the search for the Higgs particle. (During Phase 1, the ATLAS and CMS experiments at the Large Hadron Collider (LHC) focus on finding or excluding the simplest possible Higgs particle, called the Standard Model Higgs particle.) The unambiguous news is that the two experiments collectively have excluded the Standard Model Higgs particle unless its mass-energy [E = m c2] lies in a very small window between about 115 and about 128 GeV, or in a second window between about 600 to 800 GeV (which is disfavored by indirect evidence.) That’s not to say that a more complicated type of Higgs particle (or particles) might not have a mass in the excluded range, but if the simplest type of Higgs particle is the one found in nature, then the experiments are closing in on it.
The ambiguous but even more exciting news was that both ATLAS and CMS happen to have seen some hints of the Higgs particle, and some of their hints are at about the same mass, namely about 125 GeV. The question that the current article addresses is this: how confident should you be that these hints actually reflect a real signal of the real Higgs particle?
This summer, we had some hints too, and I wrote an article titled “Why the current Higgs hints rest on uncertain ground.” And indeed they did; they’re long gone now. The current situation is resting on firmer footing, but as you’ll see, I think you can make good arguments both in favor of confidence and in favor of caution. (You’ll also see that I think you can make bad arguments in favor of confidence, and I’ll try to explain why you should avoid them.) So I’m going to show you persuasive arguments that point in opposite directions, and I am not going to try to convince you which one is right. In fact, I’m going to try to convince you that while each of us will ascribe different levels of plausibility to the two arguments, it is really difficult to dismiss either of them out of hand. That’s why we need a successful 2012 run with lots of data; only then can the situation change.
The Data from the Higgs Search Update
Let’s begin with the data itself. Eight separate measurements played a role in last week’s update of the search for the Standard Model Higgs particle. Four of them each separately have a small impact, and their plots themselves don’t contain much information. Let me show you the plots from the other four. These are what I called (in this article from the summer and in this more recent article about how one searches for the Standard Model Higgs particle) the “easy” searches: the ones whose plots should tell you by eye (eventually) what is going on, because with sufficient data they should give a clearly observable peak in the data, poking up above a smooth background like an isolated volcanic cone on a gently sloping desert plain. (If you’re a layperson feeling lost already, you might instead want to listen to the excerpts from my public lecture at the Secret Science Club; these are the two searches I described.) These are the searches for a Higgs decaying to two photons, and for a Higgs decaying (through real and virtual Z particles) to “four leptons”, by which we really mean two charged lepton-antilepton pairs (specifically electron-positron and muon-antimuon pairs, which can be measured precisely.) In both of these cases, the search for the Higgs particle is relatively easy, because when one takes the two photons (or four leptons) and adds up their energies (or more precisely, combines their energies and momenta to form their invariant mass), one will find it equals the Higgs mass-energy if the photons or leptons came from a Higgs particle, while other non-Higgs sources of two photons or four leptons will be smoothly distributed. This can be seen in all four figures, which show the results of these two searches at both ATLAS and CMS. On each plot are shown data (black dots), the expected average background (a smooth distribution) and one or more examples of what a signal would look like on average (little peaks — but read the captions and labelling carefully to avoid being misled.)
Now we proceed with the arguments. Before we do that, let’s note that various statistical arguments about probabilities are going to come up. These include the look-elsewhere effect (described here) and also the question of how likely it is that two features in two different plots should be close together. The problem (as always in statistics) is that the answers to statistical questions always depend on exactly what you ask. We already know from the summer’s data that the lightweight Standard Model Higgs particle is restricted to the range 115 to 141 GeV; should we only compute the look-elsewhere effect within this restricted range of any plot, or should we compute it across the full range of the plot? The conservative thing to do is to use the full range when computing a look-elsewhere effect, but unfortunately when asking the probability that two features line up, the conservative thing to do is to use the restricted range, so it just isn’t clear what you should do, and your answers depend on what you do. Keep an eye on this, as in making the case for confidence I’ll be tacking back and forth, using the conservative argument whenever possible yet still showing there’s a strong case. You might yourself want to make a less conservative case; that’s a judgment call.
A Good Argument that the LHC Experiments are Seeing Signs of the Higgs Particle
Let’s start with the ATLAS two-photon results. These are easy to interpret, because the data is an almost featureless curve except for two significantly high bins, between 125 and 127 GeV. How significant is the excess? It is (locally — that is, within those bins) a 2.8 sigma excess, almost reaching the point of official “evidence” for something deviating from a smooth curve. With the look-elsewhere effect (that is, accounting for the fact that there are there are 80 bins on the plot) this drops to a 1.5 sigma excess — meaning the probability of having a 2.8 sigma excess somewhere on the plot is about 7 percent. That’s not so exciting, but still, it could be argued that is somewhat pessimistic, since we’re really only looking for the Standard Model Higgs particle now in the range 115-141 (other regions were removed after the HCP conference) so the number of bins where such an excess would be taken seriously is smaller than that.
Now, if that were all there was, we would not get that excited; we’ve seen equally exciting bumps in LHC plots before — even in the plots of the Higgs search for two photons. But of course, there is more.
Next we go to the ATLAS search for Higgs particles decaying to four leptons. If there were no signal, what we would have expected to see in the restricted range of 115 to 141 GeV is about 3 events, scattered around in different bins. Instead, three events were observed within 1 GeV of each other. That’s surprising; it’s quite different from what one would expect from background, and much more what one would expect from a Higgs particle signal. It’s a 2.1 sigma excess, though admittedly after the look-elsewhere effect (for this measurement alone) the probability of such an excess is somewhere in the range of about 50%. (Why so big? I think because the resolution on the measurement is 2 GeV, so the extreme closeness of the three events is somewhat of an accident. Again one could argue this is a pessimistic number.) A bit striking, but since we expected 3 events, it’s not as though we’re seeing more than anticipated in the absence of a Higgs signal. The only surprise is how close they are together.
But the really striking thing is that the two excesses just mentioned are within 2 GeV of one another. If these localized excesses were located at random bins, the probability that they would be within 2 GeV of one another is conservatively about one in 6. (Set the two-photon excess at 126 GeV; then the range 124-128 GeV, about 15% of the restricted range before this measurement, would get you within 2 GeV.) So that makes the likelihood that this is a pure fluctuation at least 6 times smaller yet. Altogether the probability for all of this to happen in these two searches is about 1 percent, conservatively.
In short, ATLAS has got something you might want to call “strong hints approaching the point of preliminary evidence” for a new particle around 125 GeV. Both the excess in two photons and the excess in four leptons are significantly bigger than expected for a Standard Model Higgs particle, so you might argue that the evidence is for a non-Standard Model Higgs particle, with an increased production rate. ATLAS’s case is further bolstered (slightly) by the small excess seen in the sensitive but subtle, and less distinctive, search for a Higgs particle decaying to a charged lepton, anti-lepton, neutrino and antineutrino (“leptons + neutrinos” for short). (It would seem that ATLAS decided that only half of its data in this search was fully ready for prime time, so the influence of this search is rather weak, but it does show a 1.4 sigma excess.) Taken together, ATLAS data shows a hint whose probability to appear as a pure fluctuation is down somewhat below 1 percent, after look-elsewhere effects. That’s a pretty decent hint!
Now we can ask whether CMS’s results are consistent with ATLAS’s case. CMS has five measurements, none of which is compelling on its own, but there is a case to be made when they are combined. The two-photon result at CMS shows a couple of 2 sigma excesses. That’s not surprising — as the CMS people say, the probability of that is about 20%. But what is interesting is that one of those excesses is within 2.5 GeV of both ATLAS’s two-photon and four-lepton excesses, which is something that again has a probability of only about 1 in 5 (see the reasoning above for ATLAS’s four-lepton results). CMS’s four-lepton results also involve a number of events strewn about, but two of them (one more than expected) lie again within 2.5 GeV of ATLAS’s excesses. And finally, CMS has small excesses in its search for Higgs particles decaying to leptons + neutrinos, to bottom quark/anti-quark pairs, and to tau lepton/anti-lepton pairs. All the excesses in the five different searches are of roughly the right size to be consistent with a Standard Model Higgs particle with a mass about 124 GeV.
So ATLAS has some hints, and CMS has some hints. How much better does the situation get if we combine them? Somewhat better, but if we’re honest we can’t go overboard here. First, the ATLAS excesses are somewhat more consistent with a Standard-Model-like Higgs particle with an enhanced production rate, while CMS’s excesses are not. That’s not an inconsistency, but it also means there isn’t exceptional consistency yet, and it means that either ATLAS got very lucky to get so large a hint in both photons and leptons, or CMS got really unlucky in not seeing signs of an enhanced non-Standard Model Higgs particle signal. Second, the ATLAS two photon excess is at 125.9 and the nearest CMS excess is at 123.5, while the stated resolution of the two photon measurements by the two experiments are better than that. Honestly, they ought to be closer together, if they’re seeing the same real thing. It’s not impossible for the backgrounds to fluctuate in such a way that they look further apart right now than they will after more data is obtained, but at least right now we can’t say they are remarkably consistent. But we certainly can’t say they disagree with one another. Let us say they are “roughly consistent”, certainly enough to add some weight to the case.
Now that’s just the evidence from the data. It’s somewhere between weak and moderate, perhaps crossing the threshold where you would officially call it “evidence” using the statistical convention that particle physicists use. But it’s not the only information we have.
We also know that the Standard Model is a remarkably successful theory, agreeing in detail with thousands of different measurements at many different experiments of widely varying types. At any given time there are disagreements here and there, but outside of dark matter and neutrino physics, none of them have stuck. And the Standard Model has the simplest possible Higgs particle in it — the so-called Standard Model Higgs particle. So the Standard Model, along with its Higgs particle, remains the best assumption we’ve got until we learn something’s wrong with it. That’s a theoretical bias, but a reasonable one. Bolstering that bias is that high precision measurements of many types allow a prediction, if we assume the Standard Model is correct, of the Higgs particle mass — a rather imprecise one, to be sure, but the preferred value of the Higgs mass would be lightweight. The most preferred value from the indirect evidence is actually below 115 GeV, but that is ruled out by the LEP experiments; 115 GeV would be the most likely value not already ruled out by experiments, but 125 GeV would still clearly be well within the natural part of the range still remaining. So a Standard Model Higgs particle at around 125 GeV is very much consistent with all the world’s experiments. And this point arguably can be combined with the evidence from the ATLAS and CMS data — preliminary as it may be — to create confidence, a strong hunch, a willingness to bet, that this is what the experiments are seeing. It makes a coherent, compelling story.
In my view this is a strong and reasonable argument, and it persuades some very reasonable people. Before we look at why there’s a strong argument that points in a different direction, I am going to point out what I consider to be a bad version of the argument, one that concludes that there is firm evidence in favor of the Higgs particle. You can skip that part if you want — it’s really more of an aside than anything — and jump to the good argument in favor of skepticism.
A Line of Argument to Avoid
A bad version of the above argument would use the success of the Standard Model as an additional source of evidence that the Higgs particle has been observed, instead of as a reason for belief, as it is used above. The reason this is a bad idea is that the Standard Model is precisely what we are trying to test through the search for the Standard Model Higgs particle, so assuming it biases the evidence. (It’s similar to weighing evidence against the most likely suspect in a murder without first having ruled out suicide as the cause of death. Assuming a murder has taken place artificially inflates the likelihood of guilt, and so the consistency of the assumption with the evidence should not itself be included in the weighing of the evidence.) Obviously if we assume the Standard Model is right, there must be a Standard Model Higgs particle in nature; and the success of the experiments in ruling out such a Higgs particle everywhere except 115 to 127 GeV then implies that it must be somewhere in the remaining 12 GeV window. Since we can only determine the Higgs mass right now to two or three GeV, that makes the probability of the Higgs being in the 125 GeV range already 15-25% before we even start weighing the data itself, artificially inflating the weight of the evidence.
Not only is this a biased argument, it also rests on a logical flaw. The past success of the Standard Model is not strongly correlated with whether there is a Standard Model Higgs particle in the LHC data. Imagine we extend the Standard Model of particle physics by adding just one type of undetectable particle — perhaps this is even the type of particle that makes up dark matter, so this is even reasonably well-motivated. Doing so will not affect any of the thousands of measurements that agree with the Standard Model, or any of the precision measurements that predict a lightweight Higgs particle. Yet this one new particle can put the Higgs discovery far out of reach. If the Higgs particle often decays to a pair of these undetectable particles, none of the search strategies currently seeking the Standard Model Higgs particle have a chance of finding it anytime soon, and discovery of the Higgs will be significantly delayed. Discovery may possibly require a search specifically aimed at an invisibly-decaying Higgs, which requires a lot of data and has not yet been undertaken.
So to use the success of the Standard Model as an ingredient in weighing the evidence in the data is faulty logic as well as artificially inflationary. It is completely consistent with all the world’s data for the Higgs to decay invisibly 90% of the time, and then Phase 1 of the Higgs particle search will exclude the Standard Model Higgs particle rather than find it. For this reason I don’t think you should assume the Standard Model is correct when evaluating the data. It seems obvious to me that you should evaluate the data first, on its own merits, and only then determine your level of confidence in the recent hints based on your prejudices regarding the Standard Model. Otherwise you will confuse “firm evidence” with “weak evidence, supported by a strong prejudice, leading to firm belief”.
A Good Argument that It’s Too Early to Be Confident that the LHC Experiments are Seeing Signs of the Higgs Particle
Before I begin, let me apologize to the experimenters involved, lest I offend. I have to take devil’s advocate positions here in order to make my point, and I certainly do not mean to cast any specific aspersions on any one of your measurements. I am just trying to illustrate the kinds of questions that a reasonable outsider might ask about your results — recognizing that although you are all professionals, you’re all still human, and we all know that even the best experimenters can be swindled by nature, or make subtle errors, when doing the hardest measurements. Nor, as I said earlier, am I recommending to anyone to take this particular position. I am aiming just to show that there is a good argument to be made.
Let’s start with the CMS two-photon data, looking at it in detail. The excess near 123.5 is not very big (2 sigma) and there is another one in the same data at around 136, along with two 2-sigma dips. The probability of getting two 2-sigma excesses somewhere in CMS’s data is 20%, not unlikely at all. Indeed, if you just showed me that data without first showing me ATLAS’s data, I’d probably conclude that I was looking at perfectly natural fluctuations. So there’s not much to go on there.
Here’s another more subtle point. The excess is best fit with a new-particle signal at 123.5 GeV, and is declining back to expectation by 126 GeV. Yet the data point that most exceeds the background curve is at 125-126 GeV. How can it be that the best fit point and the most discrepant point differ by 2 GeV, which is larger than the resolution on the measurement? Because there are four different classes of photons into which CMS divides its search, and this largest excess comes from the class with the least-reliable photons, which has the largest relative error and thus the largest probability of a large fluctuation. (Do not simply add ATLAS and CMS histograms!) And since they do differ in this way, doesn’t this teach us we really can’t interpret anything about that plot by eye? How much should you trust a measurement that you cannot yet interpret by eye as well as by statistical arguments? Altogether you might conclude that CMS’s two-photon data doesn’t really point in any clear direction. Maybe it could support a stronger case within CMS, but we need to find that case.
Perhaps the four-lepton case is stronger? Not if we look at it by eye without knowing that ATLAS has a hint at 125 GeV. There are again multiple hints in this data; it looks reasonably consistent with background, with small upward fluctuations that are completely typical with the Poisson statistics that characterize samples with very few events. The overall rate is a little high, but we’ve seen excesses like this in many other plots. Yes, there are two events in one bin at 125-127 GeV, but as you can see from the peaks drawn on the plot, a Higgs signal at 125 would be distributed from 122 to 128 GeV, so the extent to which this looks striking is misleading.
So it is only by putting two weak cases together that we really find ourselves even talking about something happening at 124 GeV. The evidence for a new particle there is very slim.
To make the case stronger we go to the measurements of the rates for Higgs decaying to the three other channels for which there is rate information but very poor mass information. Each of these rates is a bit larger than expected — by 1 sigma — none of them very significant. Worse, to interpret these measurements we have to trust the data-driven extraction of a background rate. (The data-driven extraction of a rate for a Higgs particle decaying to two leptons + two neutrinos gave us the hints of a Higgs particle at around 143 GeV this summer. I said then it rested on uncertain ground; for all I know, it still does. Similar issues affected the Tevatron’s first exclusion limits for the Higgs particle around 160 GeV; after the limits first appeared, they got worse before they got better.) It is true that the mistakes in estimating the backgrounds that one could make in the three cases are independent, which might lead one to argue that the three excesses taken together are significant. But one also has to remember that any mistake that comes from having omitted a source of background inevitably underestimates the total background; in other words, the systematic errors on many LHC measurements, including these, are not only non-Gaussian but also skewed to positive values — which makes the possibility of fake signals much larger than naive statistics would suggest. So we have to choose whether to trust that CMS estimated all of these errors correctly. Recalling how often during recent decades there have been underestimates in the determination of overall backgrounds rates in various measurements at hadron colliders, we may also reasonably choose not to, at least not until their methods have been fully vetted by independent experts. And we may not wish to rest a case for a discovery of a Higgs particle upon these searches at all.
Also, when one tries to make a strong case by combining many weak arguments, one can no longer be so confident in uncertainty estimates. It is easy to imagine underestimating them, because of the likelihood that a mistake lurks in one of the measurements, making it inappropriate to combine it with the others using standard statistical techniques. Another issue is that CMS uses the fact that the various excesses are all consistent with a Standard Model Higgs particle at 124.5 GeV. That’s fine as far as it goes, but it involves putting in a lot of assumptions. There might be no Standard Model-like Higgs particle in nature; there could be one with a non-Standard Model production rate, or non-Standard Model decay rates to b’s, tau’s and photons, and in this case the five excesses would not, in fact, be correlated as expected. Given that we are in the process of testing the Standard Model, we should be careful about assuming it in building an evidentiary case.
But without that assumption, the CMS case is a lot less convincing. Meanwhile, if you do make that assumption, you are then somewhat surprised that ATLAS has such a big excess in its searches for two leptons and four photons.
Let’s now see what we can learn from ATLAS. The same complaint about the measurement in leptons + neutrinos applies to ATLAS as to CMS — the uncertainties are hard to interpret — so for evidence let’s focus on the others. Let’s start with four leptons. It looks pretty solid: three events in one bin. But the expectation in the range from 115 to 141 GeV was for three events, and the expectation for a Standard Model Higgs signal would be two more events. So to see three events in one bin and none in any others requires the signal to be there and fluctuate up, and also for the background to fluctuate significantly down (or for one background event to land in the same bin as two signal events, etc.) Moreover, the experimental resolution in 4 leptons is about 2 GeV, which is pretty darn good, but that still means (see the figure at left) that we would expect three events from a pure signal to be spread out much more than they are. The point is that what ATLAS observes is actually overly striking, misleadingly so; it is not a particularly typical distribution of events if in fact we’re looking at the predicted background plus a Standard Model Higgs signal. Of course, with the number of events so low, there are wide fluctuations around “typical”. But it’s not the kind of distribution that immediately looks like a Higgs signal sitting over a Standard Model background. No matter what, this is a big fluctuation in either pure background or in background plus signal.
It also must be said that we tend to badly underestimate how often funny things like this happen. Something similar happened this summer, almost drowned out in the Higgs-signal hullaballoo (I was planning to write about it and it just got lost in the shuffle.) The CDF experiment reported finding 4 events that were (on the face of it) consistent with a new particle of mass of about 325 GeV decaying to two Z particles, which in turn each decayed to a lepton/anti-lepton pair. And the background in this case is really tiny! Look at the plot in the figure: Four events, isolated from any others by tens of GeV, with none to the right of them. The total number of events expected across that upper range is two or three. Is that a new particle? Why isn’t everyone jumping up and down about these four clustered events? Especially since CMS and ATLAS also have events in that region?! (CMS even has a 2 sigma excess!)
(1) Because CDF immediately looked for the other signals of the production of two Z particles: a lepton/anti-lepton pair plus a neutrino/anti-neutrino pair, and a lepton/anti-lepton pair plus a quark/anti-quark pair. I am not sure I believe their methods really excluded all possibilities, but they claim to have ruled out the possibility of a particle at 325 GeV decaying to two Z particles. Also, (2) almost any production mechanism you can think of for any such particle would hand ATLAS and CMS at least twice as many events as CDF by now. So it’s a fluke either way: either there’s nothing there and CDF was subject to a big fluke in its background, or there is a new particle there, and either CDF got a big fluctuation upward or ATLAS and CMS have large fluctuations downward. And finally, (3) in contrast to the hint of a Higgs at 125 GeV, this hint is located where no one is expecting a new particle. So we downplay these hints, and meanwhile we play up the current Higgs hints because we are expecting something there. We must not confuse evidence with prejudice, and with belief and disbelief.
And let us not forget about one of the last decade’s great (un-)discoveries in the physics of the strong nuclear interaction: pentaquarks, a new class of hadrons. I haven’t described them on this website because after several years of data and hundreds of papers, the pentaquarks all apparently turned out to be mirages. Here are some of the plots showing the evidence for the most convincing pentaquark, which had a mass of 1.54 GeV. With nine experiments seeing something similar, the evidence looks pretty good. But it wasn’t. (Thanks to a commenter for helping me find this particular plot.)
The point is that weird things happen in real data. And ATLAS was expecting three events in the search for Higgs decaying to four leptons. Maybe they got them, and they all ended up in the same bin. Could happen.
Now is it really so striking that they are so close to the two photon excess that ATLAS sees? Well, as I also emphasized in the good argument in favor, they’re borderline close; the gap between the photons events and the leptons events is almost 2 GeV, and the resolution on the leptons is about 2 GeV, so basically any cluster of events in the range between 123 and 129 GeV would have gotten our attention. That’s a pretty good chunk of the range between 115 and 141, so this coincidence of peaks is not quite as unlikely as it looks. Yes, it is moderately striking that the three events in one bin at ATLAS are near to the two photon excess. But let’s not overstate it.
Finally, what about the two-photon excess at ATLAS? It’s too big. It’s too big for a typical background fluctuation, which is why we tend to think it is signal; but it’s also too large, and very misshapen, for a typical signal fluctuation, as you can see on the plot, where the dotted red line shows what a signal ought to look like on average. What this tells you is that the specific shape of the excess is most likely driven by a background fluctuation; though it is unlikely for a large background to fluctuate that much, it is also unlikely for a small diffuse signal to do it. So the fit that tells us that any signal is most likely at 126 GeV might shift over time as the background fluctuation dies away, and perhaps it’s apparent concordance with the other evidence will die away with it.
It is also interesting that the point (126-127 GeV) in the ATLAS data that most strongly deviates from the background curve is an exceptionally low point in the CMS data. The two plots are not very much in accord.
Also, let’s not forget the wiggles in the two-photon data that we saw in the summer and last winter. They’re no less significant than the one we see now. In fact, if we look back at the two-photon data in CMS and ATLAS from the summer, the combined peaks at 120 GeV were more significant than the one we see in ATLAS now.
So again, we really have to be careful about over-ascribing significance, and underestimating the possibility of flukes, in data. And from this line of argument, one might conclude that this is really a circumstantial case. If we did not already believe that there was a strong possibility that there is a Standard Model Higgs particle at 125 GeV, we would not be persuaded of it by this data; and therefore the evidence is too weak to inspire confidence, because confidence should not be based on prejudice. The data might be pointing us toward a Higgs particle, and it might not.
There is one more issue that we should remember, and we should not be confident until it is resolved. These results are preliminary, which many commentators seem to forget, or at least not to understand. What might preliminary mean in this case? It means that there are various cross-checks and calibrations that the experiments have not yet completed. And one has to remember that the energies of all the particles observed — the leptons and the photons — have to be measured to something like 0.5% or better in order that the extracted invariant mass in each event be measured to better than 1%-2%, that is, to 1.25 – 2.50 GeV. That is not easy. [We sometimes forget how difficult a measurement this is because of the suppressed zeroes on the horizontal axes of all the plots; if we plotted the mass range from zero to 150 GeV, you'd be more impressed at what the experimenters are doing.] But the entire case for a Higgs particle rests upon this having been done correctly. The fact that this data is preliminary means that between the time that the data was presented and the time that it appears in its final version, individual events, or classes of events, might migrate, in mass, perhaps by 0.5 to 1 percent. (It is unlikely in this case that an event or two might even be removed due to its dropping below a quality requirement, but that does occasionally happen too.) These shifts could be enough, potentially, to either significantly improve or significantly worsen the concordance internally within each experiment (for instance, what if two of the ATLAS four-lepton events move down by 1 GeV, drawing the average of the four-lepton mass measurement down to 123.3 and away from the two-photon measurement at 126?). And they could worsen or improve the concordance of the two experiments’ results with each other; what if the four-lepton results at CMS move from 125 to 124 and the four-lepton peak at ATLAS moves from 124 to 125? I am not sure which of the potential uncertainties from the uncompleted calibrations and other details are included in the current error bars. But commentators who try to combine the results of the two experiments without accounting for the possibility of shifts in the results might, in my opinion, be leaving out potentially the largest uncertainty in the estimate of the significance of the combination. Since the ATLAS and CMS results are close but not perfectly aligned, especially in the case of the two-photon searches at ATLAS and CMS, one may wonder whether the final results might show significant changes, not in any one of the eight experimental results from ATLAS and CMS viewed separately, but in how well they are correlated with one another, and how consistent they are collectively with a signal of a new particle.
All this is to say that the evidence of a new particle at ATLAS and CMS is pulled together from a number of pieces of weak and in some cases questionable evidence from eight different measurements, all of which are preliminary and might shift slightly. The uncertainties and instabilities in the combination may well be underestimated. One may worry that it would only take one or two of these pieces of evidence to shift significantly, or crumble, to cause the entire case to unravel, or at least weaken sharply. The reverse might happen, of course; it might be that the final results are in greater accord, making all of us more comfortable and the combined significance larger. But until the final results are out, the significance of the combination is unstable, and one might wish to reserve judgment on whether there’s anything there in the data to be trusted.
And never forget the pentaquark debacle, its many cousins that are recorded in scientific history books, and its many cousins that are not .
I don’t know how to tell you how to choose between these two lines of argument. I know how I choose; when I see an argument in favor of caution that seems as strong or nearly as strong to me as an argument in favor of confidence, I remember how commonly false signals have fooled us throughout history, and err on the side of caution. But you won’t hear any complaints from me if you choose to be more confident — as long as you use sound reasoning, and only apply your prejudice in favor of the Standard Model when determining your level of belief, rather than in your claims of evidence.
Personally I think the chance that a Standard Model-like Higgs particle is at 125 GeV is pretty decent, so it won’t surprise me at all if it turns out to be there. That’s not merely because of the evidence in the data, which I view as pretty thin, but because it aligns with some very reasonable prejudices about nature — in particular, the very wide variety of theories which predict or at least allow a Standard Model-like Higgs particle in that mass range.
But we won’t know without more data, and with more data we will know — on that I think we all agree. And if all goes well, the LHC will take enough data in 2012 to change the highly ambiguous situation we’re in into one that creates a consensus in the community regarding the hints that ATLAS and CMS are seeing. Not that this is likely to happen very quickly; if there is a real Higgs particle there, ATLAS got lucky and got more events than expected, so it is unlikely that with double their data they will get double the signal. And if there is no signal there, it will take a lot of data (as ATLAS speaker Fabiola Gianotti emphasized) to wash the large fluctuation in the two-photon data away. (Of course CMS’s data, currently less exceptional, will be crucial in settling the case one way or the other.) So we need to be patient. No matter what each of us personally believes, the community consensus that marks a scientific achievement is probably a year away. It will be an exciting year of nail-biting anticipation, and we should not be surprised if the case first becomes more ambiguous before it becomes clear.