[This post is now out of date; for a discussion of the data showing strong arguments in favor of confidence, and against confidence, that the excesses in the data reflect the presence of a new particle, click here.]
At 2:30 CERN time today, there was a 90 minute discussion in a room packed with over 100 people, including many experimentalists from the LHC experiments ATLAS and CMS and particle theorists like myself. Not one experimentalist in the room — many of whom actually participated in the Higgs particle searches — used the word “signal” or “evidence”, and every theorist who asked about the “signal” was quickly reminded that an “excess” in a plot is very different from a “signal”. The four experimentalists who spoke very briefly (about the searches for the WW, ZZ and two-photon decays of the Higgs) all made cautionary remarks. There were reminders that the various excesses are still not very significant, and that results are still preliminary and might shift slightly in ways that could increase or decrease their internal consistency with each other. One comment made was that if you re-ran the CMS experiment many times, and there were no Higgs signal at all, you would get (just through statistical fluctuations in the background) two excesses similar to those seen in the CMS two-photon plot 20% of the time. Some very essential differences between the ATLAS and CMS analyses were discussed too. (CAUTION to those who want to naively combine the ATLAS and CMS results! make sure you account for these! Otherwise you will get the wrong answers. And do not add any ATLAS and CMS histograms either.) Various concerns about each analysis, especially the fits to the background in the two-photon searches (which are importantly different in ATLAS and CMS) were voiced during the questions. Much of the discussion was quite technical and very illuminating, focusing on many subtle details (which I have to know about, but most of you can safely ignore.)
What’s the lesson? The experiments have done a fantastic job, and have squeezed the allowed mass for a Standard Model Higgs particle down to a very small range; the hints of a Higgs signal, in each of ATLAS’ and CMS’ search for a particular Higgs decay, are still very faint; and the people who actually performed the analyses, while hopeful to greater and lesser degrees, clearly do not widely believe that the combined case, using all the hints together, is firm evidence of anything unusual — yet.
53 Responses
Dear matt,
It seems pretty clear to me that part of the disagreement on the strength of the indications in the CMS & Atlas results stems from the difference between Frequentists and Bayesians in probability theory. Frequentists are a large majority among physicists and almost all experimentalists are Frequentists. Tommaso Dorigo is the only Bayesian experimentalist I can currently name. A minority of theoreticians are Bayesians.
The situation is very different among mathematicians, I understand (One Probablist I know says he wouldn’t let his daughter marry a Frequentist).
Perhaps a short Note for lay peopleabout the divison of opinion and saying why Bayesians are more optimistic in this situation would be helpful?
Regards,
David Perkin
It might well be a useful article, though I’d have to learn more to do it properly. But I can tell you already what my conclusion will be. A scientific result is not established until both Bayesians and Frequentists agree that it is established — implying that Frequentists always determine when consensus occurs.
Hi Sergei,
No, actually what Matt said, and what I objected to, was “the people who actually performed the analyses, while hopeful to greater and lesser degrees, clearly do not widely believe that the combined case, using all the hints together, is firm evidence of anything unusual.”
Judgements based on unclear evidence are required all the time. Undergraduates decide which experiment to join / graduate school to go to with almost no knowledge of the field, yet the choice of experiment can have a huge influence on their career. Funding agencies have to decide which experiments to fund and which to pass up, without the benefit of knowing what the result will be in advance. Prudent young theorists should clearly be thinking about the 125 range – waiting until more data is available is a recipe for being scooped. And health management decisions, which are clearly more important for the individual involved than whether there is a higgs are not, are based on p<0.05.
Clearly with the higgs we can afford to wait and see, but in most cases in life we have to make a decision based on gut level beliefs about probability (which are unfortunately often wrong).
Any way, getting back to "no evidence of anything unusual": I think that "no judgement whatsoever" until 5 sigma and then "discovery" with 5 sigma is a crazy worldview. Clearly some things are more or less likely, and right now one of the things that looks more likely is that 2012 will reveal something around 125. I freely acknowledge that it might go the other way, but joining the "50-50 camp" reminds me of a TV episode:
Waitress: "Yeah, right, a riot here. What are the odds of a riot breaking out in our small town?"
Cop: "50-50."
Waitress: "How do you figure that?"
Cop: "Either there is a riot or there isn't, it's 50-50. You know, for someone who handles money all day, you're not very good at math."
Now now — look at what I’m actually saying. I’m certainly not recommending to anyone not to take the possibility of a 125 GeV Higgs seriously. Not to either theorists or experimentalists. But it would be bad if we forgot that the possibility that it isn’t there, and that we’re being fooled, is also real. We have to keep planning for that eventuality too.
Remember, compared to where we were last month, 50-50 at any one mass point is very high odds!
You can be sure that over the coming months I will be writing papers in which I will assume
(1) that the Higgs is at 125 GeV and is Standard Model-like, or
(2) that the Higgs is at 125 GeV and is not Standard-Model-like, or
(3) that what we’re seeing is a fluke and the Higgs is Standard Model-like and in the range 115-123, or
(4) that what we’re seeing is a fluke, the Standard Model is not right, and the Higgs is (or Higgses are) somewhere else.
And I am sure that lots of people will do (1) and (2), and many will do (3) and (4) also.
See today’s article for an explanation as to why I’m personally not convinced yet — which as you’ll see is not about the statistical significance itself, but about the stability and plausibility of the current statistical claims — and why I think it is nevertheless quite reasonable to disagree with me (but only on the basis of a legitimate argument.) http://profmattstrassler.com/articles-and-posts/the-higgs-particle/holiday-higgs-hints-confidence-inspiring-or-not/
I did, and I would say that while it was going on experimentalists probably thought it was “firm evidence of something unusual”. Indeed, even now, several experiments getting things wrong is also fairly unusual in my books.
Ok — on this we agree, especially if you replace “experimentalists” with “some experimentalists”. A lot of the discussion that has been taking place in these comments has a semantic element, and that’s why I am writing something that is longer and clearer than one can toss off in a comment reply.
The pentaquarks are nice examples because there were so many confirmations before the case fell apart. Now we’re talking about the Higgs, on which the direction of the field rests — and for which we have (and will have!) only two experiments. I feel (and clearly some others do too) that we must be especially careful as we proceed, and require a higher level of proof than in less crucial situations.
Gillian,
You are actually talking about “probability” that “probability” is within curtain range. While the second is based on limited set of experimental data, the first one is pure guesswork. When presented with insufficient data the hardest but prudent coats of action is make no judgement whatsoever and simply wait until more data is available. This is what Matt has been arguing all along.
I was trying to keep my post short, but because of that you misunderstood my meaning of “unusual”.
It’s not unusual to know a few tall people, but it is unusual to be tall. It’s not unusual for there to be a lottery winner, but it is extraordinarily unusual to be a lottery winner.
When experimentalists see a 3.5 sigma “bump” they don’t say “well, 50% chance it’s just a statistical fluctuation,” they say “hmmm, that’s unusual. Maybe it’s a statistical bump, but worth keeping an eye on it.”
There are exceptions, of course. If you are looking at a huge histogram, you expect to some fluctuations, and if you’ve got no reason to think that something funny should be going on there you’re less likely to worry about it. But this bump is WAY bigger than the “look elsewhere effect” in a region where it’s reasonable to expect to see something. In fact, dozens of measurements isn’t close to what’s needed to get a 3.5 sigma bump (assuming the systematics are correct), it’s more like 1000 measurements.
I’m looking forward to your next post. I don’t think you think the odds that this is a higgs/the higgs/something are 50-50, but the way you’ve been writing makes it sound like that.
If the only thing I was worried about was statistics, then yes, you are right, I would not be in the 50-50 camp. Did you notice the comment above about pentaquarks?
Matt said: “the people who actually performed the analyses, while hopeful to greater and lesser degrees, clearly do not widely believe that the combined case, using all the hints together, is firm evidence of anything unusual”
Come on. The way you say it makes it sound like they think the null hypothesis is equally likely. You’re doubling down because of a public spat, and that’s making you look equally (or more) absurd. The question to ask (yourself) is, what odds do you think an Atlas or CMS experimentalist would give you on a private bet that the higgs isn’t there. I can’t seriously believe that you think they wouldn’t bet you, say, 4-1 that there is a higgs there.
I’m not arguing that people shouldn’t be cautious or precise in what they say, I’m just pointing out that in your desire to prove Tommaso wrong you’re going overboard yourself.
“Firm evidence” of the higgs? Well, that’s up for debate. Firm evidence of “something unusual”. Clearly. A 3.5 sigma bump due to statistics alone IS unusual. Perhaps not evidence of anything, but certainly unusual.
I appreciate your comment. But I disagree that “a 3.5 sigma bump due to statistics alone is unusual”. Not when you’re doing dozens of measurements, it’s not. I have seen several 3 – 4 sigma bumps already this year, yet we are not declaring the Standard Model dead yet.
So yes. I am not the kind of person to double down. I am the kind of person to be honest. I am telling you that I am not confident, not to make a point, but because that is where I am.
I have spent the last week talking publicly and privately to ATLAS and CMS experimentalists. Some of them would probably give me a 4-1 bet. Most of the ones I talked to would not. But of course (as people who talk about bets always forget) their willingness to bet would depend on how many dollars are involved. I’m sure they’d all bet me a dollar to four (why not?) but the real question is whether they would bet their houses.
Stay tuned for a post later that may clarify where I stand. It is complex so it has taken a while to compose.
Most of the discussions in the blogs seem to be about the peaks around 123 GeV. I was wondering if there is anything near consensus about the peaks near 119 or near 130 GeV. Can those be combined with the 123 GeV peaks? Are those more likely to be statistical flukes, or from something other than a Standard Model Higgs? Can you tell us about the discussions about these?
Great series of posts, by the way!
If there is a Standard Model Higgs particle, its signals will all show up within two or three GeV of one another. The other peaks are likely just statistical flukes. The argument in favor of a Higgs at 125 GeV involves taking several bumps that by themselves look like statistical flukes but which happen to roughly coincide at around 125. If there really is a Standard Model Higgs particle there, what will happen over time is that the other bumps will diminish and die away while the peak at 125 will become sharper and more distinct. We’ll see next year whether that happens.
There have been quite a few comments on this site that treat sigmas as just another unit of measure like say eV, meter or gram. It is nothing of a kind – it has limited meaning when dealing with strictly normal distributions. In the real world sigmas are no more than very crude attempts to describe what is essentially unknown. Outcome of individual random event is unknown, but unknown is not necessarily random. Only in very few cases sigmas have clear statistical meaning, most of the time they are just an analogy of what it would like if the process in question were true random. To do even simple math on sigmas in most cases is not much different than correlating distribution of tea leaves in a cup and world events.
I’d like to ask sigma lovers to curb their enthusiasm and let this site to do what it does best – help non-physists like myself to keep up with what is happening on the cutting edge of physics.
Dear Sergei,
much like radians, the number of standard deviations is obviously a dimensionless quantity. One may add “rad” or “sigmas” to clarify what we’re talking about but the difference from electronvolts, meters, and grams is that there exists a totally canonical, exceptional unit, namely one sigma, something that doesn’t exist in the measurements of energy, length, or mass.
Moreover, it’s not true that one can’t talk about the number of sigmas if distributions are non-normal. The standard deviation always is equal to something and one may talk about its multiples. What’s not true is the Gaussian (error-function-based) dictionary between the number of sigmas and the p-value.
More generally, I think that the importance of non-Gaussianities etc. is vastly overstated by comments like yours. There are lots of reasons why normal distributions are normal – that’s also why they’re called normal. It boils down to the ground states of harmonic oscillators, central limit theorem, and many other things that imply that the normal distribution is a satisfactory description in a wide spectrum of situations.
Best wishes
Lubos
Dear Lubos,
I am happy that normal distribution works for you. I on the other hand in forty years since my university days have not yet run into problem where I could use one. I would like to continue discussion on randomness which is my field, but it is not my blog and I am afraid I have already taken to much space away from physics.
Can someone tell me which major discovery in the history of particle physics was based on a combined significance of 3 sigma or 5 sigma?
Each experiment was supposed to have 5 sensitive channels( gamgam,tautau,lvlv,4l and bbar) at 125GeV, if each one gave 1.5sigma, the combined significance would be close to sqrt(2*5*1.5^2) = 4.7sigma. You don’t see a peak in any channel, but you get something close to 5sigma, would you believe this?
The fact of the matter is no a single analysis see an excess that is significant enough to call for analysts to scrutinize the result to an extent that a discovery deserves.
The combined significance came later when most analyses were being finalized. Most analysis teams had extensive checks to make sure there were no mistakes in analysis itself. But, there’s no enough check on how the excess seen behaves. Experiments haven’t entered the stage of comparing the physics properties of the excess with signal, except the comparison of the signal strength. Unfortunately, it’s too large to be SM Higgs.
Please look at Fig. 7 of http://arxiv.org/abs/hep-ex/0504027
As you can see, there were 9 experiments seeing the Pentaquark at around 1540 GeV/c^2. The peaks are really strong and the evidence seems overwhelming. If you combined the data you might claim to get 7 sigma and everybody who doubts the peaks are obviously crazy. Still if you examined the spectra carefully enough, you will see that the peaks appear in slightly different positions and the widths are actually incompatible with each other. In many of the cases, a higher-statistics experiment repeated later on using a similar analysis produced a completely flat spectrum, so that most people had to conclude that at least some of these 9 peaks are fake. This happened less than 10 years ago. As someone not involved in the Higgs search but having a lot of past experience with exotic signal searches, I would say it is justified to be cautious, although I understand that from a theoretician’s point of view it is attractive to assume the Higgs signal is there and celebrate.
Thank you, I have been looking for this summary. It is an excellent object lesson.
As an interested outsider I would like to thank you Matt for having expressed your thoughts so openly. There is great value in that.
Again as a total outsider I would also like to say that I am puzzled by the certainty some commentators express about theories which are often presented as incomplete. The possibilities of composite Higgs, multiple Higgs, no Higgs are often mentioned. But all plots assume standard model Higgs in their modeling, and perhaps more importantly they compute backgrounds on the basis of the standard model, which is very successful yes, but full of parameters, with alternative theories often favored in some ways (yes, when discussing other aspects, like neutrino experiments, cosmology, etc.).
If the discussion has to go deeper why not (try to) mention numbers from all those alternative theories, which represent the whole range of possibilities?
Adding to this, systematic experimental uncertainties which are not much mentioned. I do believe they are well understood, but if the discussion heats up why not explicit all the details instead of remaining superficial/intuitive?
Regarding this, what is the p-value for the 2-photon event peaks differing as they do between Atlas and CMS assuming SM Higgs at 125, 126, or 124? I suspect this depends heavily on the details of accelerator and detector setups, like perhaps slight calibration issues, etc. but what is the best fit, and why?
In conclusion I would say that I understand public expositors are under the pressure of justifying investment in experimental physics, showing that they understand “well-enough” what they are doing. I think this may bias things toward discussing everything assuming well-known facts (the SM/Higgs), so perhaps reminding this possible bias is useful -which is what I try to do.
From this perspective, I greatly appreciate the uncertainty you (Matt) express, and your efforts to communicate it to outsiders, struggling with difficult simplifications.
Please anyone correct me where I am wrong.
“And I watched several theorists get a stiff warning about over-interpreting the results, in public, with 100 or more witnesses.”
This is hilarious. If the experimentalists feel this way then WHY IN HELL DID THEY CALL A PRESS CONFERENCE AT ALL?! Seems that exaggerating the importance of what has been found so far is a terrible crime if done with 100 witnesses, but it´s ok if it is done in front of millions of people all over the world!
“We would like to announce to the world that we have not found the Higgs so far. Yes, there is something “excessive” in there, but please, this is NOT a discovery. Oh no. Nobody here heard me say that! No indeed. We have not discovered anything. Zilch. Nada. The sole purpose of this press conference is just to make sure that the world knows what a fantastic job we are doing. But in saying that we are doing a fantastic job, I don´t mean to hint that we have discovered anything. Oh no, not at all…….”
You are being unfair to CERN management, I think. They called the press conference to announce their updated results on the Higgs search using the 2011 data… not to announce a discovery. Anyone who did studies on simulated 2011 data would have predicted (as I did in my posts) that there would not be enough data even for strong evidence. Of course the CERN people knew that from the beginning, months ago.
It is certain blogs that made a huge deal out of it, and got the press’s full attention. That’s not entirely CERN’s fault.
I’m just not understanding what the supposed negative role of “certain blogs” has been here. They gave an accurate preview of what CERN was going to announce, then an accurate description and analysis of what was announced. The idea that CERN held this event because of blogs is absurd. Every HEP physicist in the world, along with everyone of the millions of people interested in HEP has been dying to know the answer to the question: “what have the LHC experiments been seeing relevant to the Higgs in the data from the 2011 run?”. CERN organized this to provide an answer to that question, did a great job, and the press did a great job in covering it. I can’t figure out what problem you are obsessed with.
I’d like to ask about the standards for ‘evidence’ and ‘discovery’ in use here. As I understand it, three sigma is called evidence and five sigma a discovery. These limits are put artificially high to compensate for the look-elsewhere effect and in fact assume the LEE is fairly substantial. Correct so far?
Obviously, taking the LEE into account is a good thing, if you can put some actual numbers on it. For the individual experiments, this has been done. What would constitute ‘evidence’ and ‘discovery’ with the LEE accounted for? Obviously you can’t require the same 3 and 5 sigma then, at least if you still consider 3 and 5 without LEE correction to be valid. What would be reasonable requirements?
Additionally, if the old, existing, currently in use, standards for ‘evidence’ and ‘discovery’ say that Atlas alone clearly has ‘evidence’, what’s the reason for not calling it that?
I think the 5 sigma is to take into account that there is always likely to be some unaccounted for systematic errors or biases of order a sigma and also that although LEE is accounted for they are not accounting for the fact that there are many particle physics experiments been done and so unless there is a high threshold there will be some by chance that erroneously report a discovery.
Matt,
Thanks for your answer, what you said is reasonable. Just to clarify, I am of course aware that the discussion at CERN was not the same as a public statement since this was a cross-talk between experimentalists and theorists and thereby too technical for the general public. But still public so in this sense the care with potential dangerous statements that could be wrongly interpreated must be made exactly the same. It is obviously very different from an internal experimental meeting.
But let’s move on I am glad you agree we should move the discussion to more interestig things and I am sure I will learn with your next post.
It is fine for Matt to have hid say and call the rest of us naive theorists, but he should not then expect us to move on without responding first.
Philip,
You mean about the probability of the Higgsless scenario?
Matt: reading your previous post, I got the impression that you were a very strong Higgs supporter. But, in front of data, you have showed the greatest honesty and professionalism in this post. I must give you my biggest salute to you although my salute might not carry much weight, but it is all I got.
For myself, I must admit that I cannot reach your level of fairness. In my view, the Higgs mechanism is fundamentally wrong as it is only a shadow of the reality. Being as a shadow, it did allow the Standard Model correctly predicting many great results. Of course, my view was labeled as one of the biggest crackpot in “The CRACKPOT Page, http://www.physics.smu.edu/scalise/www/misc/crackpot/ “ for almost 20 years now. Most of physicists know about this crackpot page. Again, I appreciate your kindness of treating me as a person who is simply yearning for truth, crackpot or not.
I have been sitting back watching the discussions.
It seems that with an configuration space assigned in terms of calorimeter evidence, there is some expectancy, or not, why would you set up such parameters?
In theory then, production of testable features for such evidence were needed. How is this all to play out but given such expectancy, the understanding of a long journey to finally be satisfied realistically. This is all part of the history of the fanfare of LHC development having reached this point in time. It is as it should be.
This said nothing about the result, but it does bring together the need for cooperation. An accountability between a theorist and an experimentalist for the truth. A phenomenological approach to the current understanding of where science is, and how such experiments must continue to be developed?
The views of late have been really interesting.
Best,
Im not surprised that the professionals involved in these experiments are trying to be as disciplined as possible not to assume more than they know for sure: thats their job. But I don’t agree that the same standards as professional experimenters should be applied to the blogosphere. I completely agree, for example, that experimentalists should disregard undue theoretical biases. But for knowledgeable people assessing where we stand online, the theoretical considerations are pretty important, and are kind of silly to disregard.
Isn’t the entire point of using precise figures involving standard deviations so that we can avoid having to guess about the meaning of words like “firm”?
CMS reported the combined significance of the excesses at 1.9 sigma, which I believe corresponds to 94%, and that is after the look-elsewhere effect is included. ATLAS has 2.3 sigma/97.9%. Now I realize there is a subtle proper way to combine those, but from a glance it looks pretty clear that the combined significance will be, well. significant! ;] Also Lubos has raised a point, and I think its a good once, whether it makes sense to punish both of those figures with the same look-elsewhere effect when you combine them. That sounds a bit more cautious than necessarily justified.
I know you’ve already clarified much of this, and I dont wish to force you to repeat yourself. Im just saying it seems to me that the overall tone of many reports was basically justified. But your skeptical discipline is much appreciated, as is the rest of your work!
Matt is a theorist who works very closely with data so I think that would partly explain why his outlook is more that of an experimenter. I do sympathise with Dorigo’s Bayesian argument though: its would be a quite a coincidence that the first relatively significant joint excess is pretty much where the simplest theory says it would be.
Is this to some extent a reflection of different cultures in experiment and theory? That is, the price of being wrong as an experimenter is much higher than the price of being wrong as a theorist. I think it is fair to say that theorists are judged by their best work and not their worst. ‘Speculative’ papers do not cost theorists – in some fields they are even expected to a certain degree – but publishing a wrong result is a killer for the career of an experimentalist. Across a large fraction of hep-ph and hep-th I think theorists see their duty to physics as providing good new ideas, while the prime experimental duty to physics is to be correct.
This is just my personal take on it. However it does fit with the way experimenters are far more cautious than theorists in interpreting excesses as signals and are willing to dampen expectations rather than raise them.
There is an element of truth in that but let’s not forget that Strassler is a theorist and Dorigo is an experimetalist,
There are theorists and there are theorists; there are experimentalists and there are experimentalists. Neither I nor Dorigo are typical of either group.
I’m condensed matter rather than high energy, but I think you’ve really hit upon a major factor here. The stakes are magnified in high energy, since the total number of experiments is small – if one of those is wrong, it’s a major hit to the whole subfield.
Just took a look at the Gordy Kane/Rob Roser chat at Science magazine this afternoon.:
“Gordy Kane:
YES. an experimenter from one experiment can’t say that, but theorists see that two different experiments both saw a signal at about the same mass, and also saw additional channels, so it’s a discovery!
…
Gordy Kane:
The 5 sigma is a criteria people have chosen. i think as soon as the data are combined from two detectors, which is entirely legitimate, then the signal will indeed be over 5 sigma.”
which seems to be completely insane, on a par with:
“Comment From Tom
Does the existence of a 125 Gev Higgs give any support to supersymmetry?
Gordy Kane:
Yes. first, for a long time it has been known that the lightest higgs boson of supersymmetry should be lighter than about 135 GeV (actually closer to 140 GeV but people make assumptions), so this is consistent. Then the supersymmetric string theories as i mentioned do predict the 125 number and it is a supersymmetric lightest higgs boson.”
It seems to me that a lot of confusion comes from a failure to properly differentiate three fundamentally different kinds of probabilities in these discussions:
– distributions that are clearly defined by the laws of physics and produce clear, meaningful statistics
– deviations in experimental data that are assumed to be random but statistics are ugly and math is often questionable
– purely human feeling “what are the chances of …”
When the word “probability” is used carelessly, without clear definition of what does it mean in a particular statement, confusion and pointless arguments are always in abundance.
Agreed; see above.
Matt,
You got this last point right. The problem with the seminar you attended is precisely the fact you attended. This was a public discussion and the same care Fabiola anf Guido had to take in the announcement must be made. And it is correct to say for example that there is no signal at 125 GeV, just an excess. Therefore, nobody can even by accident publicly say there is a signal there without being silenced. But I assure you that many experimentalists do BELIEVE the excess will become a signal. It is for now just a bet, not a strong scientific point given the fact that nobody is able to decide, only the data will. I can´t reveal who answered but the question Murray asked was “is there a Higgs between 123 and 128 GeV”? You would be surprised the people who said “yes”, some of them twice to reinforce, but nearly all were in the room today.
All in all I think this is going to far. You have often very nice and instructive posts and I´m glad to read what you write since I learn. But ask yourself if some of your comments in this post are contributing to the understanding of the science. I think you already made clear you do not approve the over optimism with this since you fear if this is not real (which could be the case) this will be bad for HEP. I can sympathize with this feeling but I don´t think we are so much in risk. On the absence of a conclusion from data people are entitled to an opinion and if they want t bet this is a signal, provided they make clear it is a strictly personal opinion I see no damage and I think/hope the public will get that.
I would be much more happy if you for example could write for example a post explaining, if this were a signal (adding the big if), what else besides a SM Higgs this could be, and why? What are your favorite candidates? This would be a much nicer and healthier discussion.
Bernhard — I think you are wrong about the discussion; it was not at all like the Guido/Fabiola show, with all the media watching. It was much more aggressive and detailed (which is why I reported nothing specific that I heard, lest I get someone into trouble.)
About the tone: you are right, of course, and having made my point (that there is widespread disagreement in the field, and that theorists should be cautious in interpreting the data) I do intend to cease this unhealthy direction and address questions of this type. Partly I am reacting to a blogosphere that is not nearly cautious enough about the current excesses, and that sends the wrong message to the public, the media and young people entering the field. Had others been more responsible with their public statements, my reaction would have likewise been more responsible.
Furthermore, I want to emphasize again that I am not arguing that there is no Higgs particle there, and that we are not seeing signs of it. What I am arguing is there is a difference between saying
1. there is strong evidence in the data in favor of a new particle
2. there is weak evidence in the data, but we have a reasonable theoretical bias that causes us to believe it may well reflect a Higgs particle — one from the Standard Model.
Let me stress that: I do see that theoretical bias toward the Standard Model as a reasonable one. But it is a bias; it is not something in the data itself.
Experimentalists who cannot and do not carefully delineate the difference between evidence and belief are the most likely ones to make mistakes. (Theorists make these mistakes all the time, but the cost is vastly lower.) It is always amazing to me how many smart people cannot do this properly; it is an essential part of rigorous thinking.
This issue of “belief” in an interesting one. If you are an experimentalist inclined to “believe” a priori in the existence of a standard model Higgs boson at any mass, then it must be nearly certain that you must “believe” this particular excess will turn into evidence/discovery for the said boson. Because as Matt said, the LHC has done such a superb job of saying where the boson isn’t, that if this fluctuation goes away, there’s nowhere else left to hide. It’s the last stand for the vanilla Higgs boson.
If there actually is a vanilla Higgs at 125 GeV, I find it fascinating that it should be just in the place where it’s hardest to find. It’s as surprising to me as if it were sitting at WW threshold, ready for the Tevatron to discover. Is it really just chance, or is there a deeper reason to be uncovered?
I should remark — it is of course possible there may have been something of a selection effect at work here today. Perhaps those experimentalists who attended the session included a large number who wanted to make sure that naive theorists did not get carried away. But my point doesn’t change if that was the case: here you have first-rate experimentalists who actually participated in the analysis who aren’t even ready to say “evidence”, much less “firm evidence”. And without a consensus among the very most expert experimentalists that there is, in fact, evidence of a Higgs particle signal, wouldn’t you all agree that whatever evidence there is can’t possibly be called “firm”?
I notice that the slides shown at that meeting
https://indico.cern.ch/getFile.py/access?resId=0&materialId=slides&confId=166075
start off: “We desperately wanted a Higgs for Christmas: we got something which looks furiously like it”
It also is worth pointing out that despite all the irresponsible rumor-mongering, unapproved plot-producing, overly-firm evidence claiming bloggers, science journalists, who read both blogs and the CERN press releases, managed to put out stories over the past week as the story unfolded that were pretty accurate. On the whole they got it right: there’s some evidence for a Higgs (soft to firm, to taste) but still not enough to be sure it’s there. Science journalism gets a lot of criticism for being overly sensationalistic and just plain wrong, but this time they mostly got the story right, even though it’s a rather complex one. Anyone worrying about how the Higgs search is being covered and what the public is hearing about it might want to start worrying about something else.
Ah! — Peter, you couldn’t have fallen deeper into a trap if I’d set one for you.
The person who put that slide up is a theorist — an excellent and well-respected one, I must add, like Jack Gunion, with tremendous expertise on Higgs physics — and yet he was immediately shot down by the experimentalists in the room for having written those lines. He was specifically asked — in a not funny way — if he was joking.
As I said, I so wish you had been there.
I am aware of what Bill Murray said. I have no idea what he really asked, or who he asked. But I’m on the ground with the people who did and supervised the work. And I watched several theorists get a stiff warning about over-interpreting the results, in public, with 100 or more witnesses. So — believe what you want to believe, I don’t need to argue with you.
I’m not trying to argue, I am just trying to find a real understanding of why people are describing the same result in such different terms. I don’t believe that it is due to a lack of understanding on anyone’s part as you are suggesting. i think it is more a case of some people talking about a glass being 99% full while others are calling it 1% empty.
You complain about us using the word “signal” rather than “excess” for example, but the word is used to label the axis on some of the ATLAS plots and I have not used it in any sense that is different in meaning. Words like “firm” are obviously very subjective too. Perhaps everybody should be asked to estimate the probability that the observed “excess” would be as strong or stronger than it is if there were no Higgs. Then we might see if we really think differently.
If you said 50% full versus 50% empty, we might agree. You really don’t seem to me to take the oops-Leon 4-sigma bump, or the pentaquark 5-sigma bump, seriously enough.
If you think it is 50/50 then I take it all back. You really do have a quantitatively very different assessment of the strength of the evidence from me and I have not expressed it as strongly as some others.
“Perhaps everybody should be asked to estimate the probability that the observed “excess” would be as strong or stronger than it is if there were no Higgs.”
If you do this, could you please make it a mail question to people likely to have a good understanding rather than a poll on your blog? I’m afraid your blog posts would have an undue influence otherwise.
i’ve just posted my estimate of about 5% chance for the Higgsless scenario. I don’t suppose many other people will be willing to do the same but I agree that a poll would be useless. The detailed reasoning has to be given.
In your poll of experimenters you had 9 to 1 in favour of uncertainty, but Bill Murray in a QA session for Nature said he did a similar poll and got 10 to 0 in favour of the Higgs being found. Perhaps it just depends on how you ask the question. I think we probably all have about the same level of certainty vs uncertainty. That is clear from the results. The difference in what people say has nothing to do with failing to take into account LEE, systematics, etc.
Here is the link http://www.nature.com/news/live-qa-the-hunt-for-the-higgs-1.9642