The Future of Neutrino Physics：中微子物的未来

《The Future of Neutrino Physics：中微子物的未来》由会员分享，可在线阅读，更多相关《The Future of Neutrino Physics：中微子物的未来（25页珍藏版）》请在装配图网上搜索。

1、 Concluding Talk: PhysicsGary FeldmanHarvard UniversityPHYSTAT 05University of Oxford15 September, 2005Gary Feldman PHYSTAT 05 15 September 2005 2TopicslI will restrict my comments to two topics, both of which I am interested in and both of which received some attention at this meeting:lEvent classi

2、ficationlNuisance parameters Gary Feldman PHYSTAT 05 15 September 2005 3Event ClassificationlThe problem: Given a measurement of an event X = (x1,x2,xn), find the function F(X) which returns 1 if the event is signal (s) and 0 if the event is background (b) to optimize a figure of merit, say signal.

3、sb for discovery or ss+b for anestablishedGary Feldman PHYSTAT 05 15 September 2005 4Theoretical SolutionlIn principle the solution is straightforward: Use a Monte Carlo simulation to calculate the likelihood ratio Ls(X)/Lb(X) and derive F(X) from it. By the Neyman-Pearson Theorem, this is the optim

4、um solution.lUnfortunately, this does not work due to the “curse of dimensionality.” In a high-dimension space, even the largest data set is sparse with the distance between neighboring events comparable to the radius of the space. Gary Feldman PHYSTAT 05 15 September 2005 5Practical SolutionslThus,

5、 we are forced to substitute cleverness for brute force.lIn recent years, physicists have come to learn that computers may be cleverer than they are.lThey have turned to machine learning: One gives the computer samples of signal and background events and lets the computer figure out what F(X) is.Gar

6、y Feldman PHYSTAT 05 15 September 2005 6Artificial Neural NetworkslOriginally most of this effort was in artificial neural networks (ANN). Although used successfully in many experiments, ANNs tend to be finicky and often require real cleverness from their creators. lAt this conference, there was an

7、advance in ANNs reported by Harrison Prosper. The technique is to average over a collection of networks. Each network is constructed by sampling the weight probability density constructed from the training sample.Gary Feldman PHYSTAT 05 15 September 2005 7Trees and RuleslIn the past couple of years,

8、 interest has started to shift to other techniques, such as decision trees, at least partially sparked by Jerry Friedmans talk at PHYSTAT 03.lA single decision treehas limited power, butits power can be increasedby techniques that effectively sum many trees.A cartoon fromRoes talkGary Feldman PHYSTA

9、T 05 15 September 2005 8Rules and Bagging TreeslJerry Friedman gave a talk on rules, which effectively combines a series of trees.lHarrison Prosper gave a talk (for Ilya Narsky) on bagging (Bootstrap AGGregatING) trees. In this technique, one builds a collection of trees by selecting a sample of the

10、 training data and, optionally, a subset of the variables.lResults on significance of B g gen n at BaBarSingle decision tree 2.16 s sBoosted decision trees 2.62 s s (not optimized)Bagging decision trees 2.99 s s Gary Feldman PHYSTAT 05 15 September 2005 9Boosted Decision TreeslByron Roe gave a talk

11、on the use of boosted trees in MiniBooNE. Misclassified events in one tree are given a higher weight and a new tree is generated. Repeat to generate 1000 trees. The final classifier is a weighted sum of all of the trees.lComparisonto ANN:Also morerobust.% of signal retained52 variables21 variablesGa

12、ry Feldman PHYSTAT 05 15 September 2005 10Other TalkslThere were a couple of other talks on this subject by Puneet Sarda and Alex Gray, which I could not attend.Gary Feldman PHYSTAT 05 15 September 2005 11Nuisance ParameterslNuisance parameters are parameters with unknown true values for which cover

13、age is required in a frequentist analysis.lThey may be statistical, such as number of background events in a sideband used for estimating the background under a peak.lThey may be systematic, such as the shape of the background under the peak, or the error caused by the uncertainty of the hadronic fr

14、agmentation model in the Monte Carlo.lMost experiments have a large number of systematic uncertainties.Gary Feldman PHYSTAT 05 15 September 2005 12New Concerns for the LHClAlthough the statistical treatment of these uncertainties is probably the analysis question that I have been asked the most, Kyl

15、e Cranmer has pointed out that these issues will be even more important at the LHC.lIf the statistical error is O(1) and the systematic error is O(0.1), the the systematic error will contribute as its square or O(0.01) and it does not much matter how you treat it.lHowever, at the LHC, we may have pr

16、ocess with 100 background events and 10% systematic errors.lEven more critical, we want 5 s s for a discovery level.Gary Feldman PHYSTAT 05 15 September 2005 13Why 5 s s?lLHC searches: 500 searches each of which has 100 resolution elements (mass, angle bins, etc.) x 5 x 104 chances to find something

17、.lOne experiment: False positive rate at 5 s s (5 x 104) (3 x 10-7) = 0.015. OK.lTwo experiments:lAllowable false positive rate: 10.l2 (5 x 104) (1 x 10-4) = 10 3.7 s s required.lRequired other experiment verification: (1 x 10-3)(10) = 0.01 3.1 s s required.lCaveats: Is the significance real? Are th

18、ere common systematic errors?Gary Feldman PHYSTAT 05 15 September 2005 14A Cornucopia of TechniqueslAt this meeting we have seen a wide series of techniques discussed for constructing confidence intervals in the presence of nuisance parameters.lEveryone has expressed a concern that their methods cov

19、er, at least approximately. This appears to be important for LHC physics in light of Cranmers concerns.Gary Feldman PHYSTAT 05 15 September 2005 15Bayesian with CoveragelJoel Heinrich presented a decision by CDF to do Bayesian analyses with priors that cover. Advantage is Bayesian conditioning with

20、frequentist coverage. Possibly the maximum amount of work for the experimenter.lExample of coveragewith a single Poisson with normalization and background nuisance parameters:Flat priorsGary Feldman PHYSTAT 05 15 September 2005 16Bayesian with CoveragelExample of coverage with flat and 1/e e and 1/b

21、 priors for a 4-channel Poisson with normalization and background nuisance parametersFlat priors1/e e and 1/b priorsGary Feldman PHYSTAT 05 15 September 2005 17Frequentist/Bayesian HybridlFredrik Tegenfeldt presented a likelihood-ratio ordered (LR) Neyman construction after integrating out the nuisa

22、nce parameters with a flat priors. In a single channel test, there was no undercoverage. lWhat happens for a multi-channel case? My guess is that the confidence belt will be distorted by the use of flat priors, but that the method will still cover due to the construction.lCranmer considers a similar

23、 technique, as was used for LEP Higgs searches.lBoth are call “Cousins-Highland,” although probably neither actually is.Gary Feldman PHYSTAT 05 15 September 2005 18Profile Likelihoodl44 years ago, Kendall and Stuart told us how to eliminate nuisance parameters and do a LR construction:Gary Feldman P

24、HYSTAT 05 15 September 2005 19One (Minor) ProblemlThe Kendall-Stuart prescription leads to the problem that for Poisson problems as the nuisance parameter is better and better known, the confidence intervals do not converge to the limit of being perfectly known. The reason is that the introduction o

25、f a nuisance par-ameter breaks the discreteness of the Poisson distribution. From Punzis talkGary Feldman PHYSTAT 05 15 September 2005 20One More TrylSince this was referred to in a parallel session as “the Feldman problem” and since two plenary speakers made fun of my Fermilab Workshop plots, I wil

26、l try to explain them again.nbnbnbr = 1r 1known exactlyn b n b n b r = 1r 1known exactlyGary Feldman PHYSTAT 05 15 September 2005 21The Cousins-Highland ProblemlThis correction also solves what Bob and I refer to as the Cousins-Highland problem (as opposed to method).lCousins and Highland turned to

27、a Bayesian approach to calculate the effect of a normalization error because the frequentist approach gave an answer with the wrong sign.lWe now understand this was due to simply breaking the discreteness of the Poisson distribution.lIn one test case, using this correction reproduced the Cousins-Hig

28、hland result x/ 2.Gary Feldman PHYSTAT 05 15 September 2005 22Use of Profile LikelihoodlWolfgang Rolke presented a talk on eliminating the nuisance parameters via profile likelihood, but with the Neyman construction replaced by the-D DlnL hill-climbing approximation. This is also what MINUIT does. T

29、he coverage is good with some minor undercoverage. Cranmer also considers this method.Gary Feldman PHYSTAT 05 15 September 2005 23Full Neyman ConstructionslBoth Giovanni Punzi and Kyle Cranmer attempted full Neyman constructions for both signal and nuisance parameters.lI dont recommend you try this

30、at home for the following reasons:lThe ordering principle is not unique. Both Punzi and Cranmer ran into some problems.lThe technique is not feasible for more than a few nuisance parameters. lIt is unnecessary since removing the nuisance parameters through profile likelihood works quite well. Gary F

31、eldman PHYSTAT 05 15 September 2005 24Cranmers (Revised) ConclusionslIn Cranmers talk, he had an unexpected result for the coverage of Rolkes method(“profile”). He didin fact have an error and it is corrected here:Q ui ckTi m e?and aTI FF (LZW ) decom pressorare needed to see thi s pi cture.Gary Fel

32、dman PHYSTAT 05 15 September 2005 25Final Comments on Nuisance Parameters lMy preference is to eliminate at least the major nuisance parameters through profile likelihood and then do a LR Neyman construction. It is straightforward and has excellent coverage properties.lHowever, whatever method you choose, you should check the coverage of the method.lCranmer makes the point that if you can check the coverage, you can also do a Neyman construction. I dont completely agree, but it is worth considering.