概率与统计课件第一章

IntroductiontoProb&StatsProbabilityandMathematicalStatistics(概率与数理统计)XiZHANGCourseinformation•Prob&Stats•Probability•Statistics•Objectives•Tounderstandthebasicprinciplesinprobabilityandstatistics•Tolearnhowtosolvethereal-worldproblemswithprobabilisticandstatisticaltools•TothinkinaprobabilisticandstatisticalwaytheoryProbabilityStatisticsDistribution分布DiscreterandomvariablesContinuousrandomvariablesJointDist.MultipleR.V.sEstimation估计HypothesisTesting假设检验Regressionanalysis回归分析RealanalysisLinearalgebra试验设计随机过程数据挖掘图像处理时间序列分析大数据分析方法生物信息学精算质量管理计量经济人口科学化学计量人工智能地理信息系统可靠性分析审计药物信息遗传学ContentstobecoveredAdditionalinformation•Lecturing•Handouts（英文为主、中文注释）•Blackboardnotes•Code-basedexamples(willbeavailableatlabsessions)•ClassSchedule•Monday:8:00am~9:50am(biweekly)andThursday:8:00am~9:50am(weekly)•TAs•JuanDu(1401111633@pku.edu.cn);•BijuWang(bijuwang@pku.edu.cn)•TAisresponsibleforgradingtheassignments,labsessionsandofficehours(TBDbyTAs)•Contactinfo•Office:FounderBuilding512•Tel:82524911•Email:xi.zhang@pku.edu.cnTextbooksrecommended概率论与数理统计，何书元（第2版或第1版）AFirstCourseinProbability,the8theditionbySheldonRossMathematicalStatisticsandDataAnalysis,the3rdeditionbyJohnRiceElementaryProbabilityTheory,the4theditionbyKaiLaiChungandFaridAitSahliaCoursegrading•Assignmentssubmittedbyastudyinggroup（作业）25%•Astudygroupconsistsofupto3students,andtheywillreceivethesamegradeoneachsubmissionoftheirhomework•Courseproject（课程项目）15%(writtenreport+oralpresentation)•Eachgroupconsistsofupto6students.Studentsinthegroupwillreceivethesamegradeforthecourseproject•Courseprojectwillbeassignedaftermid-termexam•Midtermexam（随堂期中考试）20%（aroundNov.13rdor17th,TBD）•Finalexam（期末考试）40%(TBD)•Problemswillbesimilartoorevenalittlebitharderthanassignmentsandexamplesinthehandouts•Unlessanacceptableexcuse,thereisapenaltyonthelatedueassignmentsorcourseprojectreportaccordingtothefollowingrules:•Forlatesubmission,thegradewillbereducedby50%•Acurvewillbeconsidereduponyourperformancebutnotguaranteed.Gradingwillbechangedonlywhenanerrorhasbeenmade;negotiationisnotappropriate•Anyquestions?Origins•Let’sdatebackto1650’sinFrance•GamblingwasfashionableChevalierDeMereThrowingadieBlaisePascalPierredeFermatThecorrespondencebetweenPascalandFermatisthemathematicalstudyofprobabilityClassicalprobability古典概率Supposeagamehasnequallylikelyoutcomes,ofwhichmoutcomescorrespondtowinning.Thentheprobabilityofwinningism/nMenofProbabilityChristiaanHuygensJacobBernoulliAbrahamdeMoivreComtedeBuffonPierreLaplaceKolmogorovMarkovChebyshevPoissonCarlGaussYuan-ShihChow周元燊fromColumbiaUniversityKai-LaiChung钟开莱fromStanfordUniversitySamuelKarlinfromStanfordUniversitySheldonRossfromtheUniversityofSouthernCalifornia……MenofStatisticsThomasBayesRonaldFisherKarlPearsonFrancisGaltonJerzyNeymanAbrahamWaldGeorgeBoxDeterministic确定v.sStochastic随机•Ifallpast,presentandfuturevaluesofavariableareknownprecisely,withoutanyuncertainty,thenthevariableiscalledasdeterministic（确定的）.•However,inmostpracticalsituation,wecannotpredictvariablesexactly.Suchvariablesarecalledstochasticorrandom（随机的）.Thefundamentalcharacteristicofastochasticorrandomvariableistheinabilitytopreciselyspecifyitsvaluesbeforehand.CoursestructureProbabilityBasicprobabilityDiscreteRVsContinuousRVsMath.StatisticsParameterestimationHypothesistestLinearregressionanalysisDefinitionofprobabilityTwotheorems–Theoremoftotalprobability（全概率公式）–Bayes’theorem（贝叶斯公式）ProbabilityBasicprobabilityDiscreteRVsContinuousRVsMath.StatisticsParameterestimationHypothesistestLinearregressionanalysis•Distribution:PMF(概率函数)•ImportantPMFs•TransformationofRVs•Mean(均值)&variance(方差)•MultipleRVs:•Covariance(协方差)ProbabilityBasicprobabilityDiscreteRVsContinuousRVsMath.StatisticsParameterestimationHypothesistestLinearregressionanalysis•ImportantPDFs•CentralLimitTheorem(中心极限定理)•TransformationofacontinuousRVProbabilityBasicprobabilityDiscreteRVsContinuousRVsMath.StatisticsParameterestimationHypothesistestLinearregressionanalysis•Methodofmoments（矩估计法）•Maximumlikelihoodestimationmethod（极大似然估计法）•Bayesestimationmethod（贝叶斯法）ProbabilityBasicprobabilityDiscreteRVsContinuousRVsMath.StatisticsParameterestimationHypothesistestLinearregressionanalysisBasicconceptsofhypothesistest（假设检验的基本概念）–Nullandalternativehypotheses（原假设、对立假设）–TypeI&IIerrors（第一、二类错误）–Approachestohypothesistest（假设检验的步骤）Parametrichypothesistest（参数假设检验）Non-parametrichypothesistest（非参数假设检验）ProbabilityBasicprobabilityDiscreteRVsContinuousRVsMath.StatisticsParameterestimationHypothesistestLinearregressionanalysisSimplelinearregression（一元线性回归）Extensionofsimplelinearregression–Multiplelinearregression（多元线性回归）Supposewearethrowingafairdicexi123456P{X=xi}1/61/61/61/61/61/6Amorecomplicatedcasewithtwodicesyi23456789101112P{Y=yi}1/362/363/364/365/366/365/364/363/362/361/36Thesummationofthechancesisequalto1Reviewofbasicarithmetic•Counting•Howmanydifferent7-placelicenseplatesarepossibleifthefirst3placesaretobeoccupiedbylettersandthefinal4bynumbers?•Permutations(排列)•Howmanydifferentorderedarrangementsofthelettersa,b,andcarepossible?•HowmanydifferentletterarrangementscanbeformedfromthelettersPEPPER?•Exercise•亚运会参加女子十米气手枪的共有10人，其中4人来自中国，3人来自日本，2人来自韩国，另有1人来自印度。如果比赛结果只 记录 混凝土 养护记录下载土方回填监理旁站记录免费下载集备记录下载集备记录下载集备记录下载 选手的参赛国籍，那么一共有多少中可能结果？Reviewofbasicarithmetic•Combinations(组合)•Acommitteeof3istobeformedfromagroupof20people.Howmanydifferentcommitteesarepossible?•Exercise•Fromagroupof5womenand7men,howmanydifferentcommitteesconsistingof2womenand3mencanbeformed?Whatif2ofthemenarefeudingandrefusetoserveonthecommitteetogether?•Binomialcoefficient(二项式系数)Multinomial(多项式)Example燕园内学生生活区有10名保安哥，其中需要5名在小区巡逻，2名在小西门值班，另外3名留在保安室待命。把10名保安哥分成这样3组共有多少种不同分法？Exercise男篮10人打对抗赛练习，每队5人，一共有多少种分法？Somebasicdefinitions•Let’sthinkoftossingcoins•Experiments（试验）：e.g.thetossoftwocoins•Outcome/samplepoint（结果、样本点）:•Samplespace（样本空间）：Thesetofallpossibleoutcomes.•ThesamplespaceisdenotedasS.•Event（事件）:asubsetofthesamplespace•Eistheeventthataheadappearsonthefirstcoin•Impossibleevent（不可能事件）:thesetwithnooutcomesHs)},(),,(),,(),,{(TTHTTHHHS)},(),,{(THHHE•Union（并）：•ConsistofalloutcomesthatareeitherinEorinForinbothEandF.•e.g.:and•Intersection（交）：•ConsistofalloutcomesthatarebothinEandinF.•e.g.:and•If,EandFaremutuallyexclusive（互不相容）.•Complement（补）：•ConsistofalloutcomesthatarenotinE.•Subset（子集）：•Ifand,EandFareequal.FE)},(),,{(THHHE)},{(HTFappeared.headaleastatmeans)},(),,(),,{(HTTHHHFEFEEFor)},(),,(),,{(HTTHHHE)},(),,(),,{(TTHTTHFEFcEFEappeared.tailaleastatmeans)},(),,{(HTTHEFFEEFFESEFSEFSEEc•Commutativelaws（交换律）•Associativelaws（结合律）•Distributivelaw（分配律）•DeMorgan’slaw（德摩根定律）FEEFEFFE;F(EG)(EF)GGFEGFE);()(G)G)(F(EGEFFGEGGFE;)(nicicniiEE11)(nicicniiEE11)(Axiomsofprobability（概率论公理）•Supposethatanexperiment,whosesamplespaceisS,isrepeatedlyperformedunderexactlythesameconditions.ForeacheventEofthesamplespaceS,wedefinen(E)tobethenumberoftimesinthefirstnrepetitionsoftheexperimentthattheeventEoccurs.TheP(E),theprobabilityoftheeventE,isdefinedasnEnEPn)(lim)(Axiom1Axiom2Axiom3ForansequenceofmutuallyexclusiveeventsE1,E2,…1)(0EP1)(SP11)()(iiiiEPEPSomesimplepropositions•Proposition1•Proposition2If,then•Proposition3•Proposition4（inclusion-exclusionidentity,容斥恒等式）)(1)(EPEPcFE)()(FPEP)()()()(EFPFPEPFEP)...()1(...)...()1(...)()()...(211...112121212121nniiiiiiriiiiniinEEEPEEEPEEPEPEEEPnrClassicalprobability(古典概率)•Theprobabilityofaneventistheratioofthenumberofcasesfavorabletoit,tothenumberofallcasespossiblewhennothingleadsustoexpectthatanyoneofthesecasesshouldoccurmorethananyother,whichrendersthem,forus,equallypossible.(FromWikipedia)（指当随机事件中各种可能发生的结果及其出现的次数都可以由演绎或外推法得知，而无需经过任何统计试验即可计算各种可能发生结果的概率。）•特点•Predictable•Freeofexperiment•NoerrorsExamples•Apokerhandconsistsof5cards.Ifthecardshavedistinctconsecutivevaluesandarenotallofthesamesuit,wesaythatthehandisastraight（顺子）.Whatistheprobabilitythatoneisdealtastraight?•Ifitconsistsof3cardsofthesamedenomination（点数）and2othercardsofthesamedenomination,wesayitisafullhouse.Whatistheprobabilitythatoneisdealtafullhouse?Exercises•Ifnpeoplearepresentinaclassroom,whatistheprobabilitythatnotwoofthemcelebratetheirbirthdayonthesamedayoftheyear?Howlargeneednbesothatthisprobabilityislessthan1/2?•Computetheprobabilitythat10marriedcouplesareseatedatrandomataroundtable,thennowifesitsnexttoherhusband.Remarksinclassicalprobability•Classicalprobabilityabeautifulvision•Alloutcomesinthesamplespacearedetermined•Nootherunknowneffects/equaleffectsfromenvironment•Alltheresultsweinterestedcouldbededucted•Experimentalprobability（试验概率）andsubjectiveprobability（主观概率）arenecessaryMontyHallProblemAbrainteaser（脑筋急转弯）fromafamousshowinNBC(Let’smakeadeal!)You'regiventhechoiceofthreedoors:Behindonedoorisacar;behindtheothers,goatsYoupickadoor,sayNo.1,andthehost,whoknowswhat'sbehindthedoors,opensanotherdoor,sayNo.3,whichhasagoat.Hethensaystoyou,"DoyouwanttopickdoorNo.2?"Isittoyouradvantagetoswitchyourchoice?MontyHallProblemTheplayerhasanequalchanceofinitiallyselectingthecar,GoatA,orGoatBSwitchingresultsinawin2/3ofthetimeSeemoredetailsathttp://en.wikipedia.org/wiki/Monty_Hall_problembehinddoor1behinddoor2behinddoor3resultifstayingatdoor#1resultifswitchingtothedoorofferedCarGoatGoatCarGoatGoatCarGoatGoatCarGoatGoatCarGoatCarConditionalprobability（条件概率）•Theconditionalprobability𝑃(𝐸|𝐹)representstheprobabilityofeventEassumingeventFhappened•Denote𝑃(𝐸|𝐹)astheprobabilitythatEoccursgiventhatFhasoccurs•If𝑃𝐹>0,then)()()|(FPEFPFEPSEFExampleInthecardgamebridge,the52cardsaredealtoutequallyto4players—calledEast,West,North,andSouth.IfNorthandSouthhaveatotalof8spades(黑桃纸牌),whatistheprobabilitythatEasthas3oftheremaining5spades?339.01326102135Exercise•Atotalofnballsaresequentiallyandrandomlychosen,withoutreplacement,fromanurncontainingrredandbblueballs(n≤r+b).Giventhatkofthenballsareblue,whatistheconditionalprobabilitythatthefirstballchosenisblue?•YoucanalsothinkthisprobleminaneasywayThemultiplicationrule（ 乘法 99乘法表99乘法表打印九九乘法表a4打印九九乘法表免费下载大九九乘法表免费打印 规则）•AgeneralizationoftheconditionalprobabilityP(E2|E1)=P(E1E2)P(E1)P(E1E2)=P(E1)P(E2|E1)P(E1E2…En)=P(E1)P(E2|E1)…P(En|E1…En-1)ExampleAnordinarydeckof52playingcardsisrandomlydividedinto4pilesof13cardseach.Computetheprobabilitythateachpilehasexactly1ace(纸牌A).S=allwaystodivide52cardsamong4peopleE=everyonegetsanaceE3=A♣A♥A♦areallassignedtodifferentpeople=A♠A♣A♥A♦areassignedtodifferentpeopleE2=A♥A♦areallassignedtodifferentpeopleequallylikelyoutcomes𝑃𝐸=𝑃𝐸2𝑃𝐸3𝐸2𝑃(𝐸|𝐸2𝐸3)E2=A♥A♦areassignedtodifferentpeopleP(E2)=AfterassigningA♥itlookslikethis:??????????????????????????????????A♥?????????????????=3∙13/(52–1)=39/51#greycards#ofquestionmarksE3=A♣A♥A♦areallassignedtodifferentpeopleP(E3|E2)=2∙13/(52–2)=26/50AfterE2itlookslikethis:???A♦??????????????????????????????A♥?????????????????E2=A♥A♦areassignedtodifferentpeopleP(E2)=3∙13/(52–1)=39/51E3=A♣A♥A♦areallassignedtodifferentpeopleP(E3|E2)=2∙13/(52–2)=26/50E=A♠A♣A♥A♦allassignedtodifferentpeopleP(E|E3)=13/(52–3)=13/49P(E)=(39/51)(26/50)(13/49)≈0.105TheruleofaverageconditionalprobabilityP(E)=P(EF)+P(EFc)=P(E|F)P(F)+P(E|Fc)P(Fc)=P(E|F)P(F)+P(E|Fc)[1-P(F)]SEFFcEF1F2F3F4F5P(E)=P(E|F1)P(F1)+…+P(E|Fn)P(Fn)Moregenerally,ifF1,…,FnpartitionSthen全概率公式ExampleInansweringaquestiononamultiple-choicetest,astudenteitherknowstheanswerorguesses.Let𝑝betheprobabilitythatthestudentknowstheanswerand1−𝑝betheprobabilitythatthestudentguesses.Assumethatastudentwhoguessesattheanswerwillbecorrectwithprobability1/𝑚,where𝑚isthenumberofmultiple-choicealternatives.Whatistheconditionalprobabilitythatastudentknewtheanswertoaquestiongiventhatheorsheanswereditcorrectly?ExerciseAlaboratorybloodtestis95%effectiveindetectingacertaindiseasewhenitis,infact,present.However,thetestalsoyieldsa“falsepositive”（FP，伪阳性）resultfor1%ofthehealthypersonstested.(Thatis,ifahealthypersonistested,then,withprobability0.01,thetestresultwillimplythatheorshehasthedisease.)If0.5%ofthepopulationactuallyhasthedisease,whatistheprobabilitythatapersonhasthediseasegiventhatthetestresultispositive?RussianrouletteBushandKimtaketurnsspinningthe6holecylinderandshootingateachother.WhatistheprobabilitythatBushwins?outcomeHMHMMHMMMHMMMHprobability1/65/6∙1/6(5/6)2∙1/6(5/6)3∙1/6(5/6)4∙1/6P(A)=1/6+(5/6)2∙1/6+(5/6)4∙1/6+…=1/6∙(1+(5/6)2+(5/6)4+…)=1/6∙1/(1–(5/6)2)=6/11SolutionusingconditionalprobabilitiesA=“Bushwins”={H,MMH,MMMMH,…}W1=“Bushwinsinfirstround”={H}Ac=“Kimwins”={MH,MMMH,MMMMMH,…}P(A)=P(A|W1)P(W1)+P(A|W1c)P(W1c)5/61/61P(Ac)P(A)=1∙1/6+(1–P(A))∙5/611/6P(A)=1soP(A)=6/11ExerciseUrn1initiallyhas𝑛redmoleculesandurn2has𝑛bluemolecules(微粒).Moleculesarerandomlyremovedfromurn1inthefollowingmanner:Aftereachremovalfromurn1,amoleculeistakenfromurn2(ifurn2hasanymolecules)andplacedinurn1.Theprocesscontinuesuntilallthemoleculeshavebeenremoved.(Thus,thereare2𝑛removalsinall.)•Find𝑃(𝑅),where𝑅istheeventthatthefinalmoleculeremovedfromurn1isred.•Repeattheproblemwhenurn1initiallyhas𝑟1redmoleculesand𝑏1bluemoleculesandurn2initiallyhas𝑟2redmoleculesand𝑏2bluemolecules.Foranyparticularredmolecule,let𝐹betheeventthatthismoleculeisthefinaloneselected.Let𝑁𝑖betheeventthatthismoleculeisnotthe𝑖thmoleculetoberemoved,wehavennnNNFPNNNPNNPNPFNNPFPnnnn11111|||1111211Ifwenumberthenredmoleculesandlet𝑅𝑖betheeventthatredmoleculenumberjisthefinalmoleculeremovednnRPnj11111111enRPRPRPnnjjnjjForanymoleculethatisinitiallyinurn1.Asinpart(a),itfollowsthattheprobabilitythatthismoleculeisthefinaloneremovedis111111122brbrpbrifweletObetheeventthatthelastmoleculeremovedisoneofthemoleculesoriginallyinurn1,then22111111brbrpbrOP2222112221111111111||brbrccbrbrrbrbrrOPORPOPORPRPDefinitionofodds（优势/几率）•TheoddsofaneventAaredefinedby•ConsidernowahypothesisHthatistruewithprobabilityP(H),andsupposethatnewevidenceEisintroduced•ThenewoddsaftertheevidenceEhasbeenintroducedareAPAPAPAPc1EPHPHEPEHPEPHPHEPEHPccc||||cccHEPHEPHPHPEHPEHP||||ExampleAnurncontainsonetypeAcoinandonetypeBcoin.WhenatypeAcoinisflipped,itcomesupheadswithprobability1/4,whereaswhenatypeBcoinisflipped,itcomesupheadswithprobability3/4.Acoinisrandomlychosenfromtheurnandflipped.Giventhatthefliplandedonheads,whatistheoddsthatitwasatypeAcoin?Bayes’sformula(贝叶斯公式)•Supposethat𝐹1,𝐹2,…,𝐹𝑛aremutuallyexclusiveevents•Exactlyoneoftheevents𝐹1,𝐹2,…,𝐹𝑛mustoccur（完备事件组）•Weobtain•Bayes’sformulaSFnii1niiEFE1niiiniiFPFEPEFPEP11|niiijjjjFPFEPFPFEPEPEFPEFP1|||ExampleSupposethatwehave3cardsthatareidenticalinform,exceptthatbothsidesofthefirstcardarecoloredred,bothsidesofthesecondcardarecoloredblack,andonesideofthethirdcardiscoloredredandtheothersideblack.The3cardsaremixedupinahat,and1cardisrandomlyselectedandputdownontheground.Iftheuppersideofthechosencardiscoloredred,whatistheprobabilitythattheothersideiscoloredblack?Independentevents(独立事件)•IfeventEisindependentofeventF,then•TwoeventsEandFaresaidtobeindependentifandonlyif•Twoeventsthatnotindependentaresaidtobedependent（相依）•ThreeeventsE,F,andGaresaidtobeindependentifEPFEP|FPEPEFPGPFPFGPGPEPEGPFPEPEFPGPFPEPEFGP•Proposition•IfEandFareindependent,thensoareEandFc•Proof:•Since𝐸=𝐸𝐹∪𝐸𝐹𝑐ccEFPFPEPEFPEFPEPccFPEPFPEPEFP1ExampleIndependenttrialsconsistingofrollingapairoffairdiceareperformed.Whatistheprobabilitythatanoutcomeof5appearsbeforeanoutcomeof7whentheoutcomeofarollisthesumofthedice?Ifwelet𝐸𝑛denotetheeventthatno5or7appearsonthefirstn−1trialsanda5appearsonthenthtrialSinceP{5onanytrial}=4/36,andP{7onanytrial}=6/3611nnnnEPEP364361011nnEP4.018131191181391111nnnnEPThegambler’sruinproblemTwogamblers,AandB,betontheoutcomesofsuccessiveflipsofacoin.Oneachflip,ifthecoincomesupheads,Acollects1unitfromB,whereasifitcomesuptails,Apays1unittoB.Theycontinuetodothisuntiloneofthemrunsoutofmoney.Ifitisassumedthatthesuccessiveflipsofthecoinareindependentandeachflipresultsinaheadwithprobabilityp,whatistheprobabilitythatAendsupwithallthemoneyifhestartswithiunitsandBstartswithN−iunits?LetEdenotetheeventthatAendsupwithallthemoneywhenhestartswithiandBstartswithN−i,let𝑃𝑖=𝑃(𝐸).LetHdenotetheeventthatthefirstfliplandsonheadsNow,giventhatthefirstfliplandsonheads,thesituationafterthefirstbetisthatAhasi+1unitsandBhasN−(i+1),thusSimilarlyccciHEPpHEpPHPHEPHPHEPEPP|1|||1|iPHEP1|icPHEP•Let𝑞=1−𝑝withboundary•Solvetheequationsweget•Let𝑄𝑖denotetheprobabilitythatBwindsupwithallthemoney,similarly•Obviously1,...,2,111NiqPpPPiii1,00NPP21if21if/1/1pNippqpqPNii21if21if/1/1qNiNqqpqpQNiNi1iiQPP(•|F)isaprobability•Conditionalprobabilitiessatisfyallthepropertiesofordinaryprobabilities•(a)•(b)•(c)If𝐸𝑖,𝑖=1,2,…,aremutuallyexclusiveevents,then1|0FEP1|FSP11||FEPFEPiiSummary•Conditionalprobability•Bayes’sformula•Independentevents:EandFareindependentifandonlyif)()()|(FPEFPFEPFPEPEFPniiijjjjFPFEPFPFEPEPEFPEFP1|||