Moment_Generating_Function

MomentGeneratingFunctionStatistics110Summer2006Copyrightc©2006byMarkE.IrwinMomentsRevisitedSofarI’vereallyonlytalkedaboutthefirsttwomoments.Letsdefinewhatismeantbymomentsmoreprecisely.Definition.TherthmomentofarandomvariableXisE[Xr],assumingthattheexpectationexists.Sothemeanofadistributionisitsfirstmoment.Definition.ThercentralmomentofarandomvariableXisE[(X−E[X])r],assumingthattheexpectationexists.Thusthevarianceisthe2ndcentralmomentofdistribution.The1stcentralmomentusuallyisn’tdiscussedasitsalways0.The3rdcentralmomentisknownastheskewnessofadistributionandisusedasameasureofasymmetry.MomentsRevisited1Ifadistributionissymmetricaboutitsmean(f(µ−x)=f(µ+x)),theskewnesswillbe0.Similarlyiftheskewnessisnon-zero,thedistributionisasymmetric.Howeveritispossibletohaveasymmetricdistributionwithskewness=0.Examplesofsymmetricdistributionarenormals,Beta(a,a),Bin(n,p=0.5).ExampleofasymmetricdistributionsareDistributionSkewnessBin(n,p)np(1−p)(1−2p)Pois(λ)λExp(λ)2λBeta(a,b)UglyformulaThe4thcentralmomentisknownasthekurtosis.Itcanbeusedasameasureofhowheavythetailsareforadistribution.Thekurtosisforanormalis3σ4.MomentsRevisited2Notethatthesemeasuresareoftenstandardizedasintheirrawformtheydependonthestandarddeviation.Theorem.IftherthmomentofaRVisexists,thenthesthmomentexistsforalls<r.Alsothesthcentralmomentexistsforalls≤r.Soyoucan’thaveadistributionthathasafinitemean,aninfinitevariance,andafiniteskewness.Proof.Postponedtilllater.2Whyaremomentsuseful?TheycanbeinvolvedincalculatingmeansandvariancesoftransformedRVsorothersummariesofRVs.Example:WhatarethemeanandvarianceofA=piR2E[A]=piE[R2]Var(A)=pi2Var(R2)=pi2(E[R4]−(E[R2])2)SoweneedE[R4]inadditiontoE[R]andE[R2].MomentsRevisited3Example:WhatistheskewnessofX?E[(X−µ)3]=E[X3−3µX2+3µ2X−µ3]=E[X3]−3µE[X2]+2µ3soE[X],E[X2],andE[X3]areneededtocalculatetheskewness.MomentsRevisited4MomentGeneratingFunctionDefinition.TheMomentGeneratingFunction(MGF)ofarandomvariableX,isMX(t)=E[etX]iftheexpectationisdefined.MX(t)=∑xetxpX(x)(Discrete)MX(t)=∫XetxfX(x)dx(Continuous)WhethertheMGFisdefineddependsonthedistributionandthechoiceoft.Forexample,theMX(t)isdefinedforalltifXisnormal,definedfornotifXisCauchy,andfort<λifX∼Exp(λ).Forthosethathavedonesomeanalysis,forthecontinuouscase,themomentgeneratingfunctionisrelatedtotheLaplacetransformofthedensityfunction.Manyoftheresultsaboutitcomefromthattheory.MomentGeneratingFunction5WhyshouldwecareabouttheMGF?•Tocalculatemoments.ItmaybeeasiertoworkwiththeMGFthantodirectlycalculateE[Xr].•Todeterminedistributionsoffunctionsofrandomvariables.•Relatedtothis,approximatingdistributions.Forexamplecanuseittoshowthatasnincreases,theBin(n,p)“approaches”anormaldistribution.ThefollowingtheoremsjustifytheseusesoftheMGF.MomentGeneratingFunction6Theorem.IfMX(t)ofaRVXisfiniteinanopenitervalcontaining0,thenithasderivativesofallordersandM(r)X(t)=E[XretX]M(r)X(0)=E[Xr]Proof.M(1)X(t)=ddt∫∞−∞etxfX(x)dx=∫∞−∞(ddtetx)fX(x)dx=∫∞−∞xetxfX(x)dx=E[XetX]MomentGeneratingFunction7M(2)X(t)=ddtM(1)X=∫∞−∞x(ddtetx)fX(x)dx=∫∞−∞x2etxfX(x)dx=E[X2etX]Therestcanbeshownbyinduction.Thesecondpartofthetheoremfollowsfrome0=1.2AnotherwaytoseethisresultisduetotheTaylorseriesexpansionofey=1+y+y22!+y33!+...,whichgivesMomentGeneratingFunction8MX(t)=E[1+Xt+X2t22!+X3t33!+...]=1+E[X]t+E[X2]t22!+E[X3]t33!+...ExampleMGFs:•X∼U(a,b)MX(t)=∫baetxb−adx=ebt−eat(b−a)tMomentGeneratingFunction9•X∼Exp(λ)MX(t)=∫∞0etxλe−λxdx=∫∞0λe−(λ−t)xdx=λλ−tNotethatthisintegralisonlydefinedwhent<λ•X∼Geo(p),(q=1−p)MX(t)=∞∑x=1etxpqx−1=pet∞∑x=1(et)x−1qx−1=pet1−qet•X∼Pois(λ)MX(t)=∞∑x=0etxe−λλxx!=e−λ∞∑x=0(etλ)xx!=eλ(et−1)MomentGeneratingFunction10ExamplesofusingtheMGFtocalculatemoments•X∼Exp(λ)M(1)(t)=λ(λ−t)2;E[X]=1λM(2)(t)=2λ(λ−t)3;E[X2]=2λ2M(r)(t)=Γ(r+1)λ(λ−t)r+1;E[Xr]=Γ(r+1)λrMomentGeneratingFunction11•X∼Geo(p),(q=1−p)M(1)(t)=pet1−qet+pqe2t(1−qet)2;E[X]=1pM(2)(t)=pet1−qet+3pqe2t(1−qet)2+2pq2e3t(1−qet)3;E[X2]=5−6p+2p2pTheorem.IfY=a+bXthenMY(t)=eatMX(bt)Proof.MY(t)=E[etY]=E[eat+btX]=eatE[e(bt)X]=eatMX(bt)2MomentGeneratingFunction12Forexample,thisresultcanbeusedtoverifytheresultthatE[a+bX]=a+bE[X]asM(1)Y(t)=aeatMX(bt)+beatM(1)X(bt)M(1)Y(0)=aMX(0)+bM(1)X(0)=a+bE[X]Theorem.IfXandYareindependentRVswithMGFsMXandMYandZ=X+Y,thenMZ(t)=MX(t)MY(t)onthecommonintervalwherebothMGFsexist.Proof.MZ(t)=E[etZ]=E[et(X+Y)]=E[etXetY]=E[etX]E[etY]=MX(t)MY(t)2Byinduction,thisresultcanbeextendedtosumsofmanyindependentRVs.MomentGeneratingFunction13OneparticularuseofthisresultisthatitcangiveaneasyapproachtoshowingwhatthedistributionofasumofRVsiswithouthavingthecalculatetheconvolutionofthedensities.Butfirstweneedonemoreresult.Theorem.[Uniquenesstheorem]IftheMGFofXexistsfortinanopenintervalcontaining0,thenituniquelydeterminestheCDF.i.enotwodifferentdistributionscanhavethesamevaluesfortheMGFsonanintervalcontaining0.Proof.Postponed2Example:LetX1,X2,...,XnbeiidExp(λ).WhatisthedistributionofS=∑XiMS(t)=n∏i=1λλ−t=(λλ−t)nMomentGeneratingFunction14Notethatthisisn’ttheformoftheMGFforanexponential,sothesumisn’texponential.AsshowninExampleBonpage145,theMGFofaGamma(α,λ)isM(t)=(λλ−t)αsoS∼Gamma(n,λ)Thisapproachalsoleadstoaneasyproofthatthesumofindependentnormalsisalsonormal.ThemomentgeneratingfunctionforN(µ,σ2)RVisM(t)=eµt+σ2t2/2.SoifXiiid∼N(µi,σ2i),i=1,...,n,thenMPXi=n∏i=1eµit+σ2it2/2=exp(t∑µi+t2/2∑σ2i)whichisthemomentgeneratingfunctionofaN(∑µi,∑σ2i)RV.Thereisoneimportantthingwiththisapproach.WemustbeabletoidentifywhatMGFgoeswitheachdensityorPMF.MomentGeneratingFunction15Forexample,letX∼Gamma(α,λ)andY∼Gamma(β,µ)beindependent.ThentheMGFofZ∼X+YisMZ(t)=(λλ−t)α(µµ−t)βThisisnottheMGFofagammadistributionunlessλ=µ.InfactI’mnotquitesurewhatthedensitylookslikebeyondfZ(z)=∫∞0λαxα−1e−λxΓ(α)µβ(z−x)β−1e−µ(z−x)Γ(β)dxYoucansometimesusetablesofLaplacetransformsordoingsomecomplicatedcomplexvariableintegrationtoinverttheMGFtodeterminethedensityorPMF.Whilewecan’tgetthedensityeasilyinthiscase,wecanstillusetheMGFtogetthemomentsofthisdistribution.MomentGeneratingFunction16Itisalsopossibletoworkwithmorecomplicatedsituationsdescribedbyhierarchicalmodels.SupposethattheMGFsforX(MX(t))andY|X=x(MY|X(t))areknown.ThenthemarginalMGFofYisMY(t)=E[etY]=E[E[etY|X]]=E[MY|X(t)]Forexample,thiscouldbeusedtogettheMGFoftheBeta-Binomialmodel.AnothersituationwherethisisusefuliswitharandomsumsmodelwhereS=N∑i=1XiandNisrandom.ThentheMGFofSisgivenbyMS(t)=E[E[etS|N]]=E[(MX(t))N]=E[eNlogMX(t)]=MN(logMX(t))MomentGeneratingFunction17AnexampleofthismodelisS|N=n∼Bin(n,p)(=N∑i=1Bern(p))N∼Pois(λ)MX(t)=1−p+petMN(t)=eλ(et−1)SothemomentgeneratingfunctionforSisMS(t)=MN(logMX(t))=exp(λ(elog(1−p+pet)−1))=exp(λ(1−p+pet)e−1)MomentGeneratingFunction18AnotherexampleisthecompoundPoissonmodeldiscussedinthetext.TheMGFisalsodefinedforjointdistributions.IthastheformMX,Y(s,t)=E[esX+tY]Ithassimilarpropertiesastheunivariatecase.ForexamplethemixedmomentsaregivenbyE[XnYm]=∂n+m∂xn∂ymMX,Y(0,0)ThemarginalMGFscanbedetermineddirectlyfromthejointMGFasMX(s)=MX,Y(s,0);MY(t)=MX,Y(0,t)AlsoXandYandindependentifandonlyifMX,Y(s,t)=MX(s)MY(t).ThisrelatestotheideathatifXandYareindependentsoareg(X)=esXandh(Y)=etY.MomentGeneratingFunction19