app layer

2:ApplicationLayerChapter2:Applicationlayer2.1Principlesofnetworkapplications2.2WebandHTTP2.3FTP2.4ElectronicMailSMTP,POP3,IMAP2.5DNS2.6P2Pfilesharing2.7SocketprogrammingwithTCP2.8SocketprogrammingwithUDP2.9BuildingaWebserver2:ApplicationLayerChapter2:ApplicationLayerOurgoals:Principlesofnetworkapplicationdesigntransport-layerservicemodelsclient-serverparadigmpeer-to-peerparadigmPopularprotocolsthroughcasestudiesHTTPFTPSMTP/POP3/IMAPDNSNetworkprogrammingsocketAPI2:ApplicationLayerSomenetworkappsE-mailWebInstantmessagingRemoteloginP2PfilesharingMulti-usernetworkgamesStreamingstoredvideoclipsInternettelephoneReal-timevideoconferenceMassiveparallelcomputingNextgeneration:Thenetworkwillbethecomputer.MostApplicationswillrunoverthenetwork.LocalPCminimalyrequiredExample:Googlespreadsheet2:ApplicationLayerCreatinganetworkappWriteprogramsthatrunondifferentendsystemsandcommunicateoveranetwork.e.g.,Web:Webserversoftwarecommunicateswithbrowsersoftwarelittlesoftwarewrittenfordevicesinnetworkcorenetworkcoredevicesdonotrunuserapplicationcodeapplicationonendsystemsallowsforrapidappdevelopment,propagation2:ApplicationLayerChapter2:Applicationlayer2.1Principlesofnetworkapplications2.2WebandHTTP2.3FTP2.4ElectronicMailSMTP,POP3,IMAP2.5DNS2.6P2Pfilesharing2.7SocketprogrammingwithTCP2.8SocketprogrammingwithUDP2.9BuildingaWebserver2:ApplicationLayerApplicationarchitecturesClient-serverPeer-to-peer(P2P)Hybridofclient-serverandP2P2:ApplicationLayerClient-serverarchitectureserver:always-onhostpermanentIPaddressserverfarmsforscalingclients:communicatewithservermaybeintermittentlyconnectedmayhavedynamicIPaddressesdonotcommunicatedirectlywitheachother2:ApplicationLayerPureP2Parchitecturenoalways-onserverarbitraryendsystemsdirectlycommunicatepeersareintermittentlyconnectedandchangeIPaddressesexample:GnutellaHighlyscalablebutdifficulttomanage2:ApplicationLayerHybridofclient-serverandP2PSkypeInternettelephonyappFindingaddressofremoteparty:centralizedserver(s)Client-clientconnectionisdirect(notthroughserver)InstantmessagingChattingbetweentwousersisP2PPresencedetection/locationcentralized:UserregistersitsIPaddresswithcentralserverwhenitcomesonlineUsercontactscentralservertofindIPaddressesofbuddies2:ApplicationLayerCaseStudy:NapsterVsGnutellaAnyproblemwiththisarchitecture?2:ApplicationLayerProcessescommunicatingProcess:programrunningwithinahost.withinsamehost,twoprocessescommunicateusinginter-processcommunication(definedbyOS).processesindifferenthostscommunicatebyexchangingmessagesClientprocess:processthatinitiatescommunicationServerprocess:processthatwaitstobecontactedNote:applicationswithP2Parchitectureshaveclientprocesses&serverprocesses2:ApplicationLayerSocketsprocesssends/receivesmessagesto/fromitssocketsocketanalogoustodoorsendingprocessshovesmessageoutdoorsendingprocessreliesontransportinfrastructureonothersideofdoorwhichbringsmessagetosocketatreceivingprocessInternetcontrolledbyOScontrolledbyappdeveloperAPI:(1)choiceoftransportprotocol;(2)abilitytofixafewparameters(lotsmoreonthislater)sockethostorserversockethostorserver2:ApplicationLayerAddressingprocessestoreceivemessages,processmusthaveidentifierhostdevicehasunique32-bitIPaddressQ:doesIPaddressofhostonwhichprocessrunssufficeforidentifyingtheprocess?2:ApplicationLayerAddressingprocessestoreceivemessages,processmusthaveidentifierhostdevicehasunique32-bitIPaddressQ:doesIPaddressofhostonwhichprocessrunssufficeforidentifyingtheprocess?Answer:NO,manyprocessescanberunningonsamehostidentifierincludesbothIPaddressandportnumbersassociatedwithprocessonhost.Exampleportnumbers:HTTPserver:80Mailserver:25tosendHTTPmessagetogaia.cs.umass.eduwebserver:IPaddress:128.119.245.12Portnumber:80moreshortly…2:ApplicationLayerMessageFormat:App-layerprotocoldefinesTypesofmessagesexchanged,e.g.,request,responseMessagesyntax:whatfieldsinmessages&howfieldsaredelineatedMessagesemanticsmeaningofinformationinfieldsRulesforwhenandhowprocessessend&respondtomessagesPublic-domainprotocols:definedinRFCsallowsforinteroperabilitye.g.,HTTP,SMTPProprietaryprotocols:e.g.,KaZaA2:ApplicationLayerRequirementsforMessageTransport:Datalosssomeapps(e.g.,audio)cantoleratesomelossotherapps(e.g.,filetransfer,telnet)require100%reliabledatatransferTimingsomeapps(e.g.,Internettelephony,interactivegames)requirelowdelaytobe“effective”Bandwidthsomeapps(e.g.,multimedia)requireminimumamountofbandwidthtobe“effective”otherapps(“elasticapps”)makeuseofwhateverbandwidththeygetWhyisbandwidthdifferentfromtimingconstraints?2:ApplicationLayerTransportservicerequirementsofcommonappsApplicationfiletransfere-mailWebdocumentsreal-timeaudio/videostoredaudio/videointeractivegamesinstantmessagingDatalossnolossnolossnolossloss-tolerantloss-tolerantloss-tolerantnolossBandwidthelasticelasticelasticaudio:5kbps-1Mbpsvideo:10kbps-5MbpssameasabovefewkbpsupelasticTimeSensitivenononoyes,100’smsecyes,fewsecsyes,100’smsecyesandno2:ApplicationLayerInternettransportprotocolsservicesTCPservice:connection-oriented:setuprequiredbetweenclientandserverprocessesreliabletransportbetweensendingandreceivingprocessflowcontrol:senderwon’toverwhelmreceivercongestioncontrol:throttlesenderwhennetworkoverloadeddoesnotprovide:timing,minimumbandwidthguaranteesUDPservice:unreliabledatatransferbetweensendingandreceivingprocessdoesnotprovide:connectionsetup,reliability,flowcontrol,congestioncontrol,timing,orbandwidthguaranteeQ:whybother?WhyisthereaUDP?2:ApplicationLayerInternetapps:application,transportprotocolsApplicatione-mailremoteterminalaccessWebfiletransferstreamingmultimediaInternettelephonyApplicationlayerprotocolSMTP[RFC2821]Telnet[RFC854]HTTP[RFC2616]FTP[RFC959]proprietary(e.g.RealNetworks)proprietary(e.g.,Vonage,Dialpad)UnderlyingtransportprotocolTCPTCPTCPTCPTCPorUDPtypicallyUDP2:ApplicationLayerChapter2:Applicationlayer2.1Principlesofnetworkapplicationsapparchitecturesapprequirements2.2WebandHTTP2.4ElectronicMailSMTP,POP3,IMAP2.5DNS2.6P2Pfilesharing2.7SocketprogrammingwithTCP2.8SocketprogrammingwithUDP2.9BuildingaWebserver2:ApplicationLayerWebandHTTPFirstsomejargonWebpageconsistsofobjectsObjectcanbeHTMLfile,JPEGimage,Javaapplet,audiofile,…WebpageconsistsofbaseHTML-filewhichincludesseveralreferencedobjectsEachobjectisaddressablebyaURLExampleURL:www.someschool.edu/someDept/pic.gifhostnamepathname2:ApplicationLayerHTTPoverviewHTTP:hypertexttransferprotocolWeb’sapplicationlayerprotocolclient/servermodelclient:browserthatrequests,receives,“displays”Webobjectsserver:WebserversendsobjectsinresponsetorequestsHTTP1.0:RFC1945HTTP1.1:RFC2068PCrunningExplorerServerrunningApacheWebserverMacrunningNavigatorHTTPrequestHTTPrequestHTTPresponseHTTPresponse2:ApplicationLayerHTTPoverview(continued)UsesTCP:clientinitiatesTCPconnection(createssocket)toserver,port80serveracceptsTCPconnectionfromclientHTTPmessages(application-layerprotocolmessages)exchangedbetweenbrowser(HTTPclient)andWebserver(HTTPserver)TCPconnectionclosedHTTPis“stateless”servermaintainsnoinformationaboutpastclientrequestsProtocolsthatmaintain“state”arecomplex!pasthistory(state)mustbemaintainedifserver/clientcrashes,theirviewsof“state”maybeinconsistent,mustbereconciledaside2:ApplicationLayerHTTPconnectionsNonpersistentHTTPAtmostoneobjectissentoveraTCPconnection.HTTP/1.0usesnonpersistentHTTPPersistentHTTPMultipleobjectscanbesentoversingleTCPconnectionbetweenclientandserver.HTTP/1.1usespersistentconnectionsindefaultmode2:ApplicationLayerNonpersistentHTTPSupposeuserentersURLwww.someSchool.edu/someDepartment/home.index1a.HTTPclientinitiatesTCPconnectiontoHTTPserver(process)atwww.someSchool.eduonport802.HTTPclientsendsHTTPrequestmessage(containingURL)intoTCPconnectionsocket.MessageindicatesthatclientwantsobjectsomeDepartment/home.index1b.HTTPserverathostwww.someSchool.eduwaitingforTCPconnectionatport80.“accepts”connection,notifyingclient3.HTTPserverreceivesrequestmessage,formsresponsemessagecontainingrequestedobject,andsendsmessageintoitssockettime(containstext,referencesto10jpegimages)2:ApplicationLayerNonpersistentHTTP(cont.)5.HTTPclientreceivesresponsemessagecontaininghtmlfile,displayshtml.Parsinghtmlfile,finds10referencedjpegobjects6.Steps1-5repeatedforeachof10jpegobjects4.HTTPserverclosesTCPconnection.time2:ApplicationLayerNon-PersistentHTTP:ResponsetimeRoundTripTime(RTT)=timetosendasmallpackettotravelfromclienttoserverandback.Responsetime:oneRTTtoinitiateTCPconnectiononeRTTforHTTPrequest andfirstfewbytesofHTTPresponsetoreturnfiletransmissiontimetotal=2RTT+<filetransmittime>timetotransmitfileinitiateTCPconnectionRTTrequestfileRTTfilereceivedtimetimeWhatdidweneglecthere?2:ApplicationLayerPersistentHTTPNonpersistentHTTPissues:requires2RTTsperobjectOSoverheadforeachTCPconnectionbrowsersoftenopenparallelTCPconnectionstofetchreferencedobjectsPersistentHTTPserverleavesconnectionopenaftersendingresponsesubsequentHTTPmessagesbetweensameclient/serversentoveropenconnectionPersistentwithoutpipelining:clientissuesnewrequestonlywhenpreviousresponsehasbeenreceivedoneRTTforeachreferencedobjectPersistentwithpipelining:defaultinHTTP/1.1clientsendsrequestsassoonasitencountersareferencedobjectaslittleasoneRTTforallthereferencedobjects2:ApplicationLayerHTTPrequestmessagetwotypesofHTTPmessages:request,responseHTTPrequestmessage:ASCII(human-readableformat)GET/somedir/page.htmlHTTP/1.1Host:www.someschool.eduUser-agent:Mozilla/4.0Connection:closeAccept-language:fr(extracarriagereturn,linefeed)requestline(GET,POST,HEADcommands)headerlinesCarriagereturn,linefeedindicatesendofmessage2:ApplicationLayerHTTPrequestmessage:generalformat2:ApplicationLayerUploadingforminputPostmethod:WebpageoftenincludesforminputInputisuploadedtoserverinentitybodyURLmethod:UsesGETmethodInputisuploadedinURLfieldofrequestline:www.somesite.com/animalsearch?monkeys&banana2:ApplicationLayerMethodtypesHTTP/1.0GETPOSTHEADasksservertoleaverequestedobjectoutofresponseHTTP/1.1GET,POST,HEADPUTuploadsfileinentitybodytopathspecifiedinURLfieldDELETEdeletesfilespecifiedintheURLfield2:ApplicationLayerHTTPresponsemessageHTTP/1.1200OKConnectioncloseDate:Thu,06Aug199812:00:15GMTServer:Apache/1.3.0(Unix)Last-Modified:Mon,22Jun1998…...Content-Length:6821Content-Type:text/htmldatadatadatadatadata...statusline(protocolstatuscodestatusphrase)headerlinesdata,e.g.,requestedHTMLfile2:ApplicationLayerHTTPresponsestatuscodes200OKrequestsucceeded,requestedobjectlaterinthismessage301MovedPermanentlyrequestedobjectmoved,newlocationspecifiedlaterinthismessage(Location:)400BadRequestrequestmessagenotunderstoodbyserver404NotFoundrequesteddocumentnotfoundonthisserver505HTTPVersionNotSupportedInfirstlineinserver->clientresponsemessage.Afewsamplecodes:2:ApplicationLayerTryingoutHTTP(clientside)foryourself1.TelnettoyourfavoriteWebserver:OpensTCPconnectiontoport80(defaultHTTPserverport)atcis.poly.edu.Anythingtypedinsenttoport80atcis.poly.edutelnetcis.poly.edu802.TypeinaGETHTTPrequest:GET/~ross/HTTP/1.1Host:cis.poly.eduBytypingthisin(hitcarriagereturntwice),yousendthisminimal(butcomplete)GETrequesttoHTTPserver3.LookatresponsemessagesentbyHTTPserver!Shallwetry?2:ApplicationLayerChapter2:Applicationlayer2.1Principlesofnetworkapplicationsapparchitecturesapprequirements2.2WebandHTTP2.4ElectronicMailSMTP,POP3,IMAP2.5DNS2.6P2Pfilesharing2.7SocketprogrammingwithTCP2.8SocketprogrammingwithUDP2.9BuildingaWebserver2:ApplicationLayerUser-serverstate:cookiesManymajorWebsitesusecookiesFourcomponents:1)cookieheaderlineofHTTPresponsemessage2)cookieheaderlineinHTTPrequestmessage3)cookiefilekeptonuser’shost,managedbyuser’sbrowser4)back-enddatabaseatWebsiteExample:SusanaccessInternetalwaysfromsamePCShevisitsaspecifice-commercesiteforfirsttimeWheninitialHTTPrequestsarrivesatsite,sitecreatesauniqueIDandcreatesanentryinbackenddatabaseforID2:ApplicationLayerCookies:keeping“state”(cont.)servercreatesID1678foruserentryinbackenddatabaseaccessaccessoneweeklater:clientserverusualhttprequestmsgusualhttpresponse+Set-cookie:1678cookie-specificactioncookie-spectificactionCookiefileebay:87342:ApplicationLayerCookies(continued)Whatcookiescanbring:authorizationshoppingcartsrecommendationsusersessionstate(Webe-mail)Cookiesandprivacy:cookiespermitsitestolearnalotaboutyouyoumaysupplynameande-mailtositessearchenginesuseredirection&cookiestolearnyetmoreadvertisingcompaniesobtaininfoacrosssitesaside2:ApplicationLayerWebcaches(proxyserver)usersetsbrowser:WebaccessesviacachebrowsersendsallHTTPrequeststocacheobjectincache:cachereturnsobjectelsecacherequestsobjectfromoriginserver,thenreturnsobjecttoclientGoal:satisfyclientrequestwithoutinvolvingoriginserverclientProxyserverclientHTTPrequestHTTPrequestHTTPresponseHTTPresponseHTTPrequestHTTPresponseoriginserveroriginserver2:ApplicationLayerMoreaboutWebcachingCacheactsasbothclientandserverTypicallycacheisinstalledbyISP(university,company,residentialISP)WhyWebcaching?Reduceresponsetimeforclientrequest.Reducetrafficonaninstitution’saccesslink.Internetdensewithcachesenables“poor”contentproviderstoeffectivelydelivercontent(butsodoesP2Pfilesharing)2:ApplicationLayerCachingexampleAssumptionsaverageobjectsize=100,000bitsavg.requestratefrominstitution’sbrowserstooriginservers=15/secdelayfrominstitutionalroutertoanyoriginserverandbacktorouter=2secConsequencesutilizationonLAN=15%utilizationonaccesslink=100%totaldelay=Internetdelay+accessdelay+LANdelay=2sec+minutes+millisecondsoriginserverspublicInternetinstitutionalnetwork10MbpsLAN1.5Mbpsaccesslinkinstitutionalcache2:ApplicationLayerCachingexample(cont)Possiblesolutionincreasebandwidthofaccesslinkto,say,10MbpsConsequencesutilizationonLAN=15%utilizationonaccesslink=15%Totaldelay=Internetdelay+accessdelay+LANdelay=2sec+msecs+msecsoftenacostlyupgradeoriginserverspublicInternetinstitutionalnetwork10MbpsLAN10Mbpsaccesslinkinstitutionalcache2:ApplicationLayerCachingexample(cont)Installcachesupposehitrateis.4Consequence40%requestswillbesatisfiedalmostimmediately60%requestssatisfiedbyoriginserverutilizationofaccesslinkreducedto60%,resultinginnegligibledelays(say10msec)totalavgdelay=Internetdelay+accessdelay+LANdelay=.6*(2.01)secs+.4*milliseconds<1.4secsoriginserverspublicInternetinstitutionalnetwork10MbpsLAN1.5Mbpsaccesslinkinstitutionalcache2:ApplicationLayerConditionalGETGoal:don’tsendobjectifcachehasup-to-datecachedversioncache:specifydateofcachedcopyinHTTPrequestIf-modified-since:<date>server:responsecontainsnoobjectifcachedcopyisup-to-date:HTTP/1.0304NotModifiedcacheserverHTTPrequestmsgIf-modified-since:<date>objectnotmodifiedHTTPrequestmsgIf-modified-since:<date>HTTPresponseHTTP/1.0200OK<data>objectmodifiedHTTPresponseHTTP/1.0304NotModified2:ApplicationLayerQuestions?2:ApplicationLayerChapter2:Applicationlayer2.1Principlesofnetworkapplications2.2WebandHTTP2.3FTP2.4ElectronicMailSMTP,POP3,IMAP2.5DNS2.6P2Pfilesharing2.7SocketprogrammingwithTCP2.8SocketprogrammingwithUDP2.9BuildingaWebserver2:ApplicationLayerFTP:thefiletransferprotocoltransferfileto/fromremotehostclient/servermodelclient:sidethatinitiatestransfer(eitherto/fromremote)server:remotehostftp:RFC959ftpserver:port21filetransferremotefilesystemuserathostFTPserverFTPuserinterfaceFTPclientlocalfilesystem2:ApplicationLayerFTP:separatecontrol,dataconnectionsFTPclientcontactsFTPserveratport21,specifyingTCPastransportprotocolClientobtainsauthorizationClientbrowsesremotedirectorybysendingcontrolcommandsWhenserverreceivesacommand,opensTCPdataconnectiontoclientAftertransferringonefile,serverclosesconnection.ServeropensasecondTCPdataconnectiontotransferanotherfile.Controlconnection:“outofband”FTPservermaintains“state”:currentdirectory,earlierauthenticationFTPclientFTPserverTCPcontrolconnectionport21TCPdataconnectionport202:ApplicationLayerFTPTimeline2:ApplicationLayerFTPcommands,responsesSamplecommands:sentasASCIItextovercontrolchannelUSERusernamePASSpasswordLISTreturnlistoffileincurrentdirectoryRETRfilenameretrieves(gets)fileSTORfilenamestores(puts)fileontoremotehostSamplereturncodesstatuscodeandphrase(asinHTTP)331UsernameOK,passwordrequired125dataconnectionalreadyopen;transferstarting425Can’topendataconnection452Errorwritingfile2:ApplicationLayerChapter2:Applicationlayer2.1Principlesofnetworkapplications2.2WebandHTTP2.3FTP2.4ElectronicMailSMTP,POP3,IMAP2.5DNS2.6P2Pfilesharing2.7SocketprogrammingwithTCP2.8SocketprogrammingwithUDP2.9BuildingaWebserver2:ApplicationLayerElectronicMailThreemajorcomponents:useragentsmailserverssimplemailtransferprotocol:SMTPUserAgenta.k.a.“mailreader”composing,editing,readingmailmessagese.g.,Eudora,Outlook,elm,NetscapeMessengeroutgoing,incomingmessagesstoredonserverusermailboxoutgoingmessagequeuemailserverSMTPSMTPSMTP2:ApplicationLayerElectronicMail:mailserversMailServersmailboxcontainsincomingmessagesforusermessagequeueofoutgoing(tobesent)mailmessagesSMTPprotocolbetweenmailserverstosendemailmessagesclient:sendingmailserver“server”:receivingmailservermailserverSMTPSMTPSMTP2:ApplicationLayerElectronicMail:SMTP[RFC2821]usesTCPonport25toreliablytransferemaildirecttransfer:sendingservertoreceivingserverthreephasesoftransferhandshaking(greeting)transferofmessagesClosurecommand/responseinteractioncommands:ASCIItextresponse:statuscodeandphrase2:ApplicationLayerScenario:AliceEmailsBob1)AliceusesUAtocomposemessageand“to”bob@someschool.edu2)Alice’sUAsendsmessagetohermailserver;messageplacedinmessagequeue3)ClientsideofSMTPopensTCPconnectionwithBob’smailserver4)SMTPclientsendsAlice’smessageovertheTCPconnection5)Bob’smailserverplacesthemessageinBob’smailbox6)Bobinvokeshisuseragenttoreadmessage1234562:ApplicationLayerSMTPCommandstosendemailTelenetintoport25HELOhostnameMAILFROM:RCPTTORCPTTO…DATA…text….QUITYoucantrydoingthisyourself2:ApplicationLayerTrySMTPinteractionforyourself:telnetservername25see220replyfromserverenterHELO,MAILFROM,RCPTTO,DATA,QUITcommandsaboveletsyousendemailwithoutusingemailclient(reader)2:ApplicationLayerSMTP:finalwordsSMTPusespersistentconnectionsSMTPrequiresmessage(header&body)tobein7-bitASCIISMTPserverusesCRLF.CRLFtodetermineendofmessageComparisonwithHTTP:HTTP:pullSMTP:pushbothhaveASCIIcommand/responseinteraction,statuscodesHTTP:eachobjectencapsulatedinitsownresponsemsgSMTP:multipleobjectssentinmultipartmsg2:ApplicationLayerMailmessageformatSMTP:protocolforexchangingemailmsgsRFC822:standardfortextmessageformat:headerlines,e.g.,To:From:Subject:differentfromSMTPcommands!bodythe“message”,ASCIIcharactersonlyheaderbodyblankline2:ApplicationLayerMessageformat:multimediaextensionsMIME:multimediamailextension,RFC2045,2056additionallinesinmsgheaderdeclareMIMEcontenttypeThinkofimageattachmentswithyouremailmultimediadatatype,subtype,parameterdeclarationmethodusedtoencodedataMIMEversionencodeddataFrom:alice@crepes.frTo:bob@hamburger.eduSubject:Pictureofyummycrepe.MIME-Version:1.0Content-Transfer-Encoding:base64Content-Type:image/jpegbase64encodeddata....................................base64encodeddata2:ApplicationLayerMailaccessprotocolsSMTP:delivery/storagetoreceiver’sserverMailaccessprotocol:retrievalfromserverPOP:PostOfficeProtocol[RFC1939]authorization(agent<-->server)anddownloadIMAP:InternetMailAccessProtocol[RFC1730]morefeatures(morecomplex)manipulationofstoredmsgsonserverHTTP:Hotmail,Yahoo!Mail,etc.SMTPaccessprotocolreceiver’smailserverWhat’stheDifference?sender’smailserverSMTP2:ApplicationLayerPOP3protocolauthorizationphaseclientcommands:user:declareusernamepass:passwordserverresponses+OK-ERRtransactionphase,client:list:listmessagenumbersretr:retrievemessagebynumberdele:deletequitC:listS:1498S:2912S:.C:retr1S:<message1contents>S:.C:dele1C:retr2S:<message1contents>S:.C:dele2C:quitS:+OKPOP3serversigningoffS:+OKPOP3serverreadyC:userbobS:+OKC:passhungryS:+OKusersuccessfullyloggedon2:ApplicationLayerPOP3(more)andIMAPMoreaboutPOP3Previousexampleuses“downloadanddelete”mode.Bobca