Oracle数据湖集成方案

Copyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|CON-5465FillingyourDataLakewithpotabledatausingOracleDataIntegrationMikeMatthewsSeniorDirector,ProductManagementJayantMahtoSeniorProductManagerOctober2nd2017Confidential–OracleInternal/Restricted/HighlyRestrictedCopyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|SafeHarborStatementThefollowingisintendedtooutlineourgeneralproductdirection.Itisintendedforinformationpurposesonly,andmaynotbeincorporatedintoanycontract.Itisnotacommitmenttodeliveranymaterial,code,orfunctionality,andshouldnotberelieduponinmakingpurchasingdecisions.Thedevelopment,release,andtimingofanyfeaturesorfunctionalitydescribedforOracle’sproductsremainsatthesolediscretionofOracle.Confidential–OracleInternal/Restricted/HighlyRestricted3Copyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|Copyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|Develop&DeployIntegrate&ExtendOracleCloudPlatform4Analyze&PredictSecure&ManageInnovatewithaComprehensive,Open,IntegratedandHybridCloudPlatformthatisHighlyScalable,SecureandGloballyAvailablePublish&EngageCopyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|Copyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|DataManagementOracleCloudPlatform5Identity&SecurityApplicationDevelopmentContent&ExperienceSystemsManagementAnalyticsandBigDataHybridComprehensiveOpenIntegratedOracleDataCenterOraclePublicCloudYourDataCenterOracleCloudatCustomerEnterpriseIntegrationDataIntegrationBuiltonHighPerformantOracleCloudInfrastructureCopyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|Copyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|OracleCloudPlatformMomentum614,000+OracleCustomers$1.4BillionFY17OracleCloudRevenue(60%YoYGrowth)3,000+AppsintheMarketplace10PaaSCategorieswhereLeaderOracleisaIndustryCloudPlatformOracleCloudAnalystsAccordingtoPlatformCopyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|10/3/20177Copyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|ApplicationandDataIntegrationCompleteSimplifiedOpenDATAGOVERNANCEPROCESSAUTOMATIONSTREAMANALYTICSAPIMANAGEMENTAPPLICATIONINTEGRATIONDATAQUALITYBULKDATATRANSFORMATIONREALTIMEDATASTREAMINGANDDATAREPLICATIONOracleCloudPlatformforIntegration7Copyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|Copyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|Confidential–OracleInternal/Restricted/HighlyRestricted8DataLake…orDataSwamp?Copyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|KeySuccessFactorsforyourDataLakeConfidential–OracleInternal/Restricted/HighlyRestricted9Source:Knowledgent-https://knowledgent.com/whitepaper/design-successful-data-lake/TimelyaccesstodataFlexibilitytoextractandworkthedataasneededTrustinthequalityofthedataAbilitytofindandunderstandtheavailabledataCopyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|ReferenceArchitecturewithOracleDataIntegrationConfidential–OracleInternal/Restricted/HighlyRestrictedSaaSAppsOracleDataIntegrationYourDataLakeFastDataDeliveryAssuredDataTrustMetadataManagementEnterpriseDataQualityGoldenGateDataIntegratorCopyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|KeySuccessFactorsforyourDataLakeConfidential–OracleInternal/Restricted/HighlyRestricted11Source:Knowledgent-https://knowledgent.com/whitepaper/design-successful-data-lake/TimelyaccesstodataFlexibilitytoextractandworkthedataasneededTrustinthequalityofthedataAbilitytofindandunderstandtheavailabledataCopyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|WhyGoldenGate?12•TheSushiPrinciple–‘Dataisbestservedraw’•SomeofthebiggestdatalakesuseOracleGoldenGate’schangedatacapturecapabilityforreal-timeingestionfromsourcedatabases•Traditionalnormalization,aggregationandschematizationareskippedtosimplifydataflowsandimprovetimelinessandperformanceCopyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|GoldenGateforBigData(RunningOn-PremisesorCloud)ReplicatParametersBigDataPropertiesJAROracleGoldenGateforBigDataModular&PluggableArchitectureKafkaHiveHDFSHBASEFlumeCaptureTrailFilesNetworkFirewallCloudTrailFilesNativeJavaReplicatJMSMongo13ElasticCassandraJMSJDBCKinesisOSAHighPerformanceLowImpactandNon-IntrusiveFlexibleandHeterogeneousResilientandFIPSSecureBigDataandCloudCopyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|KeySuccessFactorsforyourDataLakeConfidential–OracleInternal/Restricted/HighlyRestricted14Source:Knowledgent-https://knowledgent.com/whitepaper/design-successful-data-lake/TimelyaccesstodataFlexibilitytoextractandworkthedataasneededTrustinthequalityofthedataAbilitytofindandunderstandtheavailabledataCopyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|15IntegrateAnyDataShape,Speed,Action,Volume&LocationContinuedFocusonOurVision:AnyDataLocationCloudInfrastructureAnyDataVolumeOpenSourcePlatformsAnyDataActionDataflow|PipesAnyDataSpeedLambdaAnyDataShapePolyglotCopyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|WhyOracleDataIntegrator?16•Toprovidetrueanalyticalflexibilityandaccuracy,somedatare-shapingmaybeneeded,especiallyasDataLakesareincreasinglyworkingwithMasterDataaswellasTransactionalData•ODI’sEL-Tarchitecturecanbeveryimportantwhenworkingwithlargevolumes•ThismaybedonereadingfromaDataLakeandwritingtoaDataWarehouse•ODIcanalsopushdowndatatransformationsintotheDataLakeCopyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|RawDataLayerSpeedLayerBatchLayerBigDataTransformationwithDataIntegrator17StreamingAnalyticsServingLayerRESTServicesVisualizationToolsReportingToolsDataMartsOracleDataIntegratorCaptureTrailRouteDeliverPumpGGSQOOPAPI/FileSQOOP+NativeLoadersDataIntegratorforBigDataBatchdataingestionwithSqoop,nativeloaders&OozieGeneratedatatransformationsinHive,Pig,Spark&SparkStreamingExtractdataintoexternalDBs,FilesorCloudBenefitsNoETLEnginenativeE-LTexecution,1000sofreferencesZeroFootprintdoesnotrequireanyOracleinstallonclusterLooselyCoupleddesigntimemeansyoucanreusemappinglogicinmanybigdatalanguagesCopyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|KeySuccessFactorsforyourDataLakeConfidential–OracleInternal/Restricted/HighlyRestricted18Source:Knowledgent-https://knowledgent.com/whitepaper/design-successful-data-lake/TimelyaccesstodataFlexibilitytoextractandworkthedataasneededTrustinthequalityofthedataAbilitytofindandunderstandtheavailabledataCopyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|SomedatacanonlybetrustedifitispreparedConfidential–OracleInternal/Restricted/HighlyRestricted19•DataConsumersneedaccesstoMasterDataaswellasTransactionalData•Relatingthetwocanbeverypowerful…•…butthisiswhererawdatacanbepoisonoustostrongbusinessanalytics•Incompleterecords•Hard-to-findDuplicates•Out-of-dateinformation•InconsistenciesindatacaptureCopyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|WhyOracleEnterpriseDataQuality?ProfileStandardizeMatchGovernQuicklyunderstanddatacontentDriveconformancetostandardsIdentify&mergeduplicatesMonitoreffectiveness&resolveproblemsCommonAccess/UIEnterpriseDQPlatformMarket-leadingusabilityforalltypesofdataUnparalleledtime-to-valueHighperformanceengineOut-of-the-boxglobalknowledge-baseFoundationforgovernanceprogram20Copyright©2016Oracleand/oritsaffiliates.Allrightsreserved.|OracleConfidential–Restricted21EDQ∙CollaborativeDataQualityGovernanceDataAnalysts•ImmediateDataInsight•ReusableDQServicesandRules•Transparent,self-documentingconfigurationDataStakeholders•ZeroTrainingEDQDashboard•ViewbyDataAsset,DataDomain,Rule•TrendAnalysisDataStewards•FlexibleDataReviewandRemediationoptionsinEDQCaseManagement•IntegratedwithDQRules•Fullyauditedwithcomments,attachments,history,reportsCopyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|KeySuccessFactorsforyourDataLakeConfidential–OracleInternal/Restricted/HighlyRestricted22Source:Knowledgent-https://knowledgent.com/whitepaper/design-successful-data-lake/TimelyaccesstodataFlexibilitytoextractandworkthedataasneededTrustinthequalityofthedataAbilitytofindandunderstandtheavailabledataCopyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|WhyMetadataManagementfortheDataLake?23WithoutMetadataManagementₓSilosofDataknownonlytotheirownersₓNodocumentationₓDuplicateeffortandinefficientusageₓNodatausageanalysisWithMetadataManagement:SearchableEnrichedwithdocumentationSharedknowledgeLineage/impactanalysisSemanticanalysisCopyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|ValueofEnterpriseMetadataManagement24ETLBIDashboardsAppETLETLHowwassalesfigurecalculated?HowdoIorganizemyDWandReportsWhatreportsusethemainframedata?SysAdminExecutiveBIDeveloperWheredidthisdatacomefrom?ApplicationUserWhatwillhappenifIchangethistable?CDCDataReservoirDataStewardCanItrustthesourcesofthiscustomerdata?ETLDeveloperSolvessignificantpainpointsforwidevarietyofbusinessconsumersandtechnicalstaffIwanttodesignanexperimenttomeasurethesuccessofasignuppage.WhatdatadoIhave?DataScientistGGWhichreportsusethiscustomerdata?EnterpriseArchitectCopyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|FindandUnderstandyourData25•MetadataManagement–horizontalandsemanticdatalineageforalldatasources•BusinessGlossary–simpletoolstocatalog,linkandcollaborateonbusinesstermsBusinessDataCatalogReporttoSourceLineageImpactAnalysisAudit,Versioning&DiffReportsSocial/CollaborationFeaturesAnnotationsandTaggingComprehensiveHarvesting3rdPartyBIMetadata3rdPartyETLMetadata3rdPartyDBMetadata3rdPartyModelingToolsBigDataMetadataMetadataStandardsCopyright©2016Oracleand/oritsaffiliates.Allrightsreserved.|WhatdoesPotableDatamean?OracleConfidential–Restricted26•QuicklyandEasilyConsumableandTrusted•YoucanuseGoldenGatetomakedatamorequicklyavailable,streamedinto(andthrough)theLakeusingCDC•YoucanuseODItomakethedataeasiertoconsume•Trustisnotonlyabout‘howgooditis’,butknowinghowgooditis(ornot),andwhereitcamefrom•YoucanuseEDQtoaddDataQualitydimensionstoyourdataasitisstreamedintotheLake…andtheanalyticstoolsyoualreadyusetotellyouhowgoodthedatais•YoucanuseOEMMtounderstandthedata,andwhereitcomesfromCopyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|Getasneakpeekatcutting-edgedataintegrationdesignsandreceiveafreegift!•Oracleisconstantlydevelopingnewsoftwareandfeaturesthatwillmakeyourworkeasier,andOracle'sUserExperienceteamwouldlovetogetyourfeedbackonnewdataintegrationdesigns.•Feedbacksessionswilltakeplaceatadateandtimeofyourownchoice.•Youcantakepartviawebconference,fromthecomfortandconvenienceofyourownoffice.•Ifyou’reinterested,pleasefilloutthe1-pageformathttp://bit.ly/2vIHlSguppercaseIlowercasel•Toshowourappreciation,wewillpostallparticipantstheirchoicefromawideselectionofthank-yougifts.27Copyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|Presen-tationson:28DataIntegrationProgramme–FOCUSONDOCLINKDemoStations:Hands-onLabs:OracleEnterpriseDataQualityOracleGoldenGateOracleDataIntegratorOracleDataIntegrationPlatformCloudOracleEnterpriseMetadataManagementOracleGoldenGateReal-TimeDataReplicationintheCloudHOL7715OracleEnterpriseDataQualityHOL7653ODIandOGGforBigDataHOL7708OracleDataIntegrationPlatformCloudHOL7673TheEXchangeIntegrationArea-MosconeWestTheEXchangeAnalytics&BigDataArea-MosconeWestTheEXchangeDataManagementArea-MosconeWestCopyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|29DataIntegrationProgramme–FOCUSONDOCLINKSunday,October1•LiftandShiftWorkloadstoCloudwithOracleDataIntegrationPlatformCloud[SUN6653]•DataMovementbetweenOn-Prem,FusionERPCloud,FusionHCMCloudandSalesforce[SUN7286]•AccelerateMigrationtoCloudInfrastructurewithDataIntegrationPlatform[SUN6896]Monday,October2•OracleDataIntegrationPlatformStrategyandRoadmap[CON6646]•FillingYourDataLakewithPotableData,UsingDataIntegration[CON5465]•GoldenGate:DeepDiveintoAutomatingOGGusingthenewMicroservices[CON6569]•OracleDataIntegrationPlatform:FoundationforCloudIntegration[CON6650]•OracleDataIntegrationPlatformEmpowersEnterpriseGradeBigDataSolutions[CON6893]•OracleDataIntegrationPlatformCloudDeepDive[CON6651]•OracleGoldenGateCloudService:Real-TimeDataReplicationintheCloud[HOL7715]Tuesday,October3•OracleDataIntegratorProductUpdateandStrategy[CON6654]•OracleEnterpriseDataQuality:ProductOverviewandRoadmap[CON6656]•AccelerateCloudOn-BoardingUsingOracleGoldenGateCloudService[CON6894]•OracleEnterpriseDataQualityforAllTypesofData[HOL7653]•OracleDataIntegrationPlatform:aCornerstoneforBigData[CON6655]•GoldenGate:MAAandBestPracticesforOracleGoldenGateMicroservices[CON6570]•OracleGoldenGateProductUpdateandStrategy[CON6897]Wednesday,October4•APracticalPathtoEnterpriseDataGovernancewithOracleEnterpriseDataQuality[CON6657]•OracleDataIntegratorandOracleGoldenGateforBigData[HOL7708]•IntroductiontoOracleDataIntegrationPlatformCloud[HOL7673]•AnEnterpriseDatabus:GoldenGateintheCloudWorkingwithKafkaandSpark(CON6895]•GoldenGate:BestPractices&DeepDiveonOGG12.3MicroservicesatCloud[CON6568]•OracleGoldenGateforBigData[CON6898]•OracleDataIntegrationPlatformCloudServiceGovernanceEdition[CON6652]Copyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|ConnectwithOracleIntegration@OracleDIBlogs.oracle.com/DataIntegration/OracleDataIntegrationOracleDataIntegrationOracleFMW@OracleIntegrateBlogs.oracle.com/Integration/OracleSOACopyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|31StayInformedDuringandAfterOpenWorldTwitter:@OracleExadata,@OracleBigData,@InfrastructureFollow#CloudReadyLinkedIn:OracleITInfrastructure–OracleShowcasePageOracleBigData–OracleShowcasePageCopyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|32ConvergedInfrastructureForumTuesday,Oct3from6:30-9pmSFMOMARSVPRequired:https://www.oracle.com/goto/Openworld/CIEventOct3Copyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|