flink源码浅析和运行机理梳理

上传者: luyunlong_it | 上传时间: 2022-05-17 19:09:07 | 文件大小: 4.12MB | 文件类型: ZIP
基于对flink的学习,和对flink源码的简单梳理,从flink on yarn的角度,介绍flink从启动,到提交yarn任务,以及完成整个过程涉及到的进程交互和内存管理等底层逻辑。该资源一共分为4个部分:1、yarn提交流程,2、组件通信,3、任务调度,4、内存管理 适合人群:具备一定编程基础和大数据基础知识,工作1-3年的大数据研发和运维人员 能学到什么:①flink on yarn模式之下的启动和运行原理,通过简单梳理底层逻辑加深对flink的掌握;②从底层机理的角度剖析flink是如何处理我们的需求的,从而帮助我们更好的解决运维问题。 阅读建议:此资源基于flink的源码追踪并简单分析,学习参考的时候建议下载相应版本的flink源码,并搭建虚拟机安装部署测试flink,在运行过程中结合源码和笔记学习效果更佳。有经验的开发人员可以编译并debug flink的源码,这样可以更加直观深入地研究flink的运行原理。有利于开发应用和解决问题

文件下载

资源详情

[{"title":"( 45 个子文件 4.12MB ) flink源码浅析和运行机理梳理","children":[{"title":"flink源码分析","children":[{"title":"4-内存管理","children":[{"title":"01内存模型.md <span style='color:#111;'> 3.94KB </span>","children":null,"spread":false},{"title":"05网络传输中的内存管理.md <span style='color:#111;'> 5.96KB </span>","children":null,"spread":false},{"title":"02内存分配.md <span style='color:#111;'> 5.28KB </span>","children":null,"spread":false},{"title":"04内存管理器.md <span style='color:#111;'> 3.06KB </span>","children":null,"spread":false},{"title":"03内存数据结构.md <span style='color:#111;'> 2.29KB </span>","children":null,"spread":false},{"title":"06本章小结.md <span style='color:#111;'> 241B </span>","children":null,"spread":false}],"spread":true},{"title":"2-组件通信","children":[{"title":"01AKKA和Actor模型介绍.md <span style='color:#111;'> 5.76KB </span>","children":null,"spread":false},{"title":"02flink关于RPC框架的实现.md <span style='color:#111;'> 14.37KB </span>","children":null,"spread":false},{"title":"03本章小结.md <span style='color:#111;'> 1.05KB </span>","children":null,"spread":false}],"spread":true},{"title":"0-图片","children":[{"title":"反压的过程.png <span style='color:#111;'> 101.25KB </span>","children":null,"spread":false},{"title":"flink-Graph演变过程.png <span style='color:#111;'> 238.22KB </span>","children":null,"spread":false},{"title":"flink-TM内存模型.png <span style='color:#111;'> 158.79KB </span>","children":null,"spread":false},{"title":"flink-JobGraph.png <span style='color:#111;'> 53.88KB </span>","children":null,"spread":false},{"title":"flinkRPC框架类继承图.png <span style='color:#111;'> 272.38KB </span>","children":null,"spread":false},{"title":"flink通讯过程.png <span style='color:#111;'> 181.70KB </span>","children":null,"spread":false},{"title":"flink内存模型类继承图.png <span style='color:#111;'> 28.75KB </span>","children":null,"spread":false},{"title":"MemorySegment存储示意图.png <span style='color:#111;'> 102.16KB </span>","children":null,"spread":false},{"title":"task调度2.png <span style='color:#111;'> 337.13KB </span>","children":null,"spread":false},{"title":"网络传输的内存管理请求过程.png <span style='color:#111;'> 80.54KB </span>","children":null,"spread":false},{"title":"flink-ExecutionGraph.png <span style='color:#111;'> 141.10KB </span>","children":null,"spread":false},{"title":"flink-StreamGraph.png <span style='color:#111;'> 57.55KB </span>","children":null,"spread":false},{"title":"flink-JM内存模型.png <span style='color:#111;'> 30.74KB </span>","children":null,"spread":false},{"title":"flinkRPC通信过程简图.png <span style='color:#111;'> 318.25KB </span>","children":null,"spread":false},{"title":"flink-物理执行图.png <span style='color:#111;'> 125.28KB </span>","children":null,"spread":false},{"title":"task任务调度执行.png <span style='color:#111;'> 150.87KB </span>","children":null,"spread":false},{"title":"flink的WebUI上看到的streamgraph.png <span style='color:#111;'> 156.00KB </span>","children":null,"spread":false},{"title":"task调度1.png <span style='color:#111;'> 144.00KB </span>","children":null,"spread":false},{"title":"transformations树示意图.png <span style='color:#111;'> 211.02KB </span>","children":null,"spread":false},{"title":"transformation树的简化示意图.png <span style='color:#111;'> 274.80KB </span>","children":null,"spread":false},{"title":"从source到sink.png <span style='color:#111;'> 351.68KB </span>","children":null,"spread":false},{"title":"继承图schedulerNG.png <span style='color:#111;'> 56.19KB </span>","children":null,"spread":false},{"title":"flink内存模型.png <span style='color:#111;'> 146.42KB </span>","children":null,"spread":false},{"title":"transformation封装过程.png <span style='color:#111;'> 278.28KB </span>","children":null,"spread":false},{"title":"flink提交流程-yarn-perjob.png <span style='color:#111;'> 235.61KB </span>","children":null,"spread":false},{"title":"继承图schedulingStrategy.png <span style='color:#111;'> 101.72KB </span>","children":null,"spread":false}],"spread":false},{"title":"3-任务调度","children":[{"title":"02Graph生成过程.md <span style='color:#111;'> 36.85KB </span>","children":null,"spread":false},{"title":"01Graph概念介绍.md <span style='color:#111;'> 5.50KB </span>","children":null,"spread":false},{"title":"05本章小结.md <span style='color:#111;'> 239B </span>","children":null,"spread":false},{"title":"03Task的调度和执行.md <span style='color:#111;'> 14.19KB </span>","children":null,"spread":false},{"title":"04调度和调度策略概念.md <span style='color:#111;'> 6.11KB </span>","children":null,"spread":false}],"spread":true},{"title":"1-yarn提交流程","children":[{"title":"04在yarn上部署flink集群.md <span style='color:#111;'> 79.15KB </span>","children":null,"spread":false},{"title":"03部署集群的准备&提交任务.md <span style='color:#111;'> 33.31KB </span>","children":null,"spread":false},{"title":"02执行用户代码.md <span style='color:#111;'> 13.32KB </span>","children":null,"spread":false},{"title":"01启动命令解析&读配置&封装.md <span style='color:#111;'> 20.47KB </span>","children":null,"spread":false},{"title":"05本章小结.md <span style='color:#111;'> 125B </span>","children":null,"spread":false}],"spread":true}],"spread":true}],"spread":true}]

评论信息

免责申明

【只为小站】的资源来自网友分享,仅供学习研究,请务必在下载后24小时内给予删除,不得用于其他任何用途,否则后果自负。基于互联网的特殊性,【只为小站】 无法对用户传输的作品、信息、内容的权属或合法性、合规性、真实性、科学性、完整权、有效性等进行实质审查;无论 【只为小站】 经营者是否已进行审查,用户均应自行承担因其传输的作品、信息、内容而可能或已经产生的侵权或权属纠纷等法律责任。
本站所有资源不代表本站的观点或立场,基于网友分享,根据中国法律《信息网络传播权保护条例》第二十二条之规定,若资源存在侵权或相关问题请联系本站客服人员,zhiweidada#qq.com,请把#换成@,本站将给予最大的支持与配合,做到及时反馈和处理。关于更多版权及免责申明参见 版权及免责申明