使用kafka,spark,hbase开发日志分析系统

上传者: 25288411 | 上传时间: 2025-06-03 23:22:34 | 文件大小: 232KB | 文件类型: ZIP
# 日志分析系统 ## 系统架构 本使用kafka,spark,hbase开发日志分析系统。 ![architecture](/docs/images/architecture.png "architecture") ### 软件模块 * Kafka:作为日志事件的消息系统,具有分布式,可分区,可冗余的消息服务功能。 * Spark:使用spark stream功能,实时分析消息系统中的数据,完成计算分析工作。 * Hbase:做为后端存储,存储spark计算结构,供其他系统进行调用 ## 环境部署 ### 软件版本 * hadoop 版本 : Hadoop相关软件如zookeeper、hadoop、hbase,使用的是cloudera的 cdh 5.2.0 版本。 * Kafka : 2.9.2-0.8.1.1

文件下载

资源详情

[{"title":"( 30 个子文件 232KB ) 使用kafka,spark,hbase开发日志分析系统","children":[{"title":"logcount-master日志统计分析","children":[{"title":"logcount-master","children":[{"title":".project <span style='color:#111;'> 537B </span>","children":null,"spread":false},{"title":"src","children":[{"title":"main","children":[{"title":"resources","children":[{"title":"hbase-site.xml <span style='color:#111;'> 7.48KB </span>","children":null,"spread":false},{"title":"webapps","children":[{"title":"recsys","children":[{"title":"index.html <span style='color:#111;'> 270B </span>","children":null,"spread":false}],"spread":true}],"spread":true},{"title":"logback.xml <span style='color:#111;'> 2.02KB </span>","children":null,"spread":false}],"spread":true},{"title":"java","children":[{"title":"com","children":[{"title":"wankun","children":[{"title":"logcount","children":[{"title":"kafka","children":[{"title":"TailService.java <span style='color:#111;'> 617B </span>","children":null,"spread":false},{"title":"TailLog.java <span style='color:#111;'> 3.37KB </span>","children":null,"spread":false},{"title":"MsgSender.java <span style='color:#111;'> 1.61KB </span>","children":null,"spread":false}],"spread":true},{"title":"spark","children":[{"title":"RecsysLogs.java <span style='color:#111;'> 1.66KB </span>","children":null,"spread":false},{"title":"LogStream.java <span style='color:#111;'> 4.36KB </span>","children":null,"spread":false}],"spread":true}],"spread":true}],"spread":true}],"spread":true}],"spread":true}],"spread":true}],"spread":true},{"title":"target","children":[{"title":"classes","children":[{"title":"hbase-site.xml <span style='color:#111;'> 7.48KB </span>","children":null,"spread":false},{"title":"com","children":[{"title":"wankun","children":[{"title":"logcount","children":[{"title":"kafka","children":[{"title":"MsgSender.class <span style='color:#111;'> 1.79KB </span>","children":null,"spread":false},{"title":"TailLog.class <span style='color:#111;'> 1.34KB </span>","children":null,"spread":false},{"title":"TailService.class <span style='color:#111;'> 819B </span>","children":null,"spread":false}],"spread":true},{"title":"spark","children":[{"title":"LogStream.class <span style='color:#111;'> 6.53KB </span>","children":null,"spread":false},{"title":"RecsysLogs.class <span style='color:#111;'> 3.22KB </span>","children":null,"spread":false}],"spread":true}],"spread":true}],"spread":true}],"spread":true},{"title":"META-INF","children":[{"title":"MANIFEST.MF <span style='color:#111;'> 107B </span>","children":null,"spread":false},{"title":"maven","children":[{"title":"com.wankun","children":[{"title":"logcount","children":[{"title":"pom.properties <span style='color:#111;'> 265B </span>","children":null,"spread":false},{"title":"pom.xml <span style='color:#111;'> 2.29KB </span>","children":null,"spread":false}],"spread":true}],"spread":true}],"spread":true}],"spread":true},{"title":"webapps","children":[{"title":"recsys","children":[{"title":"index.html <span style='color:#111;'> 270B </span>","children":null,"spread":false}],"spread":true}],"spread":true},{"title":"logback.xml <span style='color:#111;'> 2.02KB </span>","children":null,"spread":false}],"spread":true}],"spread":true},{"title":".settings","children":[{"title":"org.eclipse.m2e.core.prefs <span style='color:#111;'> 86B </span>","children":null,"spread":false},{"title":"org.eclipse.jdt.core.prefs <span style='color:#111;'> 238B </span>","children":null,"spread":false},{"title":"org.eclipse.core.resources.prefs <span style='color:#111;'> 150B </span>","children":null,"spread":false}],"spread":true},{"title":"README.md <span style='color:#111;'> 2.94KB </span>","children":null,"spread":false},{"title":"pom.xml <span style='color:#111;'> 2.29KB </span>","children":null,"spread":false},{"title":"docs","children":[{"title":"日志分析系统.docx <span style='color:#111;'> 60.86KB </span>","children":null,"spread":false},{"title":"images","children":[{"title":"architecture.png <span style='color:#111;'> 53.45KB </span>","children":null,"spread":false}],"spread":true},{"title":"架构.ppt <span style='color:#111;'> 261.50KB </span>","children":null,"spread":false}],"spread":true},{"title":".classpath <span style='color:#111;'> 1.05KB </span>","children":null,"spread":false},{"title":".gitignore <span style='color:#111;'> 8B </span>","children":null,"spread":false}],"spread":true}],"spread":true}],"spread":true}]

评论信息

免责申明

【只为小站】的资源来自网友分享,仅供学习研究,请务必在下载后24小时内给予删除,不得用于其他任何用途,否则后果自负。基于互联网的特殊性,【只为小站】 无法对用户传输的作品、信息、内容的权属或合法性、合规性、真实性、科学性、完整权、有效性等进行实质审查;无论 【只为小站】 经营者是否已进行审查,用户均应自行承担因其传输的作品、信息、内容而可能或已经产生的侵权或权属纠纷等法律责任。
本站所有资源不代表本站的观点或立场,基于网友分享,根据中国法律《信息网络传播权保护条例》第二十二条之规定,若资源存在侵权或相关问题请联系本站客服人员,zhiweidada#qq.com,请把#换成@,本站将给予最大的支持与配合,做到及时反馈和处理。关于更多版权及免责申明参见 版权及免责申明