Scala代码积累之spark streaming kafka 数据存入到hive源码实例，Scala代码积累之spark streaming kafka 数据存入到hive源码实例。

手把手视频详细讲解项目开发全过程，需要的小伙伴自行百度网盘下载，链接见附件，永久有效。 课程介绍 从基础知识点到安装部署、集群配置、各种服务安装到增加新节点，通过知识点 + 实际操作的方式帮助小白快速掌握CDH的安装和配置。 课程亮点 1，Cloudera Manager可视化、自动部署和配置，稳定性好 2，理论+操作，培养解决实际问题的能力。 3，涉及到大数据离线和实时主流服务安装。 适用人群 1、对大数据感兴趣的在校生及应届毕业生。 2、对目前职业有进一步提升要求，希望从事大数据行业高薪工作的在职人员。 3、对大数据行业感兴趣的相关人员。 课程内容 1. 大数据架构和技术选型 2. 虚拟机环境 3. Cloudera Manager 4. 添加HDFS服务 5. 添加Yarn服务 6. 添加Zookeeper服务 7. 添加Hive服务 8. 添加Oozie服务 9. 添加Sqoop服务 10. 添加HBase服务 11. 添加Spark服务 12. 添加Hue服务 13. 添加新服务器

Spark 2.3.3 安装包，配合本人博客《Spark 2.3 安装部署》一同使用

Linux系统 大数据开发 spark-2.1.0-bin-without-hadoop.tgz

spark3读hive1，配置spark.sql.hive.metastore.jars

emp.json员工信息

spark源码在hadoop-cdh5.7.0编译生成，用于学习hadoop和spark课程

Today's Web-enabled deluge of electronic data calls for automated methods of data analysis. Machine learning provides these, developing methods that can automatically detect patterns in data and then use the uncovered patterns to predict future data. This textbook offers a comprehensive and self-contained introduction to the field of machine learning, a unified, probabilistic approach. The coverage combines breadth and depth, offering necessary background material on such topics as probability, optimization, and linear algebra as well as discussion of recent developments in the field, including conditional random fields, L1 regularization, and deep learning. The book is written in an informal, accessible style, complete with pseudo-code for the most important algorithms. All topics are copiously illustrated with color images and worked examples drawn from such application domains as biology, text processing, computer vision, and robotics. Rather than providing a cookbook of different heuristic methods, the book stresses a principled model-based approach, often using the language of graphical models to specify models in a concise and intuitive way. Almost all the models described have been implemented in a MATLAB software package--PMTK (probabilistic modeling toolkit)--that is freely available online. The book is suitable for upper-level undergraduates with an introductory-level college math background and beginning graduate students. 优点：新，全！ 由于成书时间较晚，所以涵盖了更多最近几年的hot topic，比如Dirichlet Process 。 更重要的，是全，基本上ML领域的专有名词，你都可以在书后的index找到。说道这里，不得不佩服本书的作者Kevin Murphy，剑桥的本科，UCB的博士，MIT的博后，得到过多位大牛的真传 。 还有一个非常重要的，就是这本书配备了详尽的matlab code，你几乎可以尝试书中的每一个例子。 单从以上这几点，绝对应该把他排在所有ML教材的首位！

在开发spark2.5.8的时候用到，希望大家也能找到

项目实战：Java一站式解决Hive内用Spark取数，新建ES索引，灌入数据，并且采用ES别名机制，实现ES数据更新的无缝更新，底层采用Spark计算框架，数据较快。