Data processing, implementing related algorithms, tuning, scaling up and finally deploying are some crucial steps in the process of optimising any application.
Spark is capable of handling large-scale batch and streaming data to figure out when to cache data in memory and processing them up to 100 times faster than Hadoop-based MapReduce.This means predictive analytics can be applied to streaming and batch to develop complete machine learning (ML) applications a lot quicker, making Spark an ideal candidate for large data-intensive applications.
This book focuses on design engineering and scalable solutions using ML with Spark. First, you will learn how to install Spark with all new features from the latest Spark 2.0 release. Moving on, you’ll explore important concepts such as advanced feature engineering with RDD and Datasets. After studying developing and deploying applications, you will see how to use external libraries with Spark.
2021-08-27 14:51:14
11.47MB
1
In this talk, we will highlight major efforts happening in the Spark ecosystem. In particular, we will dive into the details of adaptive and static query optimizations in Spark 3.0 to make Spark easier to use and faster to run. We will also demonstrate how new features in Koalas, an open source library that provides Pandas-like API on top of Spark, helps data scientists gain insights from their data quicker.
2021-08-25 09:26:58
12.3MB
1
此文档共2.2万字，详细介绍了spark图计算的存储模式，图的构建，原理分析，以及计算模式，加上常用的API，PageRank排名算法等，章图算法介绍，还有实现代码练习
2021-08-24 23:01:44
2.77MB
1
涵盖大数据hadoop、spark、scala、kafka和Linux-CentOS 6 基础和理论知识最全的思维导图，有助于学习大数据的小白自学
2021-08-24 21:56:01
14.37MB
1