在本动手实验中,我们将深入探索Azure Data Factory(ADF),这是一个云原生的数据集成服务,用于创建、调度和管理ETL(提取、转换、加载)和ELT(提取、加载、转换)工作流。ADF提供了丰富的功能,如数据复制、数据转换、数据触发以及与其他Azure服务的深度集成,是构建现代化数据仓库和数据湖解决方案的关键组件。 **Azure Data Factory基础知识** Azure Data Factory 是微软云中的一个完全托管的服务,允许用户通过图形化界面或JSON脚本来创建数据集成工作流。它支持多种数据源,包括Azure内的存储服务(如Blob Storage、ADLS Gen2)以及SQL Server、Oracle、SAP等外部数据源。 **ADF管道与活动** 在ADF中,数据集成逻辑被组织成"管道",每个管道可以包含一个或多个"活动"。活动是执行特定任务的单元,如数据复制、数据转换或控制流操作。例如,"Filter Activity"根据指定条件筛选数据,"Lookup Activity"用于从另一个数据源查询数据,"ForEach Activity"则用于对集合执行迭代操作,而"Metadata Activity"用于获取数据集的元数据。 **映射数据流** ADF的一项强大特性是"Mapping Data Flows",它提供了一种可视化方式来设计和执行数据转换。数据流允许非编码人员也能进行复杂的数据转换,如选择、重命名、过滤、聚合、合并和JOIN等操作。此外,它还支持Spark引擎进行大规模并行处理,提高了处理大量数据的效率。 **Azure Key Vault集成** 在安全性和合规性方面,ADF能够与Azure Key Vault集成,用于管理连接字符串、凭据和其他敏感信息。这样可以确保数据访问的安全,并符合企业安全标准。 **ETL/ELT流程** ADF支持两种主要的数据集成模式:ETL(提取、转换、加载)和ELT(提取、加载、转换)。ETL模式在云中完成数据转换,而ELT模式则将数据加载到云存储后,再在计算层如Azure Databricks或HDInsight上执行转换。 **Web Activity** Web Activity允许在ADF管道中执行HTTP请求,这可以用于调用REST API、触发Web服务或者获取外部系统的状态信息。这对于集成各种云服务和实现自动化工作流程非常有用。 **Azure Modern Data Warehouse** ADF在构建现代化数据仓库中扮演着重要角色,它可以轻松地将数据从多个来源整合到Azure SQL Data Warehouse、Synapse Analytics或大数据平台(如ADLS Gen2、HDInsight)。 **动手实验室** "AzureDataFactoryHOL-master"压缩包可能包含了完成本实验所需的所有资源和步骤,包括教程文档、示例数据、ADF配置文件等。通过这个动手实验,参与者将学习如何创建和部署ADF管道,配置各种活动,以及如何使用映射数据流进行数据转换。 总结起来,这个动手实验将涵盖Azure Data Factory的核心概念、关键功能和最佳实践,帮助你掌握如何利用ADF构建高效、安全和可扩展的数据集成解决方案。通过实际操作,你将加深对云数据工厂的理解,并提升你的数据工程技能。
2025-06-29 11:23:20 466.39MB azure hands-on-lab
1
这部书是一本以实践为主兼顾理论的机器学习好书,豆瓣高分,本电子书质量很好
2024-08-25 18:40:04 47.53MB TensorFlow Python
1
Learn and implement quantitative finance using popular Python libraries like NumPy, pandas, and Keras Key Features Understand Python data structure fundamentals and work with time series data Use popular Python libraries including TensorFlow, Keras, and SciPy to deploy key concepts in quantitative finance Explore various Python programs and learn finance paradigms Book Description Python is one of the most popular languages used for quantitative finance. With this book, you'll explore the key characteristics of Python for finance, solve problems in finance, and understand risk management. The book starts with major concepts and techniques related to quantitative finance, and an introduction to some key Python libraries. Next, you'll implement time series analysis using pandas and DataFrames. The following chapters will help you gain an understanding of how to measure the diversifiable and non-diversifiable security risk of a portfolio and optimize your portfolio by implementing Markowitz Portfolio Optimization. Sections on regression analysis methodology will help you to value assets and understand the relationship between commodity prices and business stocks. In addition to this, you'll be able to forecast stock prices using Monte Carlo simulation. The book will also highlight forecast models that will show you how to determine the price of a call option by analyzing price variation. You'll also use deep learning for financial data analysis and forecasting. In the concluding chapters, you will create neural networks with TensorFlow and Keras for forecasting and prediction. By the end of this book, you will be equipped with the skills you need to perform different financial analysis tasks using Python
2024-07-28 12:22:48 12.44MB Python Finance TensorFlow Keras
1
使用Python的动手深度学习算法 这是Packt发布的《 的代码库。 通过使用TensorFlow实施深度学习算法和广泛的数学知识 这本书是关于什么的? 深度学习是AI领域最受欢迎的领域之一,可让您开发各种复杂程度不同的多层模型。 本书涵盖以下激动人心的功能: 实施基础到高级的深度学习算法 掌握深度学习算法背后的数学 熟悉梯度下降及其变体,例如AMSGrad,AdaDelta,Adam和Nadam 实施循环网络,例如RNN,LSTM,GRU和seq2seq模型 了解机器如何使用CNN和胶囊网络解释图像 如果您觉得这本书适合您,请立即获取! 说明和导航 所有代码都组织在文件夹中。 该代码将如下所示: J_plus = forward_prop(x, weights_plus) J_minus = forward_prop(x, weights_minus) 这是您需要的本
2024-04-10 09:45:51 127.09MB python machine-learning deep-learning
1
动手的RESTful API设计模式和最佳实践 这是Packt发布的“ 的代码存储库。 设计,开发和部署高度适应性,可扩展性和安全性的RESTful Web API 这本书是关于什么的? 本书介绍了代表性状态传输(REST)范例,该范例是一种体系结构样式,允许联网的设备通过Internet相互通信。 在本书的帮助下,您将探索面向服务的体系结构(SOA),事件驱动的体系结构(EDA)和面向资源的体系结构(ROA)的概念。 本书涵盖了为什么需要高质量的API来进行企业集成。 本书涵盖以下激动人心的功能: 探索RESTful概念,包括URI,HATEOAS和按需编码 研究无状态,分页和可发现性等核心模式 使用API​​网关优化链接微服务的端点 深入研究API身份验证,授权和API安全性实施 与服务编排一起制定复合和流程感知服务 公开用于云计算的基于RESTful协议的API 如果您觉
2024-02-18 10:22:16 304KB Java
1
hands-on_design-patterns-delphi_pdf-code.rar
2024-01-12 10:30:26 3.3MB Delphi
1
Openpose的pth模型文件,包含pose及hands
2023-12-10 23:21:19 634.17MB Openpose
1
Hands-On Enterprise Automation with Python: Automate common administrative and security tasks with the most popular language Python Invent your own Python scripts to automate your infrastructure Hands-On Enterprise Automation with Python starts by covering the set up of a Python environment to perform automation tasks, as well as the modules, libraries, and tools you will be using. We’ll explore examples of network automation tasks using simple Python programs and Ansible. Next, we will walk you through automating administration tasks with Python Fabric, where you will learn to perform server configuration and administration, along with system administration tasks such as user management, database management, and process management. As you progress through this book, you’ll automate several testing services with Python scripts and perform automation tasks on virtual machines and cloud infrastructure with Python. In the concluding chapters, you will cover Python-based offensive security tools and learn how to automate your security tasks. By the end of this book, you will have mastered the skills of automating several system administration tasks with Python. What You Will Learn Understand common automation modules used in Python Develop Python scripts to manage network devices Automate common Linux administration tasks with Ansible and Fabric Managing Linux processes Administrate VMware, OpenStack, and AWS instances with Python Security automation and sharing code on GitHub
2023-10-11 06:02:07 12.85MB 企业自动化 python ansible fabric
1
Hands-On Machine Learning with Scikit-Learn and TensorFlow + 源码
2023-06-05 17:44:36 19.2MB Machine Learning Scikit-Learn TensorFlow
1
Create visually appealing and feature-rich applications by using Qt 5 and the C++ language Qt 5, the latest version of Qt, enables you to develop applications with complex user interfaces for multiple targets. It provides you with faster and smarter ways to create modern UIs and applications for multiple platforms. This book will teach you to design and build graphical user interfaces that are functional, appealing, and user-friendly. In the initial part of the book, you will learn what Qt 5 is and what you can do with it. You will explore the Qt Designer, discover the different types of widgets generally used in Qt 5, and then connect your application to the database to perform dynamic operations. Next, you will be introduced to Qt 5 chart which allows you to easily render different types of graphs and charts and incorporate List View Widgets in your application. You will also work with various Qt modules, like QtLocation, QtWebEngine, and the networking module through the course of the book. Finally, we will focus on cross-platform development with QT 5 that enables you to code once and run it everywhere, including mobile platforms. By the end of this book, you will have successfully learned about high-end GUI applications and will be capable of building many more powerful, cross-platform applications. What You Will Learn Implement tools provided by Qt 5 to design a beautiful GUI Understand different types of graphs and charts supported by Qt 5 Create a web browser using the Qt 5 WebEngine module and web view widget Connect to the MySQL database and display data obtained from it onto the Qt 5 GUI Incorporate the Qt 5 multimedia and networking module in your application Develop Google Map-like applications using Qt 5’s location module Discover cross-platform development by exporting the Qt 5 application to different platforms Uncover the secrets behind debugging Qt 5 and C++ applications
2023-04-12 15:06:35 12.64MB QT C++ GUI QT5
1