B站 矩阵求导第一节/第二节/第三节讲义

一直对矩阵求导和向量求导云里雾里，直到这两天看到刘建平老师的博客，我认为是讲的最清楚的的一个系列，所以特地将其整理成pdf文档，并添加书签，供大家们参考。

三个矩阵求导资料： 1）矩阵求导_CS5240 2）The matrix cookbook 3）闲话矩阵求导

张贤达，矩阵分析与应用（第2版），高清带目录！解决了机器学习中的矩阵运算与求导。

标量对矩阵求导

用于求解hessenberg矩阵，完整可运行，可用于数值分析

The Matrix Cookbook，高清书籍+讲解+书签，推导每一个公式加上详细的讲解

矩阵求导详细手册-求导运算公式表文献及-PPT，文献为斯坦福大学数学系推导和总结的矩阵求导运算表，PPT为一些矩阵求导运算规则。是我在看机器学习线性回归推导时时候要用到的内容，网上比较后找了这两个质量比较高的资料，超值了。

该文档总结了矩阵对矩阵、矩阵对向量、向量对矩阵、向量对向量、元素对矩阵、元素对向量的求导法则，非常有用！

Preface page xi Acknowledgments xiii Abbreviations xv Nomenclature xvii 1 Introduction 1 1.1 Introduction to the Book 1 1.2 Motivation for the Book 2 1.3 Brief Literature Summary 3 1.4 Brief Outline 5 2 Background Material 6 2.1 Introduction 6 2.2 Notation and Classification of Complex Variables and Functions 6 2.2.1 Complex-Valued Variables 7 2.2.2 Complex-Valued Functions 7 2.3 Analytic versus Non-Analytic Functions 8 2.4 Matrix-Related Definitions 12 2.5 Useful Manipulation Formulas 20 2.5.1 Moore-Penrose Inverse 23 2.5.2 Trace Operator 24 2.5.3 Kronecker and Hadamard Products 25 2.5.4 Complex Quadratic Forms 29 2.5.5 Results for Finding Generalized Matrix Derivatives 31 2.6 Exercises 38 3 Theory of Complex-Valued Matrix Derivatives 43 3.1 Introduction 43 3.2 Complex Differentials 44 3.2.1 Procedure for Finding Complex Differentials 46 3.2.2 Basic Complex Differential Properties 46 3.2.3 Results Used to Identify First- and Second-Order Derivatives 53 viii Contents 3.3 Derivative with Respect to Complex Matrices 55 3.3.1 Procedure for Finding Complex-Valued Matrix Derivatives 59 3.4 Fundamental Results on Complex-Valued Matrix Derivatives 60 3.4.1 Chain Rule 60 3.4.2 Scalar Real-Valued Functions 61 3.4.3 One Independent Input Matrix Variable 64 3.5 Exercises 65 4 Development of Complex-Valued Derivative Formulas 70 4.1 Introduction 70 4.2 Complex-Valued Derivatives of Scalar Functions 70 4.2.1 Complex-Valued Derivatives of f (z, z∗) 70 4.2.2 Complex-Valued Derivatives of f (z, z∗) 74 4.2.3 Complex-Valued Derivatives of f (Z, Z∗) 76 4.3 Complex-Valued Derivatives of Vector Functions 82 4.3.1 Complex-Valued Derivatives of f (z, z∗) 82 4.3.2 Complex-Valued Derivatives of f (z, z∗) 82 4.3.3 Complex-Valued Derivatives of f (Z, Z∗) 82 4.4 Complex-Valued Derivatives of Matrix Functions 84 4.4.1 Complex-Valued Derivatives of F(z, z∗) 84 4.4.2 Complex-Valued Derivatives of F(z, z∗) 85 4.4.3 Complex-Valued Derivatives of F(Z, Z∗) 86 4.5 Exercises 91 5 Complex Hessian Matric