Matrix Reference Manual

Matrix Calculus

Go to: Introduction, Notation, Index

- Notation
- Derivatives of Linear Products
- Derivatives of Quadratic Products

*d/dx*(**y**)*(i)*element is*dy(i)/dx**d/d***x**(y) is a vector whose*(i)*element is*dy/dx(i)**d/d***x**(**y**) is a matrix whose^{T}*(i,j)*element is*dy(j)/dx(i)**d/dx*(**Y**) is a matrix whose*(i,j)*element is*dy(i,j)/dx**d/d***X**(y) is a matrix whose*(i,j)*element is*dy/dx(i,j)*

Note that the Hermitian transpose is not used because complex conjugates are not analytic.

In the expressions below matrices and vectors **A**, **B**, **C** do not depend on **X**.

*d/dx*(**AYB**)*=***A****d/dx*(**Y**) ***B***d/dx*(**Ay**)*=***A****d/dx*(**y**)*d/d***x****x**^{T}**A**)*=***A***d/d***x****x**)^{T}*=***I***d/d***x****x**^{T}**a**)*= d/d***x****a**^{T}**x**) =**a***d/d***X****a**^{T}**Xb**) =**ab**^{T}*d/d***X****a**^{T}**Xa**) =*d/d***X****a**^{T}**X**^{T}**a**) =**aa**^{T}*d/d***X****a**^{T}**X**^{T}**b**) =**ba**^{T}*d/dx*(**YZ**)*=***Y****d/dx*(**Z**) +*d/dx*(**Y**)*** Z**

- d/d
**x**(**Ax**+**b**)^{T}**C**(**D**x+**e**) =**A**^{T}**C(Dx+e)**+**D**^{T}**C**^{T}**(Ax+b)** - d/d
**x**(**x**^{T}**Cx**) = (**C**+**C**)^{T}**x** - [C: symmetric]: d/d
**x**(**x**^{T}**Cx**) = 2**Cx** - d/d
**x**(**x**^{T}**x**) = 2**x** - d/d
**x**(**Ax**+**b**)(^{T}**D**x+**e**) =**A**^{T}**(Dx+e)**+**D**^{T}**(Ax+b)** - d/d
**x**(**Ax**+**b**)(^{T}**A**x+**b**) = 2**A**^{T}**(Ax+b)** - [C: symmetric]: d/d
**x**(**Ax**+**b**)^{T}**C**(**A**x+**b**) = 2**A**^{T}**C(Ax+b)** *d/d***X****a**^{T}**X**^{T}**Xb**) =**X(ab**^{T}**+ ba**^{T}**)***d/d***X****a**^{T}**X**^{T}**Xa**) = 2**Xaa**^{T}*d/d***X****a**^{T}**X**^{T}**CXb**) =**C**^{T}**Xab**^{T}**+ CXba**^{T}*d/d***X****a**^{T}**X**^{T}**CXa**) =**(C + C**^{T}**)Xaa**^{T}- [
**C**:*Symmetric*]*d/d***X****a**^{T}**X**^{T}**CXa**) =**2CXaa**^{T} *d/d***X****Xa+b)**^{T}**C(Xa+b)**) = (**C+C**^{T}**)(Xa+b)a**^{T}

*d/d***x****x**^{T}**Axx**) = (^{T}**A+A**^{T}**)xx**^{T}**+x**^{T}**AxI**

*d/dx*(**Y**^{-1}) =**-Y**^{-1}*d/dx*(**Y**)**Y**^{-1}

Note: matrix dimensions must result in an *n*n* argument for tr().

*d/d***X****(X)**) =**I***d/d***X****(X**^{k}**)**) =*k***(X**^{k}^{-1}**)**^{T}*d/d***X****(AX**^{k}**)**) =**SUM**_{r=0:k-1}**(X**^{r}**AX**^{k-r}^{-1}**)**^{T}*d/d***X****(AX**^{-1}**B)**) =**-(X**^{-1}**BAX**^{-1}**)**^{T}*d/d***X****(AX**^{-1}**)**) =*d/d***X****(X**^{-1}**A)**) =**-X**^{-T}**A**^{T}**X**^{-T}

*d/d***X****(A**^{T}**XB**)) =^{T}*d/d***X****(BX**^{T}**A**)) =^{}**AB***d/d***X****(XA**^{T}**)**) =*d/d***X****tr(A**^{T}**X)**) =*d/d***X****(X**^{T}**A)**) =*d/d***X****tr(AX**^{T}**)**)**= A**

*d/d***X****(AXBX**^{T}**)**) =**A**^{T}**XB**+^{T}**AXB***d/d***X****(XAX**^{T}**)**) =**X(A+A**^{T}**)***d/d***X****(X**^{T}**AX)**) =**X**^{T}**(A+A**^{T}**)***d/d***X****(AX**^{T}**X)**) =**(A+A**^{T}**)X**

*d/d***X****(AXBX)**) =**A**^{T}**X**^{T}**B**+^{T}**B**^{T}**X**^{T}**A**^{T}- [C:
*symmetric*]*d/d***X****tr((X**^{T}**CX)**^{-1}**A**) =*d/d***X****tr(A (X**^{T}**CX)**^{-1}) =**-(CX(X**^{T}**CX)**^{-1}**)(A+A**^{T}**)(X**^{T}**CX)**^{-1} - [B,C:
*symmetric*]*d/d***X****tr((X**^{T}**CX)**^{-1}**(X**^{T}**BX)**) =*d/d***X****tr( (X**^{T}**BX)(X**^{T}**CX)**^{-1}) =**-2(CX(X**^{T}**CX)**^{-1}**)X**^{T}**BX(X**^{T}**CX)**^{-1}**+ 2BX(X**^{T}**CX)**^{-1}

Note: matrix dimensions must result in an *n*n* argument for det().

*d/d***X****(X)**) =*d/d***X****(X**^{T}**)**) = det**(X)*X**^{-T}*d/d***X****(AXB)**) = det**(AXB)*X**^{-T}*d/d***X****(AXB))**) =**X**^{-T}*d/d***X****(X**^{k}**)**) =*k**det**(X**^{k}**)*****X**^{-T}*d/d***X****(X**^{k}**)**)) =*k***X**^{-T}

- [
*Real*]*d/d***X****(X**^{T}**CX)**) = det**(X**^{T}**CX)*(C+C**^{T}**)X(X**^{T}**CX)**^{-1} - [
**C**:*Real,Symmetric*]*d/d***X****(X**^{T}**CX)**) = 2det**(X**^{T}**CX)* CX(X**^{T}**CX)**^{-1} - [
**C**:*Real,Symmetricc*]*d/d***X****(X**^{T}**CX))**) = 2**CX(X**^{T}**CX)**^{-1}

If **y** is a function of **x**, then *d***y*** ^{T}*/d

Its determinant, |*d***y*** ^{T}*/d

If f is a function of **x** then the symmetric matrix d^{2}f/d**x**^{2} = **d**/d**x*** ^{T}*(df/d

- d
^{2}/d**x**^{2}(**a**^{T}**x**) = 0 - d
^{2}/d**x**^{2}(**Ax**+**b**)^{T}**C**(**D**x+**e**) =**A**^{T}**CD**+**D**^{T}**C**^{T}**A** - d
^{2}/d**x**^{2}(**x**^{T}**Cx**) =**C**+**C**^{T } - d
^{2}/d**x**^{2}(**x**^{T}**x**) = 2**I** - d
^{2}/d**x**^{2}(**Ax**+**b**)(^{T}**D**x+**e**) =**A**^{T}**D**+**D**^{T}**A** - d
^{2}/d**x**^{2}(**Ax**+**b**)(^{T}**A**x+**b**) = 2**A**^{T}**A** - [C: symmetric]: d
^{2}/d**x**^{2}(**Ax**+**b**)^{T}**C**(**A**x+**b**) = 2**A**^{T}**CA**

The Matrix Reference Manual is written by Mike Brookes, Imperial College, London, UK. Please send any comments or suggestions to mike.brookes@ic.ac.uk