Matrix Identities

Inversion Identities

These identities are useful because it says how a matrix changes if you add a bit onto its inverse. They are variously called the Matrix Inversion Lemma, Sherman-Morrison formula and Sherman-Morrison-Woodbury formula.

• [ (I+V^HAU) non-singular]: (A^-1 + UV^H)^-1 = A - (AU(I + V^HAU)^-1)V^HA
• • [v^HAu != -1]: (A^-1 + uv^H)^-1 = A - Auv^HA/(1 + v^HAu)
  • [ (C+V^HAV) non-singular]: (A^-1 + VC^-1V^H)^-1 = A - (AV(C + V^HAV)^-1)V^HA
  • [ (I+V^HAV) non-singular]: (A^-1 + VV^H)^-1 = A - (AV(I + V^HAV)^-1)V^HA

Projection Matrix Identities

These identities show how the matrix that projects onto the column-space of X changes if extra columns are added to X. We define the projection matrices P(X)=X(X^HX)^#X^H and Q(X)=I-P(X), these respectively project onto the column space and null space of X.

• P([X Y]) = P(X) + P(Q(X)Y) = P(X) + Q(X) Y (Y^H Q(X) Y)^# Y^H Q(X)
• • [y^HQ(X)y = 0] P([X y]) = P(X)
  • [y^HQ(X)y != 0] P([X y]) = P(X) + P(Q(X)y) = P(X) + Q(X) y y^H Q(X) / (y^H Q(X) y)
  • • [y^HQ(X)y != 0] z^HP([X y])z = z^HP(X)z + |z^HQ(X) y|² / (y^H Q(X) y)
• Q([X Y]) = Q(X) - Q(X) Y (Y^H Q(X) Y)^# Y^H Q(X)

Toeplitz and Hankel Identities

If a vector is multiplied by a triangular toeplitz or hankel matrix, it is possible to exchange the components of the vector and matrix. See the notation section for definitions of TOPLITZ(), HANKEL() and the exchange matrix J.

• TOEPLITZ(a) b = TOEPLITZ(b) a
• TOEPLITZ(a)^T b = HANKEL(b) a
• HANKEL(a) J b = HANKEL(b) J a

It follows that a symmetrix Toeplitz matrix gives the following :

• (TOEPLITZ(a) + TOEPLITZ(a)^T) b = (TOEPLITZ(b) + HANKEL(b)) a

Note that the leading diagonal of the symmetric toeplitz matrix equals 2a(1) and that the first column of the matrix on the right hand side is equal to 2b.

Toeplitz Products

• TOEPLITZ(a) b = conv(a,b) = conv(b,a) = TOEPLITZ(b) a
• TOEPLITZ(a) TOEPLITZ(b) = TOEPLITZ(b) TOEPLITZ(a) = TOEPLITZ(conv(a,b)) = TOEPLITZ(TOEPLITZ(a) b)

Convolution Recursions

If an additional element is added to each of two vectors, we can express their convolutin recursively. Suppose that J is the exchange matrix and that b and q are scalars, then:

• conv([a; b], [p; q]) = [conv(a,p) ; [a; b]^T J [p; q]]
• conv([a; b]), J [p; q] = q * [a; b] + [0; conv(a; Jp)]

Durbin Recursion

If x satisfies R x = [1; 0] and R is a symmetric toeplitz matrix and hence of the form TOEPLITZ(r) + J TOEPLITZ(r) J for some r, where J is the exchange matrix. Then

• S y = [1; 0] where
  • S is the augmented symetric toeplitz matrix given by TOEPLITZ([r; s]) + J TOEPLITZ([r; s]) J
  • k = [r; s]^T J [x; 0]
  • y = ([x; 0] - k J [x; 0])/(1-k^2)

This recursion gives an efficient way of solving equations of the form R x = [1; 0].