Matrix Reference Manual

Introduction

This manual contains reference information about linear algebra and the properties of matrices. The manual is divided into the following sections:

• Properties : Properties and numbers associated with a matrix such as determinant, rank, inverse, …
• Eigenvalues : Theorems and matrix properties relating to eigenvalues and eigenvectors.
• Special : Properties of matrices that have a special form or structure such as diagonal, traingular, toeplitz, …
• Relations : Relations between matrices such as equivalence, congruence, …
• Decompositions : Decomposing matrices as sums or products of simpler forms.
• Identities : Useful equations relating matrices.
• Equations : Solutions of matrix equations
• Differentiation : Differentiating expressions involving matrices whose elements are functions of an independent variable.
• Stochastic : Statistical properties of vectors and matrices whose elements are random numbers.
• Signals : Properties of observation vectors and covariance matrices from stochastic and deterministic signals.
• Examples: 2#2 : Examples of 2#2 matrixes with graphical illustration of their properties.
• Formal Algebra : Formal definitions of algebraic constructs such as groups, fields, vector spaces, …
• Main Index

The Matrix Reference Manual is written by Mike Brookes, Imperial College, London, UK. Please send any comments or suggestions to mike.brookes@ic.ac.uk

Format of Manual Entries

The general format of each entry is as follows:

1. Definition of the term
2. Outline of why it is important
3. Geometric Interpretation
   The geometric interpretation of a matrix property or theorem is generally described for 2 or 3 dimensions. Words prefixed by + should be altered appropriately for other dimensions. Thus the word +area should be replaced by volume for 3-D spaces and by hyper-volume for larger spaces.
4. List of properties and theorems:
   • Theorems apply to real matrices or complex matrices of arbitrary shape unless explicitly stated.
   • Matrix dimensions are assumed to allow the sums and products in a theorem.
   • If a theorem explicitly involves inv(A) or division by A, then A is assumed to be non-singular.
   • If all matrices in a theorem are of a particular form, the theorem is prefaced thus: [Real] or [Complex n#n].
   • If some matrices are of a particular for, the theorem is prefaced thus: [A,B:n#n, A:symmetric].
   • denotes a hypertext link to a proof.
   • denotes a hypertext link to an example.
5. Links to related topics

Notation

The notation is based on the MATLAB software package; differences are notes below. All vectors are column vectors unless explicitly written as transposed.

• Matrices are represented as bold upper case (A), column vectors as bold lower case (a) and real or complex scalars as italic lower case (a).
• The dimensions of a matrix with 2 rows and 3 columns are specified as 2#3.
• A matrix can be specified explicitly by listing its elements and using a semicolon to separate each row. Thus [1 2 3; 4 5 6] is a matrix with 2 rows and 3 columns. This notation can be used to compose large matrices from smaller ones: [A B;C D]. Each row must have the same total number of columns and each matrix within a row must have the same number of rows.
• The dimensions of the following special matrices are normally deduced from context but are occasionally specified explicitly (e.g. 0:n#1):
  • The matrix or vector 0 consists entirely of zeros.
  • The matrix or vector 1 consists entirely of ones.
  • The matrix I denotes the square identity matrix with 1's down the main diagonal and 0's elsewhere.
  • The matrix J denotes the square exchange matrix with 1's along the main anti-diagonal and 0's elsewhere.
• +, -, and * denote matrix addition, subtraction and multiplication. The * for multiplication may be omitted.
• A^T denotes the transpose of A.
• A^H denotes the conjugate transpose of A. If A is real then A^H = A^T.
• A^-1, A^# and A⁺ denote respectively the inverse, generalized inverse and pseudoinverse of A.
• .* and ./ denote element-by-element multiplication and division
• m:n denotes a column vector of length |1+n-m| whose elements go from m to n in steps of +1 or -1 according to whether m<n or m>n. [Different from MATLAB]
• |A| and ||A||_F denote the determinant and Frobenius norm of A.
• ||a|| denotes the euclidean norm of a
• |a| denotes the absolute value of a

Subscripts

• a(i,j) denotes the element of matrix A in row i of column j. Row and column indices begin at 1.
• a(:,j) denotes the j'th column of matrix A.
• a(:) denotes the large column vector formed by concatenating all the columns of A. If A is m#n, then a(:) = [a(1,1) a(2,1) … a(m,1) a(1,2) a(2,2) … a(m,n)]^T.
• A(X,Y) defines a matrix of the same size as X and Y (which must be the same size). Subscripts are taken from corresponding positions in X and Y.[Different from MATLAB]
  • If either X or Y is given as a row or column vector, then that vector is replicated until X and Y are the same size.
  • As an example, A((1:3)^T,1:3) is a 3*3 sub-matrix extracted from the top left corner of A.

Several of the functions listed below have different meanings according to whether their argument is a scalar, vector or matrix. The form of the result is indicated by the function's typeface.

• ABS(A) and abs(a) involves taking the absolute value of each matrix or vector element.
• CHOOSE(n,r) is a matrix with n!/(r! (n-r)!) rows, each a different choice of r numbers out of the numbers 1:n. Each row is listed in ascending order.
• CONJ(A) is the complex conjugate of A.
• conv(a,b) is the convolution of a and b, i.e. a vector whose i'th element is the sum of a(j)b(i-j+1) wher j goes from 1 to i.
• det(A) or |A| is the determinant of A.
• diag(A) is the vector consisting of the diagonal elements of A.
• DIAG(a) is the diagonal matrix whose diagonal elements are the elements of a.
• DIAG(A,B,C) denotes the matrix [A 0 0; 0 B 0; 0 0 C]
• HANKEL(a) is the upper-left triangular matrix with constant anti-diagonals whose first column is a. HANKEL(a) = J TOEPLITZ(Ja).
• INV(A) or A^-1 is the inverse of A.
• KRON(A,B) is the Kronecker product of A and B. If A is m#n and B is p#q then KRON(A,B) is mp#nq and equals the block matrix [a(1,1)B ... a(1,n)B ; ... ; a(m,1)B ... a(m,n)B].
• PERM(n) is a matrix with n! rows, each a different permutation of the numbers 1:n.
• pet(A) is the permanent of A.
• prod(a) is the product of the elements of a.
• prod() is the vector formed by multiplying together the elements of each row of A. [Different from MATLAB].
• rows(A) is the number of rows in the matrix A.
• tr(A) is the trace of A.
• rank(A) is the dimension of the subspace spaned by the columns of A.
• sgn(a) equals +1 or -1 according to the sign of a.
• sgn(a) equals +1 or -1 according to the signature of the permutation needed to sort the elements of a into ascending order.
• sgn(A) is a vector giving the permutation signatures of each row of A. Each entry equals +1 or -1.
• sum(a) is the sum of the elements of a.
• sum(A) is the vector formed by summing the rows of A. [Different from MATLAB].
• sum(A) is the scalar formed by summing all the elements of A. [Different from MATLAB].
• TOEPLITZ(a) is the lower triangular matrix with constant diagonals whose first column is a. TOEPLITZ(a) = J HANKEL(Ja).

Acknowledgements

No originality is claimed for any of the material in this reference manual. The following books have in particular been very helpful:

• A Survey of Matrix Theory and Matrix Inequalities by M Marcus & H Minc, Prindle, Weber & Schmidt, 1964 / Dover, 1992
• Applied Linear Algebra by B. Noble and J.W.Daniel, Prentice-Hall, 1988
• Finite Dimensional Vector Spaces by P.R.Halmos, D Van Nostrand, 1958
• Generalized Inverses by A.Ben-Israel and T.N.E.Greville, Wiley1974
• Matrix Computations by G.H.Golub & C.F.Van Loan, John Hopkins University Press, 1983 ISBN 0-946536-00-7/05-8
• Matrix Methods in Stability Theory by S.Barnett and C.Storey, Nelson, 1970