Matrix Expectations

In all the expressions below, x is a vector of random variables with whose mean vector and covariance matrix are given by: E(x) = m and E((x-m)(x-m)^T) = S.

The expressions for cubic and quartic expectations are restricted to two special cases:

• [x:Independent] means that the components of x are independent. In particular, we require that E(x(i)^px(j)^q)=E(x(i)^p)E(x(j)^q). We define m_r=E((x-m)^r) where the r’th power is elementwise. Note that S=DIAG(m₂).
• [x:Gaussian] means that the components of x are Real and have a multivariate Gaussian pdf: (2*pi)^-n/2 |S|^-1/2exp( (x-m)^T S^-1 (x-m) ) where x has dimension n. If x is both Gaussian and Independent then m_k = diag( (S/2)^(k/2) * k! / (k/2)!)

Vectors and matrices a, A, b, B, c, C, d and D are constant (i.e. not dependent on x).

General Properties

• The covariance matrix S is Hermitian and positive semi-definite.
• S is strictly positive definite unless there is a deterministic relation between the elements of x of the form a'x = 0 for some non-zero a.
• If the elements of x are uniformly spaced samples from a continuous signal, then S is Toeplitz.
• E(tr(Y)) = tr(E(Y)) where Y depends on x.

Linear Expectations

• E(Ax + b) = Am + b
• • E(Ax) = Am
  • E(x + b) = m + b

Quadratic Expectations

• E((Ax + a)(Bx + b)^T) = ASB^T + (Am+a)(Bm+b)^T
• • E(xx^T) = S + mm^T
  • E(xa^T x) = (S + mm^T)a
  • E(x^T ax^T) = a^T(S + mm^T)
  • E((Ax)(Ax)^T) = A(S + mm^T)A^T
  • E((x + a)(x + a)^T) = S + (m+a)(m+a)^T
• E((Ax+a)^T (Bx+b)) = tr(ASB^T) + (Am+a)^T (Bm+b)
• • E(x^T x) = tr(S) + m^T m
  • E(x^TAx) = tr(AS) + m^TAm
  • E((Ax)^T (Ax)) = tr(ASA^T) + (Am)^T (Am)
  • E((x+a)^T (x+a)) = tr(S) + (m+a)^T (m+a)

Cubic Expectations

For [x:Independent] :

• E((Ax + a)(Bx + b)^T (Cx + c)) = A DIAG(B^T C) m₃ + ASB^T (Cm+c) + ASC^T (Bm+b) + tr(BSC^T)*(Am+a) + (Am+a)(Bm+b)^T (Cm+c)
• • E(xx^T x) = m₃ + 2Sm + (tr(S)+ m^T m)* m
  • E((Ax + a)(Ax + a)^T(Ax + a)) = A DIAG(A^T A) m₃ + (2ASA^T + (Am+a)(Am+a)^T)(Am+a) + tr(ASA^T) * (Am+a)
• E((Ax + a)b^T(Cx + c)(Dx + d)^T ) = ?
  • E(xa^Txx^T) = ?

For [x:Gaussian] :

• E((Ax + a)(Bx + b)^T(Cx + c)) = ASB^T(Cm+c) + ASC^T(Bm+b) + tr(BSC^T)*(Am+a) + (Am+a)(Bm+b)^T(Cm+c)
• • E(xx^Tx) = 2Sm + (tr(S)+ m^Tm)* m
  • E((Ax + a)(Ax + a)^T(Ax + a)) = (2ASA^T + (Am+a)(Am+a)^T)(Am+a) + tr(ASA^T) * (Am+a)
• E((Ax + a)b^T(Cx + c)(Dx + d)^T ) = ?
  • E(xb^Txx^T) = ?

Quartic Expectations

For [x:Independent] :

For [x:Gaussian] :

• E((Ax + a)(Bx + b)^T(Cx + c) (Dx + d)^T) = (ASB^T+(Am+a)(Bm+b)^T)(CSD^T+(Cm+c) (Dm+d)^T) + (ASC^T+(Am+a)(Cm+c)^T)(BSD^T+(Bm+b) (Dm+d)^T + (Bm+b)^T(Cm+c)* (ASD^T - (Am+a)(Dm+d)^T) + tr(BSC^T)*(ASD^T + (Am+a)(Dm+d)^T)
• • E(xx^Txx^T) = 2(S+mm^T)^2 + m^Tm* (S - mm^T) + tr(S)* (S + mm^T)
  • E(xx^TAxx^T) = (S+mm^T)(AS+Amm^T) + (SA^T+mAm^T)(S+mm^T) + m^TAm * (S - mm^T) + tr(AS)*(S + mm^T)
  • • [m=0]: E(xx^TAxx^T) = SAS + SA^TS + tr(AS)*S
  • E((x^TAx) * xx^T) =(S+mm^T)(AS+Amm^T) + (SA^T+mAm^T)(S+mm^T) + m^TAm * (S - mm^T) + tr(AS)*(S + mm^T)
  • • [m=0]: E((x^TAx) * xx^T) =SAS + SA^TS + tr(AS)*S
  • E((Ax + a)(Ax + a)^T(Ax + a) (Ax + a)^T) = 2(ASA^T+(Am+a)(Am+a)^T)² + (Am+a)^T(Am+a)* (ASA^T - (Am+a)(Am+a)^T) + tr(ASA^T)*(ASA^T + (Am+a)(Am+a)^T)
• E((Ax + a)^T(Bx + b) (Cx + c)^T(Dx + d)) = tr(AS(C^TD+D^TC)SB^T) + ((Am+a)^TB + (Bm+b)^TA)S(C^T(Dm+d) + D^T(Cm+c)) + (tr(ASB^T)+(Am+a)^T(Bm+b))(tr(CSD^T)+(Cm+c)^T(Dm+d))
• • E(x^Txx^Tx) = 2tr(S²) + 4m^TSm + (tr(S) + m^Tm)²
  • E(x^TAxx^TBx) = tr(AS(B+B^T)S) + m^T(A + A^T)S(B + B^T)m + (tr(AS)+m^TAm)(tr(BS)+m^TBm)

• • [m=0]: E(x^TAxx^TBx) = tr(AS(B+B^T)S) + tr(AS)*tr(BS)
• E(a^Txb^Txc^Txd^Tx) = (a^T(S+mm^T)b)(c^T(S+mm^T)d)+(a^T(S+mm^T)c)(b^T(S+mm^T)d)+(a^T(S+mm^T)d)(b^T(S+mm^T)c)-2a^Tmb^Tmc^Tmd^Tm
• E((Ax + a)^T(Ax + a) (Ax + a)^T(Ax + a)) = 2tr(ASA^TASA^T) + 4(Am+a)^TASA^T(Am+a) + (tr(ASA^T) + (Am+a)^T(Am+a))²