These are some of the concepts in maths, which I should know but sadly I keep forgeting them so here they are:

Integration

1. Integratio by parts: [Wiki]

Matrices

1. Useful identity involving the matrix inverse (from Bishop eq: C.5)
(P$^{-1}$ + B$^T$R$^{-1}$B)$^{-1}$B$^T$R$^{-1}$ = PB$^T$(BPB$^T$ + R)$^{-1}$
Proof
Consider above LHS : RHS
  • Multiply both side by R to the right. For LHS take R as R$^{-1^{-1}}$, and then use formula (AB)$^{-1}$ = B$^{-1}$A$^{-1}$. This would give:
    (P$^{-1}$ + B$^T$R$^{-1}$B)$^{-1}$B$^T$ = PB$^T$(R$^{-1}$BPB$^T$ + I)$^{-1}$
  • Now do the similar thing by multiplying P$^-1$ to the left, to get:
    (I + B$^T$R$^{-1}$BP)$^{-1}$B$^T$ = B$^T$(R$^{-1}$BPB$^T$ + I)$^{-1}$
  • We have B$^T$ on both side, we can write that as B$^{T^{-1^{-1}}}$, and then again apply (AB)$^{-1}$ = B$^{-1}$A$^{-1}$ to get:
    (B$^{T^{-1}}$ + R$^{-1}$BP)$^{-1}$ = (R$^{-1}$BP + B$^{T^{-1}}$)$^{-1}$
  • Thus, LHS = RHS
Special case (Bishop C.6): (I + AB)$^{-1}$A = A (I + BA)$^{-1}$
2. Woodbury identity (these are usefula s depending on dimension it is cheaper to evaluate one side than the other) (Bishop C.7):
(A + BD$^{-1}$C)$^{-1}$ = A$^{-1}$ - A$^{-1}$B(D = CA$^{-1}$B)$^{-1}$CA$^{-1}$
3. Cyclic property of trace operator for matrices (Bishop C.9)
Tr(ABC) = Tr(CAB) = Tr(BCA)