Shae's Ramblings

This post is about a multivariate version of the usual Laplace approximation of a partition function. Suppose one has a parametric distribution \(p_\theta\) given by

$$ p_\theta(\mathrm{x}) = \dfrac{1}{Z(\theta)}\exp\left(-T(\mathrm{x}) + \theta^\mathsf{T}\mathrm{x}\right) $$

and one wishes to compute the partition function $$Z(\theta) \triangleq \int_\mathcal{X}\exp\left(-T(\mathrm{x}) + \theta^\mathsf{T}\mathrm{x}\right)\,\mathrm{dx}$$

Then, given a saddle point \(\mathrm{x}^\star\) for which \(\nabla_\mathrm{x}T(\mathrm{x}^\star) = 0\), the quadratic approximation of \(T(\cdot)\) reads

$$T(\mathrm{x}) \approx T(\mathrm{x}^\star) + (\mathrm{x} – \mathrm{x}^\star)^\mathsf{T}\nabla_\mathrm{x}^2T(\mathrm{x}^\star)(\mathrm{x} – \mathrm{x}^\star) $$ where we will use the notation \(\mathrm{H}_\star \triangleq \nabla_\mathrm{x}^2T(\mathrm{x}^\star)\) for the hessian to simplify notation.

To be added.