Answer Happy

Importance Sampling
T Given a distribution p(x) on x = [X1, ... , Xp]' E RP. Suppose we want to perform inference Ep(x) [f(x)] using importance sampling, with q(x) as the pro- posal distribution. According to importance sampling, we draw L i.i.d. samples x(L) from q(x), and we have x(1) L 1 Extr16)*() Ep.)f (2) + (*u: ui i=1 x 9(x) when p = a. where the (unnormalized) importance weights ui P(20) 9(x(i)): a) Find the mean and variance of the unnormalized importance weights, i.e., Eq(z) [w] and Varg(z) [ui] b) Prove the following lemma: Ep(a) [@] > 1, and the equality holds only c) A measure of the variability of two components in vector u = (U1,..., UL]T is given by Eq(ar) [(u; u;)?]. Assume that both p and q can be factorized, i.e. p(x) = 112-1 Pi (ri) and g(x) = 11.-14: (Li). Show that Eq(e) [(u: – 4;)?) has Li ; exponential growth with respect to D. d) Use the conclusion in c) to explain why the standard importance sampling does not scale well with dimensionality and would blow up high-dimensional = = . i= cases.