Answer Happy

In class, we have learnt that the Ridge Regression can alleviate the overfitting problem by mini- mizing the objective function using L2 regularization Gridge = arg -X0||2 + 1||0|13 In this question, we explore the Lasso Regression by minimizing the following objective function using Lį regularization glasso = arg -X8|2 + $||0||1 m where the L2 ridge penalty || 0 || 3 = = 0; is replaced by the Lį lasso penalty ||0|| 1 Σ2 |θη. (a) Suppose XTX is the identity matrix I. Compute the estimate of 0 under the lasso penalty. Hint : the objective function is not smooth any more with respect to 0. Thus, when calculating the derivative, we can take the three cases (@j > 0, 4; < 0, and di 0) of the variable (i, Vi into consideration, separately. (b) With the assumption XTX = I, compare the estimation given by Ll and L2 regu- larization to the original estimation 0 = (XTX)-°XTY of the linear regression. Explain the difference in one or two sentences.