Answer Happy

Question 1: 14 marks Consider the following model for the binary classification task with outcome Y € {0,1}. There are 2 features X1 and X2. The model is, P(Y = 1 | X1, X2) = 0(1+ w1X1 + w2X2) where 0 () is the sigmoid function (o(2) = ), and wi, and w2 are scalars. = 1 1+e A binary classifier is constructed by deciding Y = 1 if P(Y = 1 | X1, X2) > 0.5 and otherwise deciding Y = 0. (a) (4 marks] Show that this model leads to a linear (affine) boundary decision within the X1, X2 plane. (b) (4 marks] Consider a variation of the previous model by using the square of each feature such that P(Y = 1 | X1, X2) = 0(1+w_X+ w2X3), and the same binary classification is used as before. What is now the shape of the boundary decision (within the X1, X2 plane)? (c) [6 marks] Consider we observe m samples {(x1, yı),..., Im, Ym)} with r; = (Ili, x2i) € R2 and yi € {0,1}. To estimate the parameter of the model defined in (b), it is usual to consider the following cost function: = m 1 J(w) ń Llŷi, yi) m i=1 where Llŷi, yi) = -(y;logùi + (1 – yi)log(1 – û)) and ĝi = 0(1+wirii + W2r2i) (a) (2 marks] Define the gradient of the cost function (J(w)) (b) [2 marks] Define the hessian matrix of the cost function H = 02 (J(w)) (c) (2 marks] Show that the function J(w) is convex. (Hint: show that the hessian matrix is is positive semidefinite)