- Problem 1 5 Pt With The Given Wine Dataset I E Wine Data Perform The Expectation Maximization Em Algorithm F 1 (112.14 KiB) Viewed 45 times
- Problem 1 5 Pt With The Given Wine Dataset I E Wine Data Perform The Expectation Maximization Em Algorithm F 2 (68.95 KiB) Viewed 45 times
- Problem 1 5 Pt With The Given Wine Dataset I E Wine Data Perform The Expectation Maximization Em Algorithm F 3 (142.3 KiB) Viewed 45 times
Problem 1. (5 pt) With the given wine dataset (i.e., "wine.data"), perform the Expectation-Maximization (EM) algorithm for fitting a mixture of 3 Gaussians. In fact, data in the dataset is 178 X (1+13) consisting of 178 13-featured samples of 3 different wine cultivars. (Hint: See pages 59, 60, 79, and 82 in lecture note 5.) 1) Apply the PCA process developed in the HW2 to the data for dimensionality reduction. In other words, project the 13-featured data to the top 2 principal directions. Here, the number of clusters is 3 different cultivars. (Hint: The pre-processing part (i.e., PCA, here) is provided inside the codes. Otherwise, the machine learning toolbox, built-in functions, and open-sourced libraries are allowed for this pre-processing part ONLY.) 2) Create an EM algorithm by your own words in MATLAB. Fill in the blanks of the code in the given "test_wine.m" file. Any machine learning toolbox and the built-in functions (e.g., gmdistribution (mu,sigma), fitgmdist(X,k), and cluster(gm, X)) are NOT allowed. (Hint: When initializing covariance Ex, add a regularization to make sure that it is not rank deficient and also positive definite.) 3) Test the EM iterations against the PCA data resulting from 1). Here, due to fitting a mixture of 3 Gaussians, K = 3. In addition, just choose to run 150 iterations. However, you can change the termination criterion for the loop to whether the solution changes big enough between two adjacent iterations. (Hint: Use the built-in function munpdf (PCA data, fk, Ex) to compute mixing proportion Thi in the E-step.) 4) Plot the followings on a single figure at every iteration to show an animation effect. 1
• Plot the PCA data points using mixing proportion as colors. The data point locations will not change over iterations, but the color may change. (Hint: Use built-in function scatter(---, 'filled').) • Plot the centers of the Gaussian. The 3 centers change locations at each iteration until the solution converges. (Hint: Use built-in function scatter(---, 'filled').) Draw the contour of a fitted mixture of Gaussian densities. First, evaluate the densities on the grid points, and then draw the contour. (Hint: Use a built-in function munpdf (grid points, lk, Ek) to evaluate the density and another built-in function contour(grid_r, grid_y, Z) that specifies the grid_x and grid_y coordinates for the values in Z.) 5) Analyze and discuss your results. Animated GIF (or video) submission is optional. 6) (Bonus 1 pt) Load the data file in Python. Do the same above on the scikit-learn library (i.e., sklearn. mixture) for validation. Compare the results.
Problem 2. (5 pt) With the given boys' and girls' pictures dataset (i.e., "boys.mat" and "girls.mat"), perform feature selection that finds out best pixels for classifying photos of boys and girls. In fact, the dimension of the raw data in the dataset is 4225 x 461 or 176, consisting of 65x65 pixels of all students' faces including 461 boys and 176 girls. (Hint: See pages 50, 51, and 52 in lecture note 6.) 1) Discretize the density using the KDE. Here, kernel bandwidth h = 1 and use Gaussian kernel function. (Hint: Use the built-in function pdist2() to compute pairwise distance between each feature and the grid points that is divided by 100 evenly from 0 to 1.) (Hint: The pre-processing part (i.e., KDE, here) is provided inside the codes. Otherwise, the machine learning toolbox, built-in functions, and open-sourced libraries are allowed for this pre-processing part ONLY.) 2) Create the Mutual Information (MI) algorithm for feature selection by your own words in MATLAB as follows: • Calculate joint density distribution between i-th feature and label/class, and then normalize the joint distribution p(x,y)(x, y). (Hint: joint distribution = marginal x conditional distribution.) • Calculate marginal distribution of i-th feature and label/class. That is, Marginal feature distribution px(x) and Marginal class distribution py(y). • Compute the cross product px(x) < py(y). • Mutual information I = ,y P(X,Y)(x, y) log PX (0) PY (9) Here, we define 0 x log 0 = 0. Basically, fill in the blanks of the code in the given "feature_selection-pixel_kde.m” file. Any machine learning toolbox and metafeatures() built-in function are NOT allowed. 3) Implement the MI algorithm against the given data. 4) Visualize feature mutual information. (Hint: Use the built-in function imagesc(-.-, 65, 65) that displays image with scaled colors.) 5) Analyze and discuss your results. 6) (Bonus 1 pt) Load the data file in Python. Do the same above on the scikit-learn library (i.e., sklearn. feature_selection.mutual_info_classif) for validation. Compare the results.