Directions and Overview
The main purpose of this assignment is for you to gainexperience using tree-based methods to solve simple regressionproblems. In this assignment, you will fit a Gradient-BoostedRegression Tree, a Random Forest, and a DecisionTree to a noisy 3D sinusoidal data set. Since these models canbe trained very quickly on the supplied data, I want you to firstmanually adjust hyperparameter values and observe their influenceon the model's predictions. That is, you should manually sweep thehyperparameter space and try to hone in on the optimalhyperparameter values, again, manually. (Yep, thatmeans guess-and-check: pick some values, train the model, observethe prediction curve, repeat.)
But wait, there's more! Merely attempting to identify theoptimal hyperparameter values is not enough. Be sure to really geta visceral understanding of how altering a hyperparameter in turnalters the model predictions (i.e. the prediction curve). This ishow you will build your machine learning intuition!
So, play around and build some models. When you are done playingwith hyperparameter values, you should try to set these values tothe optimal values manually (you're likely going tobe way off). Then, retrain the model. Next inthis assignment, we will perform several grid searches, so you'llbe able to compare your "optimal" hyperparameter values with thosecomputed from the grid search.
We will visualize model predictions for theoptimal Gradient-Boosted Regression Tree, a RandomForest, and Decision Tree models that were determined bythe grid searches. Next, you will compute the generalization erroron the test set for the three models.
- I Am Working On This Assignment And Seems Not To Be Able To Pass This One Spot I Am Getting Crazy Looking Curve The As 1 (345.41 KiB) Viewed 51 times
def plot3Ddata (data): scat_x = X_train[ 'x'] scat_y= X_train[ 'y'] scat_z = z_train ['z'] fig = plt.figure(figsize=[15,15]) ax1 = fig.add_subplot (1,2,1, projection = '3d') xLabel = axl.set_xlabel('x', fontsize=20, color='r') yLabel = axl.set_ylabel('y', fontsize=20, color='r') zLabel = axl.set_zlabel('z', fontsize=20, color='r') plt.xlim (0,14) plt.ylim(-6,6) axl.view_init(1,90) axl.scatter3D(scat_x, scat_y, scat_z, c=scat_z, cmap ="Blues") plt.show() fig= plt.figure (figsize=[15, 15]) ax2 = fig.add_subplot (1,2,2, projection = '3d') xLabel = ax2.set_xlabel('x', fontsize=20, color='r') yLabel = ax2.set_ylabel('y', fontsize=20, color='r') zLabel = ax2.set_zlabel('z', fontsize=20, color='r') plt.xlim (0,14) plt.ylim(-6,6) ax2.view_init(38,1) ax2.scatter3D(scat_x, scat_y, scat_z, c=scat_z, cmap ="Blues") plt.show() fig= plt.figure (figsize=[15, 15]) ax3 = fig.add_subplot (2,2,1, projection = '3d') xLabel = ax3.set_xlabel('x', fontsize=20, color='r') yLabel = ax3.set_ylabel('y', fontsize=20, color='r') zLabel = ax3.set_zlabel('z', fontsize=20, color='r') plt.xlim (0,14) plt.ylim(-6,6) ax3.view_init(47,37) ax3.scatter 3D (scat_x, scat_y, scat_z, c=scat_z, cmap ="Blues") plt.show() fig= plt.figure(figsize=[15,15]) ax4 = fig.add_subplot (2,2,2, projection = '3d') xLabel = ax4.set_xlabel('x', fontsize=20, color='r') yLabel = ax4.set_ylabel('y', fontsize=20, color='r') zLabel = ax4.set_zlabel('z', fontsize=20, color='r') plt.xlim (0,14) plt.ylim(-6,6) ax4.view_init(24,22) ax4.scatter3D(scat_x, scat_y, scat_z, c=scat_z, cmap ="Blues") plt.show()
def plotscatter 3Ddata (fit_x, fit_y, fit_z, scat_x, scat_y, scat_z): fig= plt.figure (figsize=[15,15]) ax11 = fig.add_subplot (1,2,1, projection = '3d') xLabel = ax11.set_xlabel('x', fontsize=20, color='r') yLabel = ax11.set_ylabel('y', fontsize=20, color='r') zLabel = axll.set_zlabel('z', font size=20, color='r') plt.xlim (0,14) plt.ylim(-6,6) axl1.view_init(1,90) ax11.scatter 3D (scat_x, scat_y, scat_z, c=scat_z, cmap ="Blues") ax11.plot3D(fit_x, fit_y, fit_z, 'black') plt.show() fig plt.figure (figsize=[15,15]) ax12= fig.add_subplot (1,2,2, projection = '3d') xLabel = ax12.set_xlabel('x', fontsize=20, color='r') yLabel = ax12.set_ylabel('y', font size=20, color='r') zLabel = ax12.set_zlabel('z', fontsize=20, color='r') plt.xlim (0,14) plt. ylim(-6,6) ax12.view_init(38,1) ax12.scatter3D(scat_x, scat_y, scat_z, c=scat_z, cmap ="Blues") ax12.plot3D (fit_x, fit_y, fit_z, 'black') plt.show() fig= ax13 plt.figure(figsize=[15,15]) fig.add_subplot (2,2,1, projection = '3d') xLabel = ax13.set_xlabel('x', fontsize=20, color='r') yLabel = ax13.set_ylabel('y', fontsize=20, color='r') zLabel = ax13.set_zlabel('z', font size=20, color='r') plt.xlim (0,14) plt. ylim(-6,6) ax13.view_init(47,37) ax13.scatter3D(scat_x, scat_y, scat_z, c=scat_z, cmap ="Blues") ax13.plot3D(fit_x, fit_y, fit_z, 'black') plt.show() fig = plt.figure(figsize=[15,15]) ax14 = fig.add_subplot (2,2,2, projection = '3d') xLabel = ax14.set_xlabel('x', fontsize=20, color='r') yLabel = ax14.set_ylabel('y', fontsize=20, color='r') zLabel = ax14.set_zlabel('z', font size=20, color='r') plt.xlim (0,14) plt.ylim(-6,6) ax14.view_init(24,22) ax14.scatter3D(scat_x, scat_y, scat_z, c=scat_z, cmap ="Blues") ax14.plot3D(fit_x, fit_y, fit_z, 'black') plt.show()
Import and Split Data Complete the following: [3]: 1. Begin by importing the data from the file called 3DSinusoidal.csv. Name the returned DataFrame data. 2. Call train_test_split() with a test_size of 20%. x and y will be your feature data and z will be your response data. Save the output into X_train, X_test, z_train, and z_test, respectively. Specify the random_state parameter to be 42 (do this throughout the entire note book). [2]: from sklearn.model_selection import train_test_split data = pd. read_csv ("3DSinusoidal.csv") X = data[['x', 'y'll z = data[['z']] X_train, X_test, z_train, z_test= train_test_split(X,Z, Plot Data Simply plot your training data here, so that you know what you are working with. You must define a function called plot 3Ddata, which accepts a Pandas DataFrame (composed of 3 spatial coordinates) and uses scatter3D() to plot the data. Use this function to plot only the training data (recall that you don't even want to look at the test set, until you are ready to calculate the generalization error). You must place the definition of this function in the existing code cell of the above Preliminaries section, and have nothing other than the function invocation in the below cell. test_size = 0.2, random_state = 42) You must emulate the graphs shown in the respective sections below. Each of the graphs will have four subplots. Note the various viewing angles that each subplot presents - you can achieve this with the view_init() method. Be sure to label your axes as shown. #train_df = pd. DataFrame (X_train[ 'x'],X_train[ 'y'],z_train[ 'z']), scat_x = X_train[ 'x'] scat_y=X_train[ 'y'] scat_zz_train[ 'z'] plot 3Ddata (data)
A Quick Note In the following sections you will be asked to plot the training data along with the model's predictions for that data superimposed on it. You must write a function called plotscatter 3Ddata (fit_x, fit_y, fit_z, scat_x, scat_y, scat_z) that will plot this figure. The function accepts six parameters as input, shown in the function signature. All six input parameters must be NumPy arrays. The Numpy arrays called fit_x and fit y represent the x and y coordinates from the training data and fit_z represents the model predictions from those coordinates (i.e. the prediction curve). The three Numpy arrays called scat_x, scat_y, and scat_z represent the x, y, and z coordinates of the training data. You must place the definition of the plotscatter 3Ddata (fit_x, fit_y, fit_z, scat_x, scat_y, scat_z) function in the existing code cell of the above Preliminaries section. (The function header is already there - you must complete the function definition.) You will use the plotscatter 3Ddata() function in each of the below Plot Model Predictions for Training Set portion of the three Explore 3D Data sections, as well as the Visualize Optimal Model Predictions section. Important: Below, you will be asked to plot the model's prediction curve along with the training data. Even if you correctly train the model, you may find that your trendline is very ugly when you first plot it. If this happens to you, try plotting the model's predictions using a scatter plot rather than a connected line plot. You should be able to infer the problem and solution with the trendline from examining this new scatter plot of the model's predictions. Explore 3D Data: GradientBoosting Regressor Fit a GradientBoostingRegressor model to this data. You must manually assign values to the following hyperparameters. You should "play around" by using different combinations of hyperparameter values to really get a feel for how they affect the model's predictions. When you are done playing, set these to the best values you can for submission. (It is totally fine if you don't elucidate the optimal values here; however, you will want to make sure your model is not excessively overfitting or underfitting the data. Do this by examining the prediction curve generated by your model. You will be graded, more exactly, on the values that you calculate later from performing several rounds of grid searches.) learning_rate= <value> max_depth = <value> n_estimators= <value> random_state = 42
from sklearn.ensemble import Gradient BoostingRegressor gbrt Gradient BoostingRegressor (max_depth = 2, n_estimators = 50, learning_rate= 0.1, random_state = 42) gbrt.fit (X_train, z_train) fit_z = gbrt.predict (X_train) #X_train = pd.DataFrame() #x=X_train.values #fit_x = x #y=X_train.values #fit y = y #train_dfl = pd.concat([X_train, fit_z], axis=1) #sorted_df =train_df1.sort_values (['x'], ascending=True) # sort values by column x` fit_xX_train[ 'x'] fit_y=X_train[ 'y'] DataConversionWarnin /Users/jelenabreedlove/opt/anaconda3/lib/python3.9/site-packages/sklearn/utils/validation.py:63: g: A column-vector y was passed when a ld array was expected. Please change the shape of y to (n_samples, ), for exam ple using ravel(). return f(*args, **kwargs) ### Plot Model Predictions for Training Set Use the plotscatter 3Ddata (fit_x, fit_y, fit_z, scat_x, scat_y, scat_z)` function to plot the data and the prediction curve. plotscatter 3Ddata (fit_x, fit_y, fit_z, scat_x, scat_y, scat_z) yo 14 12 10 8 6 4 2 0
Explore 3D Data: RandomForestRegressor Fit a RandomForest Regressor model to this data. You must manually assign values to the following hyperparameters. You should "play around" by using different combinations of hyperparameter values to really get a feel for how they affect the model's predictions. When you are done playing, set these to the best values you can for submission. (It is totally fine if you don't elucidate the optimal values here; however, you will want to make sure your model is not excessively overfitting or underfitting the data. Do this by examining the prediction curve generated by your model. You will be graded, more exactly, on the values that you calculate later from performing several rounds of grid searches.) min_samples_split = <value> max_depth = <value> n_estimators = <value> random_state = 42 In [] ### ENTER CODE HERE ### Plot Model Predictions for Training Set Use the plotscatter 3Ddata (fit_x, fit_y, fit_2, scat_x, scat_y, scat_z) function to plot the data and the prediction curve. In [ ] ### ENTER CODE HERE ### Explore 3D Data: Decision TreeRegressor Fit a DecisionTreeRegressor model to this data. You must manually assign values to the following hyperparameters. You should "play around" by using different combinations of hyperparameter values to really get a feel for how they affect the model's predictions. When you are done playing, set these to the best values you can for submission. (It is totally fine if you don't elucidate the optimal values here; however, you will want to make sure your model is not excessively overfitting or underfitting the data. Do this by examining the prediction curve generated by your model. You will be graded, more exactly, on the values that you calculate later from performing several rounds of grid searches.) splitter = <value> max_depth = <value> min_samples_split = <value> random_state = 42 In [ ] # ## ENTER CODE HERE ### Plot Model Predictions for Training Set Use the plotscatter3Ddata (fit_x, fit_y, fit_2, scat_x, scat_y, scat_z) function to plot the data and the prediction curve. In [ ] ### ENTER CODE HERE ###