Ij E Hw6 Movie Counterdataset Open 1 Hw7 2 3 Import Org Apache Spark Sql 4 Import Org Apache Log4j 5 Import Org Apa 1 (66.4 KiB) Viewed 44 times
Ij E Hw6 Movie Counterdataset Open 1 Hw7 2 3 Import Org Apache Spark Sql 4 Import Org Apache Log4j 5 Import Org Apa 2 (59.25 KiB) Viewed 44 times
IJ E HW6 Movie CounterDataset Open 1//HW7 2 3 import org.apache.spark.sql._ 4 import org.apache.log4j._ 5 import org.apache.spark.sql.types. [Integer Type, LongType, StructType} 6 -/Downloads 15 16 17 8 // Create case class with schema of u.data 10 // user ID: Int, movieID: Int, rating: Int, timestamp: Long 11 12 13 // Create schema when reading u.data 14 7//Count up how many of each star rating exists in the MovieLens 100K data set. Save val userRatingsschema= new StructType() .add("userID", IntegerType, nullable = true) .add("movieID", IntegerType, nullable = true) .add("rating", IntegerType, nullable = true) .add("timestamp", LongType, nullable = true) 18 19 20 // Load up the data into spark dataset 21 // Use tab as separator "\t", 22// load schema from user Ratingsschema and 23// force case class to read it as dataset 24 25 26 // Select only ratings column 27 11 (The file format is useCTO movieID. ||| cating timestamol C Tab Width: 8 B Ln 1, Col 1 INS X
Open set. E HW6 Movie CounterDataset -/Downloads 8 9 // Create case class with schema of u.data 10 // userID: Int, movieID: Int, rating: Int, timestamp: Long 11 12 13 // Create schema when reading u.data 14 15 16 17 18 19 20 // Load up the data into spark dataset 21 // Use tab as separator "\t", val user Ratings Schema= new StructType() .add("userID", IntegerType, nullable = true) .add("movieID", IntegerType, nullable = true) .add("rating", IntegerType, nullable = true) .add("timestamp", LongType, nullable = true) Save 22 // Load schema from user Ratingsschema and 23 // force case class to read it as dataset 24 25 26 // Select only ratings column 27 // (The file format is user ID, movieID, rating, timestamp) 28 31 // Sort the resulting dataset by count column 32 33 // Print results from the dataset 34 C Tab Width: 8 ||| 29 // Count up how many times each value (rating) occurs using grougBy and count 30 D Ln 1, Col 1 INS
Join a community of subject matter experts. Register for FREE to view solutions, replies, and use search function. Request answer by replying!