Page 1 of 1
Question 6 {15 points): A Hadoop cluster is built from commodity hardware that frequently fails. Thus, fault tolerance i
Posted: Sun May 15, 2022 2:03 pm
by answerhappygod
- Question 6 15 Points A Hadoop Cluster Is Built From Commodity Hardware That Frequently Fails Thus Fault Tolerance I 1 (36.11 KiB) Viewed 40 times
Question 6 {15 points): A Hadoop cluster is built from commodity hardware that frequently fails. Thus, fault tolerance is critical for data storage and distribution computation on a Hadoop cluster. You have a Hadoop cluster with a manager node and 1000 worker nodes. You have installed HDFS on this Hadoop cluster to store large files on the disks of the worker nodes (a.k.a. data nodes in HDFS) and you also have set up the MapReduce framework to run MapReduce jobs using the CPUs and RAM of the worker nodes (a.k.a. task trackers in MapReduce). Suppose that, when you were running a MapReduce job on this Hadoop cluster, one of the used worker nodes failed. Please describe how HDFS automatically recovers from this failed worker node. Please also describe how the MapReduce framework continues the execution of your MapReduce job by minimizing disruption caused by this failed worker node.