Lab 7: Hadoop Setup

In this lab, you’ll set up Apache Hadoop. This includes HDFS (Hadoop Distributed File System) and YARN (Yet Another Resource Negotiator), A.K.A. MapReduce 2.0.

Head over to the setup guide to get started.

A few hints:

• Make sure you have passwordless ssh set up first
• Confirm that your directory in /bigdata/students/$(whoami) exists
• There’s a script that will do most of the configuration work for you. You shouldn’t have to do much (if any) editing of the XML files.

Submission

After you’ve successfully set up HDFS, stored a file in it, and ran a yarn job (probably Word Count!), send me a screenshot to get checked off for the lab.