Common Hadoop (YARN) Errors

While installing or running hadoop one gets different errors at different times. Here is a list of some that i could think of:

  1. class org.apache.hadoop.mapred.ShuffleHandler not found - This errors comes in the YARN nodemanager. This is due to fact that the system could not find YARN map-reduce folder. In CDH installation check for /usr/lib/hadoop-mapreduce folder. If its not there, there were installation problems. If its there, then its missing from classpath and other environment variables. hadoop-mapreduce is not to be confused with hadoop-mapreduce-0.20.x
  2. java.lang.NoClassDefFoundError: org/apache/hadoop/mapreduce/lib/partition/InputSampler$Sampler-  This is due to missing environment variable. Check HADOOPHOME, HADOOPMAPRED_HOME etc.
  3. ENOENT: No such file or directory at Method)- check HADOOPMAPREDHOME.
  4. ERROR org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Caught exception in status-updater - Check yarn-sites.xml for resource manager address, check if its
  5. File /path/to/file could only be replicated to 0 nodes instead of minReplication (=1) - Data Node disk is Full
  6. Data Node is Busy with block report and block scanning
  7. If Block Size is Negative value(dfs.block.size in hdfs-site.xml)
  8. Cannot initialise Cluster. Please check your configuration for and the correspond server addresses. - This error comes when you have configured hadoop YARN as your hadoop framework but you are trying to run an application developed for MRv1.
  9. Check HADOOPMAPREDHOME. For MRv1  it must be set to /usr/lib/hadoop-0.20-mapreduce folder.