By default, files are created in /tmp/hadoop- folder. This formats the hdfs file system and configures it on the local system. Run the following command to initialize the metadata for the hadoop cluster. If disk utilization goes above the configured threshold, yarn will report the node instance as unhealthy nodes with error 'local-dirs are bad'.įrom a terminal window switch to the hadoop home folder (the folder which contains various sub folders such as bin and etc). This was required in my system since my disk utilization was 95% and the default value for this is 90%. This tells yarn to continue operations when disk utilization is below 98.5. Note the use of disk utilization threshold above. JAVA_HOME, HADOOP_COMMON_HOME, HADOOP_HDFS_HOME, HADOOP_CONF_DIR, CLASSPATH_PREPEND_DISTCACHE, HADOOP_YARN_HOME, HADOOP_MAPRED_HOME Modify various hadoop configuration files to properly setup hadoop and yarn. This is for archiving purposes, just in case Apple decides to remove the download, so people can still download this in the future. export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/Home This is the Install DMG of OS X El Capitan, downloaded from Apple's website just before this upload. usr/libexec/java_home /Library/Java/JavaVirtualMachines/jdk1.8.0_121.jdk/Contents/HomeĬopy the output of the command and use it to configure JAVA_HOME variable in etc/hadoop/hadoop-env.sh.
