All Hadoop commands are invoked by the
When you run these commands, you can specify the MapReduce mode in two different ways:
hadoopkeyword and specify the mode explicitly, where
classicmode refers to Hadoop 1.x and
yarnmode refers to Hadoop 2.x.
hadoop2keyword and do not specify the mode.
For example, the following commands are equivalent:
root@testnode100:/opt/mapr/conf/conf.d# hadoop2 conf | grep mapreduce.map.memory.mb <property><name>mapreduce.map.memory.mb</name><value>1024</value></property> root@testnode100:/opt/mapr/conf/conf.d# hadoop -yarn conf | grep mapreduce.map.memory.mb <property><name>mapreduce.map.memory.mb</name><value>1024</value></property>
The following syntax summary applies to all commands:
hadoop [-yarn|-classic] [--config confdir] [COMMAND] [GENERIC_OPTIONS] [COMMAND_OPTIONS] hadoop1 [--config confdir] [COMMAND] [GENERIC_OPTIONS] [COMMAND_OPTIONS] hadoop2 [--config confdir] [COMMAND] [GENERIC_OPTIONS] [COMMAND_OPTIONS]
Hadoop has an option parsing framework that employs parsing generic options as well as running classes.
Specifies the Hadoop version:
Aternatively, you can use a
If you use a
If this variable is not set, the command runs in the mode set in the hadoop version file on the node (
Overwrites the default Configuration directory. Default is
Various commands with their options are described in the following sections.
The common set of options supported by multiple commands.
Various command options are described in the following sections.
MapR supports the following
hadoop commands for Hadoop 1.x:
Manipulates MapReduce jobs.
Runs a MapReduce admin client.
Runs a pipes job.
Gets information about job queues.
MapR supports the following
hadoop commands for Hadoop 2.x:
Creates a Hadoop archive, a file that contains other files. A Hadoop archive always has a
The hadoop script can be used to invoke any class.
Prints the class path needed to access the Hadoop JAR and the required libraries.
For Hadoop2, some
For example, if you run the
hadoop job command, you see this message:
# hadoop job DEPRECATED: Use of this script to execute mapred command is deprecated. Instead, use the mapred command for it.
The syntax for the
mapred command is:
mapred [--config confdir] COMMAND
Commands used with
|Runs job history servers as a standalone daemon|
|The job history server admin interface|
|Manipulates MapReduce jobs|
|Runs a |
|Gets information regarding |
MapR does not support the following Hadoop commands:
Implement the Tool interface to make the following command-line options available for many of the Hadoop commands.
The following generic options are supported by the distcp, fs, job, mradmin, pipes, and queue Hadoop commands:
Add the specified configuration files to the list of resources available in the configuration.
Set a value for the specified Hadoop configuration property.
Set the URI of the default filesystem.
Specify a jobtracker for a given host and port. This command option is a shortcut for
Specify files to be copied to the map reduce cluster.
Specify JAR files to be included in the classpath of the mapper and reducer tasks.
Specify archive files (JAR, tar, tar.gz, ZIP) to be copied and unarchived on the task node.