hadoop job command command enables you to manage MapReduce jobs.
hadoop job [Generic Options] [-submit <job-file>] [-status <job-id>] [-counter <job-id> <group-name> <counter-name>] [-kill <job-id>] [-unblacklist <job-id> <hostname>] [-unblacklist-tracker <hostname>] [-set-priority <job-id> <priority>] [-events <job-id> <from-event-#> <#-of-events>] [-history <jobOutputDir>] [-list [all]] [-list-active-trackers] [-list-blacklisted-trackers] [-list-attempt-ids <job-id> <task-type> <task-state>] [-kill-task <task-id>] [-fail-task <task-id>] [-blacklist-tasktracker <hostname>] [-showlabels]
The following command options are supported for for
Submits the job.
Prints the map and reduce completion percentage and all job counters.
Prints the counter value.
Kills the job.
Removes a tasktracker job from the jobtracker's blacklist.
Admin only. Removes the TaskTracker at
Changes the priority of the job. Valid priority values are
Prints the events' details received by jobtracker for the given range.
Prints job details, failed and killed tip details.
Prints all active tasktrackers.
Prints blacklisted tasktrackers. the TaskTracker nodes that JobTracker blacklisted with the reason for blacklisting.
Lists the IDs of task attempts.
Kills the task. Killed tasks are not counted against failed attempts.
Fails the task. Failed tasks are counted against failed attempts.
Pauses all current tasktracker jobs and prevent additional jobs from being scheduled on the tasktracker.
Dumps label information of all active nodes.
The following generic options are supported for the the
hadoop job command command:
-conf <configuration file>,
-fs <local|file system URI>,
-libjars <libjar1,libjar2,libjar3,...>, and and
-archives <archive1,archive2,archive3,...>. For more information on generic options, see see Generic Options.
hadoop job -submit command command enables you to submit a job to the specified jobtracker.
Stopping Jobs Gracefully
Use the the
hadoop kill command command to stop a running or queued job.
Viewing Job History Logs
Run the the
hadoop job -history command command to view the history logs summary in specified directory.
Additional details about the job such as successful tasks and task attempts made for each task can be viewed by adding the the
-all option option:
$ hadoop job -history all output-dir
hadoop job command command when run as root or using using
sudo can can be used to manually blacklist tasktrackers:
Manually blacklisting a tasktracker pauses any running jobs and prevents additional jobs from being scheduled.
For a detailed discussion see see TaskTracker Blacklisting.