Listing current running Hadoop Jobs and Killing running Jobs

When you have jobs running in Hadoop, you can use the map/reduce web view to list the current running jobs however what if you would need to kill any current running job because the submitted jobs started malfunctioning or in worst case scenario, the job is stuck in infinite loops. I have seen several scenarios when a submitted job got stuck in problematic states due to code defect in map/reduce job or the Hadoop cluster itself. In any of such situation, you would need to manually kill the job which is already started.

To kill a currently running Hadoop job first you need Job ID and then Kill the job using the as following commands:

  • Hadoop job -list
  • Hadoop job –kill <JobID>

To list current running job in Hadoop Command shell please use below command:

On Linux:      $ bin/hadoop job –list
On Windows:    HADOOP_HOME = C:AppsDist
               HADOOP_HOMEbinHadoop job list

Above command will return job details as below:

[Linux]

1 jobs currently running
JobId                  State          StartTime       UserName
job_201203293423_0001   1             1334506474312   avkash

[Windows]
c:appsdist>hadoop job -list
1 jobs currently running
JobId                  State   StartTime       UserName        Priority        SchedulingInfo
job_201204011859_0002   1       1333307249654   avkash       NORMAL          NA

Once you have the JobID you can use the following command to kill the job:

 
On Linux:      $ bin/hadoop job -kill jobid
On Windows:    HADOOP_HOME = C:AppsDist
               HADOOP_HOMEbinHadoop job –kill <Job_ID>

[Windows]
c:appsdist>hadoop job -kill job_201204011859_0002
Killed job job_201204011859_0002
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s