Programmatically retrieving Task ID and Unique Reducer ID in MapReduce

For each Mapper and Reducer you can get Task attempt id and Task ID both. This can be done when you set up your map using the Context object. You may also know that the when setting a Reducer an unique reduce ID is used inside reducer class setup method. You can get this ID as well.

There are multiple ways you can get this info:

1. Using JobConf Class.

  • JobConf.get(“mapred.task.id”) will provide most of the info related with Map and Reduce task along with attempt id.

2. You can use Context Class and use as below:

  • To get task attempt ID – context.getTaskAttemptID()
  • Reducer Task ID – Context.getTaskAttemptID().getTaskID()
  • Reducer Number – Context.getTaskAttemptID().getTaskID().getId()

Keyword: Hadoop, Map/Reduce, Jobs Performance, Mapper, Reducer

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s