Handling Permission denied error on HDFS

When you try creating a folder or a file on HDFS you may hit the following error:

ubuntu@HADOOP_CLUSTER:~$ hdfs dfs -mkdir /abc

mkdir: Permission denied: user=ubuntu, access=WRITE, inode=”/user”:hdfs:hadoop:drwxr-xr-x
This particuler problem could happen because current logged user may not be part of hadoop group.

So lets find out who is the current logged user:

ubuntu@HADOOP_CLUSTER:~$ whoami
ubuntu

Now we can find out the list of groups the user “ubuntu” is associated with:

ubuntu@HADOOP_CLUSTER:~$ groups ubuntu
ubuntu : ubuntu adm dialout cdrom floppy audio dip video plugdev netdev admin

As we can see above that user ubuntu is not part of hadoop group so we will go ahead and add this user to hadoop group:

ubuntu@HADOOP_CLUSTER:~$ sudo adduser ubuntu hadoop
Adding user `ubuntu’ to group `hadoop’ …
Adding user ubuntu to group hadoop
Done.

Now we can try the exact same command we tried before:

ubuntu@HADOOP_CLUSTER:~$ hdfs dfs -mkdir /abc

The command is successfull and if we list files and folder at HDFS we can find our folder as below:

ubuntu@HADOOP_CLUSTER:~$ hdfs dfs -ls /
Found 5 items
drwxr-xr-x – ubuntu hadoop 0 2013-06-05 05:58 /abc
-rw-r–r– 1 ubuntu hadoop 28865 2013-06-04 16:14 /history.log
drwxr-xr-x – mapred hadoop 0 2013-06-04 15:56 /home
drwxrwxrwt – mapred hadoop 0 2013-06-05 05:48 /tmp
drwxr-xr-x – hdfs hadoop 0 2013-06-05 05:53 /user

To make sure that user “ubuntu” is part of “hadoop” group we can verify as below:

ubuntu@HADOOP_CLUSTER:~$ groups ubuntu
ubuntu : ubuntu adm dialout cdrom floppy audio dip video plugdev netdev admin hadoop

Note: IF above command does not work then you can use the following:

$ sudo -u hdfs hdfs dfs -mkdir /user/newfolder

Keywords: Hadoop, HDFS, Error

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s