Handling Apache Hive Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask

Recently I hit the following  Error when starting Apache hive in my server:

ubuntu@HIVE_SERVER:~$ hive
log4j:ERROR Could not find value for key log4j.appender.DEBUG
log4j:ERROR Could not instantiate appender named “DEBUG”.
log4j:ERROR Could not find value for key log4j.appender.WARN
log4j:ERROR Could not instantiate appender named “WARN”.
log4j:ERROR setFile(null,true) call failed.
java.io.FileNotFoundException: /var/log/hive/hive.log (No such file or directory)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:212)
at java.io.FileOutputStream.<init>(FileOutputStream.java:136)
at org.apache.log4j.FileAppender.setFile(FileAppender.java:294)
at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165)
at org.apache.log4j.DailyRollingFileAppender.activateOptions(DailyRollingFileAppender.java:223)
at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)
at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104)
at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:842)
at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768)
at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514)
at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580)
at org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:415)
at org.apache.hadoop.hive.common.LogUtils.initHiveLog4j(LogUtils.java:52)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:629)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:613)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
log4j:ERROR Either File or DatePattern options are not set for appender [DRFA].
Logging initialized using configuration in file:/etc/hive/conf.dist/hive-log4j.properties
Hive history file=/tmp/ubuntu/hive_job_log_ubuntu_201306111657_1917799460.txt
hive> show tables;
FAILED: Error in metadata: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
hive> quit;

Potential Root Cause:

This is mainly because the I have had previous hive installation in the machine and uninstalled it. Not all the files were removed completelty and the /var/log/hive/hive.log file was not there and most of the settings were messed up.

Even when I created an empty /var/log/hive/hive.log ($touch /var/log/hive/hive.log and chmod 777 /var/log/hive/hive.log) still the problem persisted.

Removing Hive:

ubuntu@HIVE_SERVER:~$ sudo apt-get remove hive
Reading package lists… Done
Building dependency tree
Reading state information… Done
The following packages were automatically installed and are no longer required:
libopts25 libcap2 hbase ntp
Use ‘apt-get autoremove’ to remove them.
The following packages will be REMOVED:
hive
0 upgraded, 0 newly installed, 1 to remove and 26 not upgraded.
After this operation, 175 MB disk space will be freed.
Do you want to continue [Y/n]? Y
(Reading database … 57309 files and directories currently installed.)
Removing hive …
Processing triggers for man-db …

Checking what else is left after removal:

ubuntu@HIVE_SERVER:~$ sudo find / -name “hive”
/etc/hive
/usr/lib/hive
/var/lib/hive

Deleting all of above residual files in Hive:

ubuntu@HIVE_SERVER:~$ sudo rm -rf /etc/hive
ubuntu@HIVE_SERVER:~$ sudo rm -rf /usr/lib/hive
ubuntu@HIVE_SERVER:~$ sudo rm -rf /var/lib/hive

Verifying that none of hive is in my machine:

ubuntu@HIVE_SERVER:~$ sudo find / -name “hive”

Installing Hive (from cloudera distribution)

ubuntu@HIVE_SERVER:~$ sudo apt-get install hive
Reading package lists… Done
Building dependency tree
Reading state information… Done
The following NEW packages will be installed:
hive
0 upgraded, 1 newly installed, 0 to remove and 26 not upgraded.
Need to get 0 B/32.4 MB of archives.
After this operation, 175 MB of additional disk space will be used.
Selecting previously unselected package hive.
(Reading database … 50478 files and directories currently installed.)
Unpacking hive (from …/hive_0.10.0+78-1.cdh4.2.1.p0.8~precise-cdh4.2.1_all.deb) …
Processing triggers for man-db …
Setting up hive (0.10.0+78-1.cdh4.2.1.p0.8~precise-cdh4.2.1) …
update-alternatives: using /etc/hive/conf.dist to provide /etc/hive/conf (hive-conf) in auto mode.

Listing hive in my machine:

ubuntu@HIVE_SERVER:~$ sudo find / -name “hive”
/etc/hive
/run/hive
/usr/lib/hive
/usr/lib/hive/lib/php/packages/serde/org/apache/hadoop/hive
/usr/lib/hive/docs/api/org/apache/hadoop/hive
/usr/lib/hive/docs/api/org/apache/hive
/usr/lib/hive/bin/hive
/usr/share/doc/hive
/usr/share/doc/hive/examples/test-plugin/src/org/apache/hive
/usr/bin/hive
/var/lib/hive
/var/log/hive

Testing Hive:

ubuntu@HIVE_SERVER:~$ hive
Logging initialized using configuration in jar:file:/usr/lib/hive/lib/hive-common-0.10.0-cdh4.2.1.jar!/hive-log4j.properties
Hive history file=/tmp/ubuntu/hive_job_log_ubuntu_201306111709_559685140.txt
hive> show tables;
OK
Time taken: 14.431 seconds
hive> quit;

Its working!!

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s