rbhive broken on cdh3u4 release/update (Apache Hive 0.7.1-cdh3u4)

[fuentesjr]

I should mention that it was working fine on the cdh3u3 release.

bash-3.2$ cat test_rbhive.rb
require 'rbhive'

results = RBHive.connect('hive.server.address') do |connection|
connection.fetch "select state_code from zip_code"
end

results.each do |data|
puts "data = #{data.inspect}"
end

bash-3.2$ ruby test_rbhive.rb
Connecting to hive.server.address on port 10000
Executing Hive Query: select state_code from zip_code
/home/someuser/.rvm/gems/ruby-1.9.2-p290@somegemset/gems/rbhive-0.2.94/lib/thrift/thrift_hive.rb:26:in recv_execute': Query returned non-zero code: 10, cause: FAILED: Error in semantic analysis: Unable to fetch table zip_code (HiveServerException) from /home/someuser/.rvm/gems/ruby-1.9.2-p290@somegemset/gems/rbhive-0.2.94/lib/thrift/thrift_hive.rb:17:inexecute'
from /home/someuser/.rvm/gems/ruby-1.9.2-p290@somegemset/gems/rbhive-0.2.94/lib/rbhive/connection.rb:140:in execute_unsafe' from /home/someuser/.rvm/gems/ruby-1.9.2-p290@somegemset/gems/rbhive-0.2.94/lib/rbhive/connection.rb:81:inblock in fetch'
from /home/someuser/.rvm/gems/ruby-1.9.2-p290@somegemset/gems/rbhive-0.2.94/lib/rbhive/connection.rb:145:in block in safe' from <internal:prelude>:10:insynchronize'
from /home/someuser/.rvm/gems/ruby-1.9.2-p290@somegemset/gems/rbhive-0.2.94/lib/rbhive/connection.rb:145:in safe' from /home/someuser/.rvm/gems/ruby-1.9.2-p290@somegemset/gems/rbhive-0.2.94/lib/rbhive/connection.rb:80:infetch' from test_rbhive.rb:4:in block in <main>' from /home/someuser/.rvm/gems/ruby-1.9.2-p290@somegemset/gems/rbhive-0.2.94/lib/rbhive/connection.rb:14:inconnect'
from test_rbhive.rb:3:in `

'

[forward-bot]

Hi Sal,

We upgraded one of our Hive servers to CDH3U4 and were able to query it using RBHive.
Is there any chance the CH3U4 over-wrote your Hive config files which would mean Hive doesn't know about the existing tables?

The upgrade does ask if it can install new config files on Ubuntu (not sure what OS you're using).

You can check if it's an RBHive issue or a Hive issue by trying out the same query in the Hive CLI on the actual Hive server. If the query works on the Hive CLI then it would be an RBHive issue, otherwise it might be something to do with your Hive installation.

Let me know what you find.

Thanks.

Abs

On 27 Jun 2012, at 23:05, Sal Fuentes wrote:

I should mention that it was working fine on the cdh3u3 release.

bash-3.2$ cat test_rbhive.rb
require 'rbhive'

results = RBHive.connect('hive.server.address') do |connection|
connection.fetch "select state_code from zip_code"
end

results.each do |data|
puts "data = #{data.inspect}"
end

bash-3.2$ ruby test_rbhive.rb
Connecting to hive.server.address on port 10000
Executing Hive Query: select state_code from zip_code
/home//.rvm/gems/ruby-1.9.2-p290@****/gems/rbhive-0.2.94/lib/thrift/thrift_hive.rb:26:in recv_execute': Query returned non-zero code: 10, cause: FAILED: Error in semantic analysis: Unable to fetch table zip_code (HiveServerException) from /home/****_/.rvm/gems/ruby-1.9.2-p290@**_**/gems/rbhive-0.2.94/lib/thrift/thrift_hive.rb:17:in execute'
from /home//.rvm/gems/ruby-1.9.2-p290@**/gems/rbhive-0.2.94/lib/rbhive/connection.rb:140:in execute_unsafe' from /home/****_/.rvm/gems/ruby-1.9.2-p290@**_**/gems/rbhive-0.2.94/lib/rbhive/connection.rb:81:in block in fetch'
from /home/****_/.rvm/gems/ruby-1.9.2-p290@**/gems/rbhive-0.2.94/lib/rbhive/connection.rb:145:in block in safe' from <internal:prelude>:10:insynchronize'
from /home/****/.rvm/gems/ruby-1.9.2-p290@_/gems/rbhive-0.2.94/lib/rbhive/connection.rb:145:in safe' from /home/****_/.rvm/gems/ruby-1.9.2-p290@**_**/gems/rbhive-0.2.94/lib/rbhive/connection.rb:80:in fetch' from test_rbhive.rb:4:inblock in <main>' from /home/****_/.rvm/gems/ruby-1.9.2-p290@**_**/gems/rbhive-0.2.94/lib/rbhive/connection.rb:14:in connect'
from test_rbhive.rb:3:in`

'

---

Reply to this email directly or view it on GitHub:
#2

[fuentesjr]

I tried the same query in the Hive CLI on the same Hive server the test script connects to with the following results:

hive> select state_code from zip_code limit 3;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201206270101_16742, Tracking URL = http://jobtracker-prod.server.addr:50030/jobdetails.jsp?jobid=job_201206270101_16742
Kill Command = /home/t/hadoop_current/bin/hadoop job -Dmapred.job.tracker=jobtracker-prod.server.addr:9001 -kill job_201206270101_16742
2012-06-29 17:42:33,427 Stage-1 map = 0%, reduce = 0%
2012-06-29 17:42:39,475 Stage-1 map = 33%, reduce = 0%
2012-06-29 17:42:42,508 Stage-1 map = 100%, reduce = 0%
2012-06-29 17:42:45,531 Stage-1 map = 100%, reduce = 100%
Ended Job = job_201206270101_16742
OK
NY
NY
NY
Time taken: 20.205 seconds
hive> exit;

I also tweaked the script and ran it again:

bash-3.2$ cat test_rbhive.rb
require 'rbhive'

results = RBHive.connect('tron.ev1.yellowpages.com') do |connection|
#connection.fetch "select state_code from zip_code"
connection.fetch "describe zip_code"
end

results.each do |data|
puts "data = #{data.inspect}"
end

bash-3.2$ ruby test_rbhive.rb
Connecting to hive.server.addr on port 10000
Executing Hive Query: describe zip_code/home/someuser/.rvm/gems/ruby-1.9.2-p290@somegemset/gems/rbhive-0.2.94/lib/thrift/thrift_hive.rb:26:in recv_execute': Query returned non-zero code: 9, cause: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask (HiveServerException) from /home/someuser/.rvm/gems/ruby-1.9.2-p290@somegemset/gems/rbhive-0.2.94/lib/thrift/thrift_hive.rb:17:inexecute'
from /home/someuser/.rvm/gems/ruby-1.9.2-p290@somegemset/gems/rbhive-0.2.94/lib/rbhive/connection.rb:140:in execute_unsafe' from /home/someuser/.rvm/gems/ruby-1.9.2-p290@somegemset/gems/rbhive-0.2.94/lib/rbhive/connection.rb:81:inblock in fetch'
from /home/someuser/.rvm/gems/ruby-1.9.2-p290@somegemset/gems/rbhive-0.2.94/lib/rbhive/connection.rb:145:in block in safe' from <internal:prelude>:10:insynchronize'
from /home/someuser/.rvm/gems/ruby-1.9.2-p290@somegemset/gems/rbhive-0.2.94/lib/rbhive/connection.rb:145:in safe' from /home/someuser/.rvm/gems/ruby-1.9.2-p290@somegemset/gems/rbhive-0.2.94/lib/rbhive/connection.rb:80:infetch'
from test_rbhive.rb:5:in block in <main>' from /home/someuser/.rvm/gems/ruby-1.9.2-p290@somegemset/gems/rbhive-0.2.94/lib/rbhive/connection.rb:14:inconnect' from test_rbhive.rb:3:in `

'

Other information:

cat /etc/redhat-release
CentOS release 5.4 (Final)

Let me know if there is any other information that will be helpful.

[forward-bot]

Hi Sal,

Thanks for the update. We're about to upgrade our cluster to the latest release of CDH in the coming days. If we get the same issue with RBHive then we will release a fix. At the moment we are not able to reproduce the problem therefore are not able to diagnose and fix the bug.

Unfortunately I'm not able to give you any timeframes on when this will happen but will send you an update when it does.

Thanks

Abhinay Mehta

On 29 Jun 2012, at 18:56, Sal Fuentes wrote:

I tried the same query in the Hive CLI on the same Hive server the test script connects to with the following results:

hive> select state_code from zip_code limit 3;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201206270101_16742, Tracking URL = http://jobtracker-prod.server.addr:50030/jobdetails.jsp?jobid=job_201206270101_16742
Kill Command = /home/t/hadoop_current/bin/hadoop job -Dmapred.job.tracker=jobtracker-prod.server.addr:9001 -kill job_201206270101_16742
2012-06-29 17:42:33,427 Stage-1 map = 0%, reduce = 0%
2012-06-29 17:42:39,475 Stage-1 map = 33%, reduce = 0%
2012-06-29 17:42:42,508 Stage-1 map = 100%, reduce = 0%
2012-06-29 17:42:45,531 Stage-1 map = 100%, reduce = 100%
Ended Job = job_201206270101_16742
OK
NY
NY
NY
Time taken: 20.205 seconds
hive> exit;

I also tweaked the script and ran it again:

bash-3.2$ cat test_rbhive.rb
require 'rbhive'

results = RBHive.connect('tron.ev1.yellowpages.com') do |connection|
#connection.fetch "select state_code from zip_code"
connection.fetch "describe zip_code"
end

results.each do |data|
puts "data = #{data.inspect}"
end

bash-3.2$ ruby test_rbhive.rb
Connecting to hive.server.addr on port 10000
Executing Hive Query: describe zip_code/home/someuser/.rvm/gems/ruby-1.9.2-p290@somegemset/gems/rbhive-0.2.94/lib/thrift/thrift_hive.rb:26:in recv_execute': Query returned non-zero code: 9, cause: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask (HiveServerException) from /home/someuser/.rvm/gems/ruby-1.9.2-p290@somegemset/gems/rbhive-0.2.94/lib/thrift/thrift_hive.rb:17:inexecute'
from /home/someuser/.rvm/gems/ruby-1.9.2-p290@somegemset/gems/rbhive-0.2.94/lib/rbhive/connection.rb:140:in execute_unsafe' from /home/someuser/.rvm/gems/ruby-1.9.2-p290@somegemset/gems/rbhive-0.2.94/lib/rbhive/connection.rb:81:inblock in fetch'
from /home/someuser/.rvm/gems/ruby-1.9.2-p290@somegemset/gems/rbhive-0.2.94/lib/rbhive/connection.rb:145:in block in safe' from <internal:prelude>:10:insynchronize'
from /home/someuser/.rvm/gems/ruby-1.9.2-p290@somegemset/gems/rbhive-0.2.94/lib/rbhive/connection.rb:145:in safe' from /home/someuser/.rvm/gems/ruby-1.9.2-p290@somegemset/gems/rbhive-0.2.94/lib/rbhive/connection.rb:80:infetch'
from test_rbhive.rb:5:in block in <main>' from /home/someuser/.rvm/gems/ruby-1.9.2-p290@somegemset/gems/rbhive-0.2.94/lib/rbhive/connection.rb:14:inconnect' from test_rbhive.rb:3:in `

'

Other information:

cat /etc/redhat-release
CentOS release 5.4 (Final)

Let me know if there is any other information that will be helpful.

---

Reply to this email directly or view it on GitHub:
#2 (comment)

[andytinycat]

Closed as we can't reproduce.