[CARBONDATA-1285] resolved compilation error in hiveembedded server2 when running hiveexample #1154

anubhav100 · 2017-07-10T09:54:54Z

This Pr Will Resolved Compilation error in HiveEmbedded Server2

corrected the pom to use hiveserver2 b y adding hive-service and hive-jdbc dependency
Tested by Running HIve Example

asfgit · 2017-07-10T09:54:56Z

Can one of the admins verify this patch?

asfgit · 2017-07-10T09:54:56Z

Can one of the admins verify this patch?

anubhav100 · 2017-07-10T09:57:01Z

@chenliang613 ping

CarbonDataQA · 2017-07-10T10:07:23Z

Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/2989/

CarbonDataQA · 2017-07-10T10:08:39Z

Build Success with Spark 1.6, Please check CI http://144.76.159.231:8080/job/ApacheCarbonPRBuilder/400/

chenliang613 · 2017-07-13T02:41:08Z

@anubhav100 i cam compile successfully on current master. so this pr is for fixing compilation issues or HiveExample issues ?

anubhav100 · 2017-07-13T02:46:17Z

@chenliang613 for fixing HiveExampleIssues

chenliang613 · 2017-07-13T06:39:36Z

integration/hive/pom.xml

+        <dependency>
+            <groupId>org.apache.spark</groupId>
+            <artifactId>spark-hive_2.11</artifactId>
+            <version>2.1.0</version>


for spark version, why not using ${spark.version}

@chenliang613 corrected

chenliang613 · 2017-07-13T06:46:33Z

I tested it , got the below log msg :
2017-07-13 14:45:59 WARN ShellBasedUnixGroupsMapping:86 - got exception trying to get groups for user anonymous
org.apache.hadoop.util.Shell$ExitCodeException: id: anonymous: no such user

at org.apache.hadoop.util.Shell.runCommand(Shell.java:464)
at org.apache.hadoop.util.Shell.run(Shell.java:379)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:589)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:678)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:661)
at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:83)
at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:52)
at org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.getGroups(JniBasedUnixGroupsMappingWithFallback.java:50)
at org.apache.hadoop.security.Groups.getGroups(Groups.java:89)
at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1352)
at org.apache.hadoop.hive.ql.security.HadoopDefaultAuthenticator.setConf(HadoopDefaultAuthenticator.java:64)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at org.apache.hadoop.hive.ql.metadata.HiveUtils.getAuthenticator(HiveUtils.java:439)
at org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:715)
at org.apache.hadoop.hive.ql.session.SessionState.getAuthenticator(SessionState.java:1391)
at org.apache.hadoop.hive.ql.session.SessionState.getUserFromAuthenticator(SessionState.java:984)
at org.apache.hadoop.hive.ql.metadata.Table.getEmptyTable(Table.java:177)
at org.apache.hadoop.hive.ql.metadata.Table.<init>(Table.java:119)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.addDbAndTabToOutputs(SemanticAnalyzer.java:11004)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeCreateTable(SemanticAnalyzer.java:10900)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genResolvedParseTree(SemanticAnalyzer.java:10047)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10128)
at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:209)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:227)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:424)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:308)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1122)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1116)
at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:110)
at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:181)
at org.apache.hive.service.cli.operation.Operation.run(Operation.java:257)
at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:388)
at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:375)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)
at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36)
at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59)
at com.sun.proxy.$Proxy32.executeStatementAsync(Unknown Source)
at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:274)
at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:486)
at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1313)
at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1298)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

2017-07-13 14:45:59 WARN UserGroupInformation:1355 - No groups available for user anonymous
2017-07-13 14:45:59 WARN ShellBasedUnixGroupsMapping:86 - got exception trying to get groups for user anonymous
org.apache.hadoop.util.Shell$ExitCodeException: id: anonymous: no such user

at org.apache.hadoop.util.Shell.runCommand(Shell.java:464)
at org.apache.hadoop.util.Shell.run(Shell.java:379)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:589)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:678)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:661)
at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:83)
at org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:52)
at org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.getGroups(JniBasedUnixGroupsMappingWithFallback.java:50)
at org.apache.hadoop.security.Groups.getGroups(Groups.java:89)
at org.apache.hadoop.security.UserGroupInformation.getGroupNames(UserGroupInformation.java:1352)
at org.apache.hadoop.hive.ql.security.HadoopDefaultAuthenticator.setConf(HadoopDefaultAuthenticator.java:65)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at org.apache.hadoop.hive.ql.metadata.HiveUtils.getAuthenticator(HiveUtils.java:439)
at org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:715)
at org.apache.hadoop.hive.ql.session.SessionState.getAuthenticator(SessionState.java:1391)
at org.apache.hadoop.hive.ql.session.SessionState.getUserFromAuthenticator(SessionState.java:984)
at org.apache.hadoop.hive.ql.metadata.Table.getEmptyTable(Table.java:177)
at org.apache.hadoop.hive.ql.metadata.Table.<init>(Table.java:119)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.addDbAndTabToOutputs(SemanticAnalyzer.java:11004)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeCreateTable(SemanticAnalyzer.java:10900)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genResolvedParseTree(SemanticAnalyzer.java:10047)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10128)
at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:209)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:227)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:424)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:308)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1122)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1116)
at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:110)
at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:181)
at org.apache.hive.service.cli.operation.Operation.run(Operation.java:257)
at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:388)
at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:375)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)
at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36)
at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59)
at com.sun.proxy.$Proxy32.executeStatementAsync(Unknown Source)
at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:274)
at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:486)
at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1313)
at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1298)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

2017-07-13 14:45:59 WARN UserGroupInformation:1355 - No groups available for user anonymous

anubhav100 · 2017-07-13T10:19:11Z

@chenliang613 these log levels are because of unprivelleged user when connecting user to thriftserver

CarbonDataQA · 2017-07-13T10:33:19Z

Build Success with Spark 1.6, Please check CI http://144.76.159.231:8080/job/ApacheCarbonPRBuilder/486/

CarbonDataQA · 2017-07-13T10:34:36Z

Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/3075/

chenliang613 · 2017-07-14T03:43:21Z

LGTM

anubhav100 changed the title ~~[CARBONDATA-1285] resolved compilation error in hiveembedded server2~~ [CARBONDATA-1285] resolved compilation error in hiveembedded server2 when running hiveexample Jul 13, 2017

chenliang613 reviewed Jul 13, 2017

View reviewed changes

resolved compilation error in hiveembedded server2

7efb58c

anubhav100 force-pushed the CARBONDATA-1285 branch from ab15f9a to 7efb58c Compare July 13, 2017 10:16

asfgit closed this in a03f7b1 Jul 14, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[CARBONDATA-1285] resolved compilation error in hiveembedded server2 when running hiveexample #1154

[CARBONDATA-1285] resolved compilation error in hiveembedded server2 when running hiveexample #1154

anubhav100 commented Jul 10, 2017

asfgit commented Jul 10, 2017

asfgit commented Jul 10, 2017

anubhav100 commented Jul 10, 2017

CarbonDataQA commented Jul 10, 2017

CarbonDataQA commented Jul 10, 2017

chenliang613 commented Jul 13, 2017

anubhav100 commented Jul 13, 2017

chenliang613 Jul 13, 2017

anubhav100 Jul 13, 2017

chenliang613 commented Jul 13, 2017

anubhav100 commented Jul 13, 2017

CarbonDataQA commented Jul 13, 2017

CarbonDataQA commented Jul 13, 2017

chenliang613 commented Jul 14, 2017

[CARBONDATA-1285] resolved compilation error in hiveembedded server2 when running hiveexample #1154

[CARBONDATA-1285] resolved compilation error in hiveembedded server2 when running hiveexample #1154

Conversation

anubhav100 commented Jul 10, 2017

asfgit commented Jul 10, 2017

asfgit commented Jul 10, 2017

anubhav100 commented Jul 10, 2017

CarbonDataQA commented Jul 10, 2017

CarbonDataQA commented Jul 10, 2017

chenliang613 commented Jul 13, 2017

anubhav100 commented Jul 13, 2017

chenliang613 Jul 13, 2017

Choose a reason for hiding this comment

anubhav100 Jul 13, 2017

Choose a reason for hiding this comment

chenliang613 commented Jul 13, 2017

anubhav100 commented Jul 13, 2017

CarbonDataQA commented Jul 13, 2017

CarbonDataQA commented Jul 13, 2017

chenliang613 commented Jul 14, 2017