You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
use dbt-hive connection hive Error reporting "Unable to establish connection to Hive server: Error while compiling statement: FAILED: ParseException line 1:0 cannot recognize input near '/' '*' '{'"
#129
Closed
1043038630 opened this issue
Sep 22, 2023
· 4 comments
Using unsafe methods:
profiles.yml:
dbt_hive_test:
outputs:
dev:
type: hive
host: 192.168.1.143
port: 10000
schema: "default"
target: dev
Error running dbt
(dbt-hive-test) [root@localhost dbt_hive_test]# dbt run
09:00:34 Running with dbt=1.4.8
09:00:34 Unable to do partial parsing because profile has changed
09:00:36 Found 2 models, 0 tests, 0 snapshots, 0 analyses, 334 macros, 0 operations, 0 seed files, 0 sources, 0 exposures, 0 metrics
09:00:36
09:00:37
09:00:37 Finished running in 0 hours 0 minutes and 1.04 seconds (1.04s).
09:00:38 Encountered an error:
Runtime Error
Runtime Error
Unable to establish connection to Hive server: Error while compiling statement: FAILED: ParseException line 1:0 cannot recognize input near '/' '*' '{'
(dbt-hive-test) [root@localhost dbt_hive_test]# dbt run
09:04:16 Running with dbt=1.4.8
09:04:16 Found 2 models, 0 tests, 0 snapshots, 0 analyses, 334 macros, 0 operations, 0 seed files, 0 sources, 0 exposures, 0 metrics
09:04:16
09:04:17
09:04:17 Finished running in 0 hours 0 minutes and 0.75 seconds (0.75s).
09:04:18 Encountered an error:
Runtime Error
Runtime Error
Unable to establish connection to Hive server: Error while compiling statement: FAILED: ParseException line 1:0 cannot recognize input near '/' '*' '{'
Use safe methods:
profiles.yml:
dbt_hive_test:
outputs:
dev:
type: hive
host: 192.168.1.143
port: 10000
auth_type : LDAP
use_http_transport: false
use_ssl: false
user : "uid=query,ou=People,dc=pudu,dc=com"
password : "000000"
schema: "default"
target: dev
I confirm that the above method can link to hive,I tested it with beeline。
[root@cdhmaster bin]# beeline --color=true -u "jdbc:hive2://cdhslave2:10000/default" -n uid=query,ou=People,dc=pudu,dc=com -p 000000
Connecting to jdbc:hive2://cdhslave2:10000/default
Connected to: Apache Hive (version 2.1.1-cdh6.3.2)
Driver: Hive JDBC (version 2.1.1-cdh6.3.2)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 2.1.1-cdh6.3.2 by Apache Hive
Testing with dbt is also normal
(dbt-hive-test) [root@localhost dbt_hive_test]# dbt debug
08:59:39 Running with dbt=1.4.8
dbt version: 1.4.8
python version: 3.9.17
python path: /root/miniconda3/bin/python
os info: Linux-3.10.0-1160.92.1.el7.x86_64-x86_64-with-glibc2.17
Using profiles.yml file at /root/.dbt/profiles.yml
Using dbt_project.yml file at /root/gitlab/dbt/dbt-cdh6.3.2-test/dbt_hive_test/dbt_project.yml
Configuration:
profiles.yml file [OK found and valid]
dbt_project.yml file [OK found and valid]
Required dependencies:
- git [OK found]
Connection:
host: 192.168.1.143
schema: default
user: uid=query,ou=People,dc=pudu,dc=com
Connection test: [OK connection ok]
All checks passed!
But there will be errors when running
(dbt-hive-test) [root@localhost dbt_hive_test]# dbt run
09:04:16 Running with dbt=1.4.8
09:04:16 Found 2 models, 0 tests, 0 snapshots, 0 analyses, 334 macros, 0 operations, 0 seed files, 0 sources, 0 exposures, 0 metrics
09:04:16
09:04:17
09:04:17 Finished running in 0 hours 0 minutes and 0.75 seconds (0.75s).
09:04:18 Encountered an error:
Runtime Error
Runtime Error
Unable to establish connection to Hive server: Error while compiling statement: FAILED: ParseException line 1:0 cannot recognize input near '/' '*' '{'
I checked the logs of hive JDBC,It is speculated that the SQL converted from dbt has added annotations that Hive does not recognize,But I'm not sure /* */
2023-09-22 16:17:12,456 INFO org.apache.hadoop.hive.ql.Driver: [202ed937-5191-4d28-934a-341c95b622af HiveServer2-Handler-Pool: Thread-56]: Compiling command(queryId=hive_20230922161712_ef9241ae-634a-451e-a68d-8992a0932fa7): /* {"app": "dbt", "dbt_version": "1.4.8", "profile_name": "dbt_hive_test", "target_name": "dev", "connection_name": "list_schemas"} */
show databases
2023-09-22 16:17:12,476 ERROR org.apache.hadoop.hive.ql.Driver: [202ed937-5191-4d28-934a-341c95b622af HiveServer2-Handler-Pool: Thread-56]: FAILED: ParseException line 1:0 cannot recognize input near '/' '*' '{'
org.apache.hadoop.hive.ql.parse.ParseException: line 1:0 cannot recognize input near '/' '*' '{'
at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:221)
at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:75)
at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:68)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:564)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1425)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1398)
at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:205)
at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:290)
at org.apache.hive.service.cli.operation.Operation.run(Operation.java:260)
at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:505)
at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:491)
at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:295)
at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:507)
at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1437)
at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1422)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
The text was updated successfully, but these errors were encountered:
Using unsafe methods:
profiles.yml:
dbt_hive_test:
outputs:
dev:
type: hive
host: 192.168.1.143
port: 10000
schema: "default"
target: dev
Error running dbt
Use safe methods:
profiles.yml:
dbt_hive_test:
outputs:
dev:
type: hive
host: 192.168.1.143
port: 10000
auth_type : LDAP
use_http_transport: false
use_ssl: false
user : "uid=query,ou=People,dc=pudu,dc=com"
password : "000000"
schema: "default"
target: dev
I confirm that the above method can link to hive,I tested it with beeline。
Testing with dbt is also normal
But there will be errors when running
I checked the logs of hive JDBC,It is speculated that the SQL converted from dbt has added annotations that Hive does not recognize,But I'm not sure /* */
The text was updated successfully, but these errors were encountered: