Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix compatibility issues with SqlFirehose #9365

Merged
merged 2 commits into from Feb 15, 2020
Merged

Conversation

a2l007
Copy link
Contributor

@a2l007 a2l007 commented Feb 14, 2020

Fixes #9359.

Description

SqlFirehoseFactory is now fixed to be compatible with the deprecated FiniteFirehoseFactory . Will look to add InputSource and InputFormat support soon.


This PR has:

  • been self-reviewed.
  • been tested in a test Druid cluster.

@jihoonson
Copy link
Contributor

Would you fix the TC failure?

Screen Shot 2020-02-14 at 1 44 13 PM

@jihoonson jihoonson added the Bug label Feb 14, 2020
Copy link
Contributor

@jihoonson jihoonson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thank you for fixing it!

@jihoonson jihoonson merged commit 043abd5 into apache:master Feb 15, 2020
@MoushmiDas
Copy link

MoushmiDas commented Feb 27, 2020

Hi Team,

I am trying to Load the data from Postgresql to Apache Druid(apache-druid-0.17.0) , but I am getting below error:

"SQL firehose error with org.apache.druid.segment.realtime.firehose.SqlFirehoseFactory cannot be cast to org.apache.druid.data.input.FiniteFirehoseFactory."

seems the issue is already resolve in the below Ticket request:

*SQL firehose error #9359
*Fix compatibility issues with SqlFirehose #9365

Could you please help me to get the new io-configuration for the postgresql.

Currently we are using below io-config for which we are getting above error:
"ioConfig": {
"type": "index",
"firehose": {
"type": "sql",
"database": {
"type": "postgresql",
"connectorConfig": {
"connectURI": "jdbc:postgresql://<Host_Name>:<PortNo.>/<DB_Name>",
"user": "<User_Name>",
"password": "<Pass_word>"
}
},
"sqls": [
"select * from Table;"
]
}
}

@a2l007
Copy link
Contributor Author

a2l007 commented Feb 27, 2020

Hi @MoushmiDas ,the fix for this is merged to master and so unfortunately it is unavailable with the 0.17.0 release. You could however build druid from source with the master branch. Instructions here: https://druid.apache.org/docs/latest/development/build.html

@jihoonson jihoonson added this to the 0.18.0 milestone Mar 26, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

SQL firehose error
4 participants