Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(inputs.sql): Add FlightSQL support #13091

Merged
merged 3 commits into from
May 5, 2023
Merged

feat(inputs.sql): Add FlightSQL support #13091

merged 3 commits into from
May 5, 2023

Conversation

srebhan
Copy link
Contributor

@srebhan srebhan commented Apr 14, 2023

This adds support for FlightSQL databases.

@telegraf-tiger telegraf-tiger bot added area/sql feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin plugin/input 1. Request for new input plugins 2. Issues/PRs that are related to input plugins labels Apr 14, 2023
@srebhan
Copy link
Contributor Author

srebhan commented Apr 14, 2023

Build fails due to upstream issue (reported as apache/arrow#35133).

@srebhan srebhan marked this pull request as draft April 14, 2023 16:41
@srebhan
Copy link
Contributor Author

srebhan commented Apr 14, 2023

This PR apache/arrow#35137 is required to go into arrow first so we can get rid of the replace statement.

@srebhan srebhan marked this pull request as ready for review April 27, 2023 14:53
@srebhan srebhan added the ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review. label Apr 27, 2023
powersj
powersj reviewed Apr 27, 2023
View reviewed changes
Copy link
Contributor

@powersj powersj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am weary of changing errors to warnings with the wide support of drivers this has. Could you provide some more context in line?

docs/SQL_DRIVERS_INPUT.md Outdated Show resolved Hide resolved
plugins/inputs/sql/sql.go Outdated Show resolved Hide resolved
plugins/inputs/sql/sql.go Outdated Show resolved Hide resolved
plugins/inputs/sql/sql.go Outdated Show resolved Hide resolved
@srebhan srebhan removed the ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review. label May 5, 2023
@srebhan
Copy link
Contributor Author

srebhan commented May 5, 2023

Welcome to dependency hell... If I use the arrow version supporting FlightSQL without prepare statement, github.com/apache/arrow/go/v12@42527e153286b462ae57e82f876a2443816c82d1 (which is the latest v12 I can find in git), go downgrades bigquery to a v11 arrow and builds fail for 32-bit platforms.
When using bigquery v1.51.1 which works on 32-bit platforms it comes with arrow v12.0.0 which go thinks is newer than the above snapshot. However, the tagged version does not have the "FlightSQL without prepare statement" patch 🤦...

@telegraf-tiger
Copy link
Contributor

telegraf-tiger bot commented May 5, 2023

Download PR build artifacts for linux_amd64.tar.gz, darwin_amd64.tar.gz, and windows_amd64.zip.
Downloads for additional architectures and packages are available below.

⚠️ This pull request increases the Telegraf binary size by 5.19 % for linux amd64 (new size: 184.2 MB, nightly size 175.1 MB)

📦 Click here to get additional PR build artifacts

Artifact URLs

DEB RPM TAR GZ ZIP
amd64.deb aarch64.rpm darwin_amd64.tar.gz windows_amd64.zip
arm64.deb armel.rpm darwin_arm64.tar.gz windows_arm64.zip
armel.deb armv6hl.rpm freebsd_amd64.tar.gz windows_i386.zip
armhf.deb i386.rpm freebsd_armv7.tar.gz
i386.deb ppc64le.rpm freebsd_i386.tar.gz
mips.deb riscv64.rpm linux_amd64.tar.gz
mipsel.deb s390x.rpm linux_arm64.tar.gz
ppc64el.deb x86_64.rpm linux_armel.tar.gz
riscv64.deb linux_armhf.tar.gz
s390x.deb linux_i386.tar.gz
linux_mips.tar.gz
linux_mipsel.tar.gz
linux_ppc64le.tar.gz
linux_riscv64.tar.gz
linux_s390x.tar.gz

@srebhan srebhan added the ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review. label May 5, 2023
@powersj powersj merged commit fc4e896 into influxdata:master May 5, 2023
5 checks passed
@chenrui333 chenrui333 mentioned this pull request Jun 13, 2023
Merged
@srebhan srebhan added this to the v1.27.0 milestone Jun 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@powersj powersj powersj approved these changes

Assignees

@powersj powersj

Labels
area/sql feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin plugin/input 1. Request for new input plugins 2. Issues/PRs that are related to input plugins ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review.
Projects
None yet
Milestone
v1.27.0
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@srebhan @powersj