Skip to content

Add a Deltalake connector #509

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 13 commits into from
Closed

Conversation

malonsocasas
Copy link

Issue #310 :

Description of changes:
Add a first version of a Deltalake connector (https://github.com/delta-io/delta) so that Athena can interpret table schemas/partitions automatically (some current limitations are listed in the README).

Note that my first attempt was to use: https://github.com/delta-io/connectors. However, the library being in Scala and depending on a lot of packages, it makes it unsuitable for a Lambda code (with a limitation of 250 MB).

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@avirtuos
Copy link
Contributor

Thanks so much for the contribution, it is off to a great start. I posted a few questions.

@avirtuos avirtuos added the enhancement New feature or request label Oct 25, 2021
@malonsocasas
Copy link
Author

Thanks so much for the contribution, it is off to a great start. I posted a few questions.

Thanks @avirtuos for your feedback ! How can I have access to your questions ? It seems I can't see them in Github

@slomkarafa
Copy link

wazzup guys :D
@malonsocasas @avirtuos any updates on this topic?
Thanks

@artloder
Copy link

Hey @avirtuos I'm following along as well to see if this PR can be merged; it's listed on https://delta.io/integrations as a "preview" release but it would be great if this were an official part of awslabs/aws-athena-query-federation.

@ormahler
Copy link

@malonsocasas Nice job!
@avirtuos any updates on this topic?

@slomkarafa
Copy link

Hello, is any one working on this topic? Because I am and I do not want to duplicate with someone ;)

@slomkarafa
Copy link

Hello, hello its me again, as this PR is stale I have created new one #825 which is an extension for that one :p cheers

@henrymai
Copy link
Contributor

See: #825 (comment)

@henrymai henrymai closed this Dec 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants