Skip to content

Add a Deltalake connector - #509

Closed
malonsocasas wants to merge 13 commits into
awslabs:masterfrom
malonsocasas:master
Closed

Add a Deltalake connector#509
malonsocasas wants to merge 13 commits into
awslabs:masterfrom
malonsocasas:master

Conversation

@malonsocasas

Copy link
Copy Markdown

Issue #310 :

Description of changes:
Add a first version of a Deltalake connector (https://github.com/delta-io/delta) so that Athena can interpret table schemas/partitions automatically (some current limitations are listed in the README).

Note that my first attempt was to use: https://github.com/delta-io/connectors. However, the library being in Scala and depending on a lot of packages, it makes it unsuitable for a Lambda code (with a limitation of 250 MB).

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@avirtuos

Copy link
Copy Markdown
Contributor

Thanks so much for the contribution, it is off to a great start. I posted a few questions.

@avirtuos avirtuos added the enhancement New feature or request label Oct 25, 2021
@malonsocasas

Copy link
Copy Markdown
Author

Thanks so much for the contribution, it is off to a great start. I posted a few questions.

Thanks @avirtuos for your feedback ! How can I have access to your questions ? It seems I can't see them in Github

@slomkarafa

Copy link
Copy Markdown

wazzup guys :D
@malonsocasas @avirtuos any updates on this topic?
Thanks

@artloder

Copy link
Copy Markdown

Hey @avirtuos I'm following along as well to see if this PR can be merged; it's listed on https://delta.io/integrations as a "preview" release but it would be great if this were an official part of awslabs/aws-athena-query-federation.

@ormahler

Copy link
Copy Markdown

@malonsocasas Nice job!
@avirtuos any updates on this topic?

@slomkarafa

Copy link
Copy Markdown

Hello, is any one working on this topic? Because I am and I do not want to duplicate with someone ;)

@slomkarafa

Copy link
Copy Markdown

Hello, hello its me again, as this PR is stale I have created new one #825 which is an extension for that one :p cheers

@henrymai

Copy link
Copy Markdown
Contributor

See: #825 (comment)

@henrymai henrymai closed this Dec 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants