[Kernel][Snapshots] Kernel Table API to read arbitrary snapshots

[abhishekrb19]

Feature request

Which Delta project/connector is this regarding?

Related to the Druid-Delta Lake connector apache/druid#15755. Currently in 3.0.0, the Table interface only supports reading the latest snapshot. I see the same for the pre-release version 3.1.0. How feasible would it be to add support for reading arbitrary snapshots in the Delta Kernel?

• Spark
• Standalone
• Flink
• Kernel
• Other (fill in here)

Willingness to contribute

The Delta Lake Community encourages new feature contributions. Would you or another member of your organization be willing to contribute an implementation of this feature?

• Yes. I can contribute this feature independently.
• Yes. I would be willing to contribute this feature with guidance from the Delta Lake community.
• No. I cannot contribute this feature at this time.

[allisonport-db]

Hey @abhishekrb19 I'm actually working on this now, I'll link you on the PR when it's ready.

Cross-linking this with #2275 for reference.

[abhishekrb19]

Thank you, @allisonport-db! Looking forward to that functionality.

[tdas]

@abhishekrb19 can i ask what is your interest this feature? or Kernel in general? Are you planning to integrate with Kernel?

[tdas]

@abhishekrb19 Actually, never mind my questions, I looked at the PR apache/druid#15755 you linked. This is amazing, you have already used Delta Kernel to integrate Delta Lake support in Apache Druid!

Can I ask how and when this support is going to be released?

[abhishekrb19]

@tdas, thanks! We plan to include this in the upcoming Druid 29 release (date TBD).

[tdas]

@abhishekrb19 thank you for the info. in 3.1, we made a few API changes, and added data skipping support. are you interested in using 3.1? If so, we can chat and help out. Would love to get your feedback in general about using Kernel the APIs.

[abhishekrb19]

@tdas, yes, 3.1.0 looks exciting, the PR to upgrade the connector in Druid is here: apache/druid#15842
I will share some feedback on the APIs once I'm done plumbing in scan filters. Thanks!

[abhishekrb19]

Resolved by #2607