Skip to content

mbenenso/beam-deltalake

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Reading Delta Lake Data from Beam

General Info:

All files, except org.apache.beam.sdk.io.DeltaFileIO are from Daltalake Standalone Reader. I was not able to use delta standalone jar, because of dependency conflicts, so have to re-build it, with proper dependencies. There are no changes in the java/scala code.

org.apache.beam.sdk.io.DeltaFileIO is modified from Beam's FileIO : access to Beam's Filesystem has been replaced to the access to DeltaLake Standalone.

Usage example: beam-deltalake-example

About

Reading Dalta Lake data from Beam

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published