Skip to content
This repository has been archived by the owner on Apr 8, 2024. It is now read-only.

Alluxio integration #7

Open
gm-spacagna opened this issue Sep 6, 2018 · 1 comment
Open

Alluxio integration #7

gm-spacagna opened this issue Sep 6, 2018 · 1 comment

Comments

@gm-spacagna
Copy link

I would encourage to leverage Alluxio technology, in particular the fuse mount feature, for synchronizing local file system with remote data repositories such as S3.
Let's suppose the data is very large and you do not want to fully copy it locally, you can store the data in S3 and read smartly from the Alluxio tiered cache via the local file system.
lazydata should then only check consistency of files, store and track the metadata attributes into git.

@rstojnic
Copy link
Owner

rstojnic commented Sep 6, 2018

I'm not very familiar with Alluxio, would you be able to provide a pull request?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants