-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Master files should sync to local storage #25
Comments
Note, we decided this might be able to be handled manually during the pilot. |
@davidschober new masters will be stored on S3 until this get figured out. old masters will remain on Isilon, correct? also, let's give this a deadline of July 15. |
Yes. We may sync occasionally. Masters are safe in s3
…Sent from my iPhone
On Jun 5, 2017, at 1:56 PM, ccaizzi ***@***.***> wrote:
@davidschober new masters will be stored on S3 until this get figured out. old masters will remain on Isilon, correct? also, let's give this a deadline of July 15.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.
|
I'll be interested in seeing how much is costs to sync the masters back down. |
@davidschober let's schedule a meeting with @mbklein to align this with our preservation strategy. |
@mbklein and @Toputnal and @d-venckus let's discuss our sync strategy this week. Michael, this is something Jim and David can setup a local worker to sync from the s3 nightly correct? Using using something like http://docs.aws.amazon.com/cli/latest/reference/s3/sync.html |
I'm currently syncing masterfiles to my local, mirrored disks on my desktop. I will request a new isilon share and will move the files from my desktop to the new isilon share so as to avoid downloading everything from amazon more than once. |
I am copying the data from my desktop to the Isilon on farchive21 via avalonweb1p. After the initial copy. I will look into setting up a nightly cron job on avalonweb1p to just pull down diffs from s3. |
That's correct. Currently the only bucket we care about in production. |
@Toputnal I'm going to close this as complete. I will write docs about where things are syncing to. |
Description
Master files in Avalon should sync to local storage during the pilot phase. We have not made a decision about long term storage of files, and it is out of scope for this project. For now we want to have a copy of renamed master files (not derivatives) synced to local storage.
Done looks like
The text was updated successfully, but these errors were encountered: