Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scanner database files get incorrectly cached when public on GCS #219

Open
willcrichton opened this issue Jul 25, 2018 · 0 comments
Open

Comments

@willcrichton
Copy link
Member

If a bucket has public ACLs by default, or if anything else causes files like db_metadata.bin to get set to public by default, then Google Cloud automatically applies a caching mechanism: https://cloud.google.com/storage/docs/consistency#cache-control

This causes incorrect behavior where writes to the metadata files aren't seen until after the cache expires, e.g. on ingest, the video metadata is successfully written, but subsequent attempts to read that metadata fail until an hour later.

The best solution is to expose cache control as a part of Storehouse, and have Scanner prevent caching of all metadata files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant