Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Check if generated file was modified before uploading to S3 #2

Merged
merged 2 commits into from Jul 5, 2012

Conversation

Projects
None yet
2 participants

rlander commented Jul 5, 2012

I think it's kinda wasteful to upload all files to S3 on every ./manage staticsitegen run, so I refactored _s3render_path to check the md5 hash against the key's etag (if it exists).

Boto's #compute_md5 method expects a file object, so the function creates an in-memory file using cStringIO, a faster StringIO. If you find any problems, I could switch it for an in-disk NamedRemporaryFile.

rlander added some commits Jul 5, 2012

Check if generated file was modified before uploading to S3, skips if…
… unmodified.

Check if generated file was modified before uploading to S3.
Refactoring : #_s3render_path now only responsible for checking etags,
generating paths and generating static files; #_upload_to_s3
responsible for actual uploading.
Owner

mtigas commented Jul 5, 2012

Works as expected and results in super fast re-deploys when only updating certain files.

This. Is. Awesome.

Thanks!

mtigas added a commit that referenced this pull request Jul 5, 2012

Merge pull request #2 from rlander/master
Check if generated file was modified before uploading to S3

@mtigas mtigas merged commit f7fb60a into mtigas:master Jul 5, 2012

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment