New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new derivatives monster PR #889

merged 89 commits into from Nov 20, 2017


None yet
1 participant

jrochkind commented Oct 26, 2017

Create derivatives using our own custom architecture, which uses vips, and stores on S3.

Refactor custom viewer to use json-based image list.

Add feature-flag-triggered download links on both viewer and show page, per image. For these new images.

Adjust front-end image provision to have feature flags allowing our new stored-on-S3 derivatives, for both thumbs and downloads.

Lots of tweaks to get it right. Sorry, lots of commits here.

jrochkind added some commits Oct 12, 2017

controller action to deliver json config for viewer list of images
Not yet used. Should make loading a show page quicker, cause JS can request
all this separately. Makes extension to more download types both more clear
and more performant, this is a better seperation of layers and having JS
get delivered straight data instead of having to put it in the DOM first.
image viewer uses image info from JSON endpoint, not pre-selected in DOM
Should make the original page load faster, this is a bit cleaner architecture all around,
will make it somewhat less confusing to add more download options I hope
an sRGB profile licensed for free use, from…



Phew, color management is complicated, but this seems to work.

vips needs you to supply the sRGB.icc profile for conversion to it.

jrochkind added some commits Oct 31, 2017

rake tasks for new style derivatives
   rake chf:deriatives:s3:create
   rake chf:derivatives:legacy:create

   rake chf:derivatives:create # picks right one depending on Env config

   rake chf:derivatives:s3:clean_orphaned # uses efficient S3 LIST operations
leave enough room for scrollbars on viewer thumbs, appropriately
make small viewer have enough height for thumbs, can't totally explain
AddFileToFileSet should trigger reindex of parent if representative
This used to happen in the derivatives creation job. That was the wrong place for it (regenerating derivaties should not cause
expensiev reindex). Removed it there -- it broke batch upload. Adding it here where it belongs.
even more expansive progress bar for derivatives
Use this complicated one for all progress bars?
restructured CreateDerivativesOnS3Service a bit to be cleaner and sim…

And support faster lazy checking (without downloading the source if not neccesary), which we have here

@jrochkind jrochkind changed the title from WIP new derivatives to new derivatives monster PR Nov 20, 2017

@jrochkind jrochkind merged commit 6deca63 into master Nov 20, 2017

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
continuous-integration/travis-ci/push The Travis CI build passed

@jrochkind jrochkind deleted the customized_derivatives_architecture branch Nov 20, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment