New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates to manifest storage #4430

Merged
merged 6 commits into from Aug 2, 2018

Conversation

Projects
None yet
2 participants
@davidfischer
Contributor

davidfischer commented Jul 25, 2018

Edit 2018-08-01: I removed the actual storage backends. They will be used in a separate repository with deployment customizations. This PR is now relatively minor.

This builds on #4427. This can be merged into that branch or into master and that one can be closed.

  • Because the manifest storage parses files looking for referenced files (eg. theme.css references fonts) all the referenced files must be there. This fixes a few issues found there.
  • I do question whether readthedocs/core/storage.py should be in this repository and not in our extensions. I don't think it is particularly useful to anyone and is particular to our deploy process.
  • This PR removes the references to this storage in readthedocs/settings/base.py. I think that in dev the default Django storage is fine. This should only be used in prod.
  • There will still be a linting error with this PR as the linter will try to import from Azure and django-storages and fail. I can fix this up although as I mentioned above I think this code might be better elsewhere.

Testing

  • Install dependencies

    $ pip install azure django-storages "azure-storage==0.20.3"
    
  • Set settings (readthedocs/settings/base.py or other appropriate settings)

    STATICFILES_STORAGE = 'readthedocs.core.storage.MediaAzureStorage'
    AZURE_ACCOUNT_NAME = 'YOUR ACCOUNT'
    AZURE_ACCOUNT_KEY = 'YOUR KEY'
    AZURE_CONTAINER = 'media'
    AZURE_MEDIA_STORAGE_CONTAINER = AZURE_CONTAINER
    
  • Collect static files

    $ npm run build      # takes about 10 minutes the first time and 2.5 mins thereafter
    
@davidfischer

This comment has been minimized.

Show comment
Hide comment
@davidfischer

davidfischer Jul 25, 2018

Contributor

This should be AZURE_MEDIA_STORAGE_CONTAINER, right?

Contributor

davidfischer commented on readthedocs/core/storage.py in 94cab0c Jul 25, 2018

This should be AZURE_MEDIA_STORAGE_CONTAINER, right?

@davidfischer

This comment has been minimized.

Show comment
Hide comment
@davidfischer

davidfischer Jul 25, 2018

Contributor

I think it's ok to just use settings.AZURE_BUILD_STORAGE_CONTAINER rather than doing getattr. If these storages are set in STATICFILES_STORAGE then it should blow up if the appropriate azure settings are missing.

Contributor

davidfischer commented on 94cab0c Jul 25, 2018

I think it's ok to just use settings.AZURE_BUILD_STORAGE_CONTAINER rather than doing getattr. If these storages are set in STATICFILES_STORAGE then it should blow up if the appropriate azure settings are missing.

davidfischer added some commits Jul 25, 2018

Updates for the manifest storage
- Fix missing file references with a symlink
- Fix missing font

@davidfischer davidfischer requested a review from rtfd/core Jul 25, 2018

@ericholscher

Yea, the settings change was just for testing. This can probably all just go in ext since it isn't useful for anyone else running RTD.

@davidfischer

This comment has been minimized.

Show comment
Hide comment
@davidfischer

davidfischer Aug 1, 2018

Contributor

Ok. I'm going to remove the changes to readthedocs/core/storage.py from this PR. The other changes are still required if we want to use manifest storage.

Contributor

davidfischer commented Aug 1, 2018

Ok. I'm going to remove the changes to readthedocs/core/storage.py from this PR. The other changes are still required if we want to use manifest storage.

Remove the ManifestStorage
- This will be added to -ext

@ericholscher ericholscher merged commit 42cd557 into master Aug 2, 2018

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
@ericholscher

This comment has been minimized.

Show comment
Hide comment
@ericholscher

ericholscher Aug 2, 2018

Member

👍 merging these since they are small simple fixes.

Member

ericholscher commented Aug 2, 2018

👍 merging these since they are small simple fixes.

@stsewd stsewd deleted the davidfischer/manifest-storage-update branch Aug 15, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment