-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
2.8.x: Qld Government core updates: Incorporate delete() and download() into IUploader and ResourceUpload for better cloud integrations #5766
Conversation
…sernames (login cookie forgery) - as sysadmins can already do anything so allow sysadmins to edit usernames - add nose test to ensure that sysadmins can change account names - adjust test password handling to match other tests - redirect after updating username - include extra environment when redirecting - use 'submit_and_follow' helper to simplify proper redirection - adjust test steps to better match successful user edit test - update context username after updating it in the account
- don't update the package modification timestamp if we're only updating the resources - don't update package modification timestamp more than once every ten seconds, this should reduce lock contention on resource updates - add comment to explain the delay on updating modification timestamp,
…load, Upload with functionality from controller resource_download to allow better cloud integrations - Add metadata to IUploader interface's - Add new resource_file_metadata_show get api to surface metadata checks, usage case is archiver or similar verifying files have not change or disappeared - Hook in delete uploader in resource delete action logic, use upload download logic fully - Remove unneeded code as its now in the ResourceUpload class, call upload delete when a resource is also deleted - Uploader and interface update so signature has filename - Allow filenames to be preserved in uploader - Add logging if delete method not found - Need package dict to call uploader correctly - Add metadata to uploader interface, next step, surface it via package…
ca3b845
to
9045839
Compare
This PR should be done against master branch as in practice no new interfaces are added to old releases. |
Thanks for the tip @Zharktas , We are on 2.8.x atm so I'm working towards seeing how well cherry-picking these commits to master goes. |
@Zharktas Note that some of these are bug fixes, like ignoring auth checks during password resets (which becomes necessary if you set |
Will do @amercader |
part 1 of breakout pr's #5791 |
Closing due to inactivity and as we do not support CKAN 2.8 anymore. Thanks for submitting! |
This is the work that has been completed by Qld Government in maintaining https://www.data.qld.gov.au and https://www.publications.qld.gov.au.
This includes a change to the IUploader interface which allows better integrations for archiver plugin or similar in verifying files with AWS s3 plugin for resource files off disk. (Allowing nearly immutable web front of house instances for high scaling without the need of a network file store for uploads)
This is backwards compatible, if a plugin extends IUploader but does not incorporate download, then it will fallback to previous functionality.
Proposed fixes:
Incorporate delete() and download() into IUploader, update ResourceUpload, Upload with functionality from controller resource_download to allow better cloud integrations
Other Fixes
Features:
Please [X] all the boxes above that apply