Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ISSUE-222: Composer bin service. Enqueue and delete temp files #223

Merged
merged 4 commits into from
Jun 4, 2022

Conversation

DiegoPino
Copy link
Member

See #222

This pull adds:

  • New EventType and Event Subscriber. Event is triggered during and ADO's file persistence and when we are sure it happened. The Subscriber fills a queue.
  • The queue worker checks if the file is there, if its safe, if its not used by an Entity, if it has been longer or === than the max time to live and deletes it. If all matches, except time it re-enqueues (Double check Hydroponics since we are using a new D9 Class here)
  • There is also a hook that allows other modules to inform where/what is safe.
  • There is also a new Config to be seen at /admin/config/archipelago/filepersisting
    image

The queue needs to be configured to Run hopefully using Hydroponics but testing can be also done via the System Queue manager.

Other modules (e.g SB Flavors) will invoke the event too to mark its own temps are "compostable"♻️

@patdunlavey @alliomeria @aksm @giancarlobi If any of you see anything strange in the code let me know?

This commit adds:
- New Event Type
- New Event Subscriber for Temp Files
- Settings for time to live
- Queue worker
- Small adaptations (Some of this depends on D9 but i do not want to leave people behind so might as well double test on D8, that is why i use complete classes and check if they exist)
- Event dispatch when and ADO File is copied into final destination
- Schema update
@DiegoPino DiegoPino added enhancement New feature or request Events and Subscriber Digital Preservation Symfony Services Configuration Drupal 9 The version that comes after 8 and before 10 PHP8 The one that comes after 7 labels May 19, 2022
@DiegoPino DiegoPino added this to the 1.0.0 milestone May 19, 2022
@DiegoPino DiegoPino self-assigned this May 19, 2022
Totally! Thanks

Co-authored-by: Albert Min <albert.min@gmail.com>
@DiegoPino DiegoPino merged commit 7e07661 into 1.0.0 Jun 4, 2022
@patdunlavey
Copy link
Collaborator

Hi @DiegoPino , I'm just catching up on your amazing work. I absolutely ❤️ this! Thank you so much! Initial testing seems to be good.

The only fly in the ointment that I see is that for temporary AMI files it leaves the empty directories. I'm not sure if there's an easy solution for that. After the $this->filesystem->delete($uri), might it make sense to check to see if there are any remaining files in the deleted file's parent directory, and if not, remove the directory?

@DiegoPino
Copy link
Member Author

Interesting! Good catch. Yes, that is a good idea. Can you open an issue? I can code that. Thanks so much. Also, maybe we should add an extra option that is lower than 6 hours?

@DiegoPino DiegoPino deleted the ISSUE-222 branch September 2, 2022 15:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Configuration Digital Preservation Drupal 9 The version that comes after 8 and before 10 enhancement New feature or request Events and Subscriber PHP8 The one that comes after 7 Symfony Services
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants