Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AEWeb cache (files and transactions) #744

Merged

Conversation

bchamagne
Copy link
Member

@bchamagne bchamagne commented Dec 6, 2022

Description

Adds LRU and LRUDisk modules to cache data on memory and on disk respectively. The algorithm used for eviction is the Least Recent Used, meaning that the oldest files read are evicted when space is required. The max size is in bytes and in config so users can control how much cache they want to support.
Theses caches are used in the AEWeb controller to cache the reference transaction (queried on every request) and to cache the files' content.

There is a few things to discuss:

  • the hash of the cache file name (currently base64(cache_key))
  • where to put the modules (currently in archethic_cache)
  • do a cast instead of a call for the put fn to not wait for I/O (will not be able to return succes boolean)

There will be a conflict when #742 is merged.

Fixes #644

Type of change

  • New feature (non-breaking change which adds functionality)

How Has This Been Tested?

Tests added to the suite.

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published in downstream modules

lib/archethic_cache/lru.ex Outdated Show resolved Hide resolved
@bchamagne
Copy link
Member Author

@Neylix reviewed this with me today. He proposed a refactoring to make the LRUDisk module simpler. In a few words, storing the size as well as the value in the ETS map will make everything easier.

lib/archethic_cache/lru.ex Outdated Show resolved Hide resolved
lib/archethic_cache/lru.ex Outdated Show resolved Hide resolved
lib/archethic_cache/lru_disk.ex Outdated Show resolved Hide resolved
@bchamagne
Copy link
Member Author

This PR is blocked by #747 and we will resume development only when it's merged.
Reason: the web_hosting_controller.ex has been refactored in mentionned ticket.

@bchamagne bchamagne marked this pull request as draft December 21, 2022 09:41
@bchamagne bchamagne marked this pull request as ready for review January 5, 2023 10:31
@bchamagne bchamagne requested a review from Neylix January 5, 2023 10:31
@bchamagne
Copy link
Member Author

This is now ready and tested!

@bchamagne bchamagne added web hosting issue regarding web hosting enhancements labels Jan 5, 2023
@samuelmanzanera samuelmanzanera merged commit 5b39bb7 into archethic-foundation:develop Jan 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core team Assigned to the core team enhancements web hosting issue regarding web hosting
Projects
None yet
Development

Successfully merging this pull request may close these issues.

AEWeb: improve files response by caching their content
4 participants