You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Reading through old issues, I found some potentially lower hanging fruit that can really help our memory usage in #4720 The way we load our indexers is a huge memory sink, since Cardigann currently holds the entire YAML file in static memory and reparses the definition on every use of the tracker. So this enhancement has potentially two parts that can be implemented in one of several configurations to significantly lower running memory usage:
We can "lazy load" the cardigann indexers. Possible implementation: we index all .yml files but not load in their definition unless they're a configured indexer. We read enough in to get the Add Indexer configuration page working, but then close the file and remove the definition from running memory. We could also delete the definition from memory when the indexer is deleted from the active indexer list. (dynamic load + unload)
We can revamp how cardigann indexers are loaded in memory. Instead of holding so many strings to the point where the YAML file is almost completely loaded and held in memory, it might be good to roll them into enums or some smaller data type. Maybe even using reflection to basically compile a "new class" based on the definition when we do.
The text was updated successfully, but these errors were encountered:
I support this feature request but with low priority.
I think we have to finish the httpclient refactor and fix several open issues before working on this.
Reading through old issues, I found some potentially lower hanging fruit that can really help our memory usage in #4720 The way we load our indexers is a huge memory sink, since Cardigann currently holds the entire YAML file in static memory and reparses the definition on every use of the tracker. So this enhancement has potentially two parts that can be implemented in one of several configurations to significantly lower running memory usage:
We can "lazy load" the cardigann indexers. Possible implementation: we index all .yml files but not load in their definition unless they're a configured indexer. We read enough in to get the Add Indexer configuration page working, but then close the file and remove the definition from running memory. We could also delete the definition from memory when the indexer is deleted from the active indexer list. (dynamic load + unload)
We can revamp how cardigann indexers are loaded in memory. Instead of holding so many strings to the point where the YAML file is almost completely loaded and held in memory, it might be good to roll them into enums or some smaller data type. Maybe even using reflection to basically compile a "new class" based on the definition when we do.
The text was updated successfully, but these errors were encountered: