-
Notifications
You must be signed in to change notification settings - Fork 72
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Centralized metadata storage #5
Comments
Cache in memory as well as storage. |
Affects the targets an the store. Maybe we need another class. But then it might smell of a data class |
Order of implementation:
|
Everything in a target is metadata except for the key and command. A new metadata level would be awkward. Fields of a target:
|
That's already a lot. Maybe we should decompose this.
|
Targets can now read and write their own values. They should be able to read and write their own metadata too. |
Dependency tracking and hashing needs to be centralized too. |
No need to reconstruct a whole target class from storage either. But it needs to be planned out thoughtfully. Need to sit back and let this one linger. |
Let's look at |
Let's consider a custom Another thing: let's keep a copy of the current DB in memory (at least the relevant records) and push in bulk at regular time intervals instead of transacting once per target. That should scale better in terms of speed. |
fstpackage/fst#91 would be super nice for this. |
The |
But all we need to worry about for quite some time will be the trivial meta and the data.table-based meta. And that's only to make testing faster. |
Got a meta class. Next steps:
|
For performance. Need some kind of database.
The text was updated successfully, but these errors were encountered: