Join GitHub today
GitHub is home to over 36 million developers working together to host and review code, manage projects, and build software together.Sign up
Cache problems #2806
I still highly suspect that these disabled parts of
However, I cannot verify that, because I simply cannot produce a cache hit. This is because a cache miss is caused, if the resolver returns NO_UPDATE for any of the backends.
A cache hit only happens, if CACHE_HIT was returned for all backends.
Not sure, whether this is intended or not. I also couldn't find any test cases for the cache, through which I could find out how to get a cache hit.
My suspicion of the bug in
It is intended, yes. Without even testing it, I also think that you are right with your suspicion.
The current implementation (the diabled parts) yield about 80x improvement vs. about 10x improvement when the positions are run every time. The slow part was mostly spec afair. This is why we chose to disable these positions, as nobody was using them and the spec plugin was slow there.
If you feel like something essential was disabled, feel free to enable it again. If the plugins at that position are fast, it won‘t even be noticed.
You also need do a
As for the NO_UPDATE case in
For the second case not calling plugins is definitely correct. This is because the idea behind
Case 1 on the other hand requires a more strict interpretation of what
Anyway, I am marking this issue as
Are you sure about this? Where does
Other than that, CACHE_HIT should execute exactly the same global positions as NO_UPDATE.
Yes, just tried it.
Anyway the nature of
Right now both execute no plugins, except
To make it correct, CACHE_HIT has to execute
That is really bad, but also makes no sense to me. CACHE_HIT can only occur, where a resolver reports that the file timestamps have not changed. What you are describing would mean that those
If you know of a specification of the global plugin positions, please refer me to it. This is exactly the point where I stopped working on the implementation of the global plugins. I asked about the semantics of some positions, but nobody really knows what each position should do. Now we have even more positions, but the semantics are still undocumented (to my knowledge).
What seems to be another problem to me here are contradicting design goals. On the one hand we want to cache the data from kdbGet, on the other hand we have plugins injecting keys dynamically during a get operation. During the design of the cache we discussed that a good mmap cache implementation would make a good argument against allowing such semantics (injecting dynamic data like timestamps etc.).
That being said, as I already mentioned, we can enable any needed positions.
Are these the same as in NO_UPDATE or do you mean literally all positions here?
I don't know of any specification either. I just know about
That it is true, but right now I think just running
The keys from
Thank you for this beautiful discussion.
Yes, I am fine with that.
The only specification we have is doc/decisions/global_plugins.md which is unfortunately not finished and not implemented. @mpranj will also not implement it now as he also wants to finish his thesis. Maybe @mpranj is right and we should add positions when they are needed with the semantics as needed. This might lead to a mess if we are not very attentive to new positions but it would avoid many useless positions.
Actually, we want the semantics that it will only run once after kdbOpen (also in the case of cache hits) if the parentKey contains proc. In later executions it will not be needed as cmd-line args and env cannot change after an executable has been started. (We ignore setenv as this would be only misuse of Elektra.)
We definitely need a special case for procstorage by design.
Thank you for elaborating! I think your last proposal sounds good for now too (but in general, this is something we need to think about).
Again, if they really do not belong to a backend, then they are never cached. I tested this sometime by hand but I'll write a test right now to confirm it. Otherwise it's a bug in my implementation.
Actually, it does not make sense to cache proc keys at all.
The spec could have changed and therefore the result might change. Maybe the best solution is to specify: If the cache is used you may only call
Like I said I think the problem is the cascading mountpoint from
If you want to try it yourself:
Import this spec as
 mountpoint = erm.conf [emptydirs] opt/arg = none opt/long = dir opt = d description = remove empty directories [files/#] description = the files that shall be deleted args = remaining env = FILES [recursive] opt/#1 = R opt/#0/arg = none opt/#0 = r opt/#1/arg = none opt = #1 description = remove directories and their contents recursively opt/#0/long = recursive [interactive] opt/arg/name = WHEN opt/#1 = I opt/#0/arg = optional opt/#0 = i opt/#1/arg = none opt = #1 description = prompt according to WHEN: never, once (-I), or always (-i), without WHEN, prompt always opt/#0/long = interactive opt/#1/flagvalue = once opt/#0/flagvalue = always [nopreserve] opt/arg = none opt/long = no-preserve-root description = do not treat '/' specially [singlefs] opt/arg = none opt/long = one-file-system description = when removing a hierarchy recursively, skip any directory that is on a file system different from that of the corresponding line argument [showversion] opt/arg = none opt/long = version description = output version information and exit [preserve] opt/arg/name = all opt/arg = optional opt/long = preserve-root description = do not remove '/' (default), with 'all', reject any command line argument on a separate device from its parent opt/flagvalue = root [verbose] opt/arg = none opt/long = verbose opt = v description = explain what is being done env = VERBOSE [force] opt/arg = none opt/long = force opt = f description = ignore nonexistent files and arguments, never prompt
EDIT: forgot the most important step:
Spec-mount the specification:
kdb spec-mount /sw/org/erm/#0/current
END OF EDIT
Set values for all the namespaces:
cd $ELEKTRA_SRC_ROOT/examples kdb set -N dir /sw/org/erm/#0/current/verbose 0 kdb set -N user /sw/org/erm/#0/current/verbose 0 kdb set -N system /sw/org/erm/#0/current/verbose 0
(Note: The cd is important for the dir namespace)
Now you can complie and debug
With the specload plugin the spec changes would hopefully only be "sane" and compatible to the previous ones. If you drop command-line options or change types you would need a new major version of the spec.
An application cannot (and should not) know if a cache is used.
kdbGet() executed repeatedly without config changes should be a no-op.
Thanks for the test! I would love to see this as shell-recorder test.
Of course specload is validation and:
But from an application that was started with
So I think for now (for LCDproc) it is safe to assume that command-line options are not changed for running applications, so changes in spec/ are ignored. And with this assumption, changes in proc/ are not possible due to its semantics.