ini: incompatible with mmap cache

[mpranj]

As discovered in #2457, the current cache design and implementation is incompatible when using ini as storage.

The reason is that ini uses a special meta key internal/ini/order to preserve ordering. The problem is that one and the same key, say user/some/config can have different orderings (e.g. #3 vs. #5) depending on the mount points and the parentKey of the kdbGet invocation.

The cache always assumes that unchanged configuration files result in the same keyset, which is not true with ini. The cache can also decide to return a larger keyset, if the requested parentKey is within a larger, already cached keyset.

Due to the incompatibility, we decided in #2457 to disable the cache when ini is used.

I assume that having wrong ordering keys would result in scrambled, possibly incorrect config files. Maybe @tom-wa can clarify whether this is correct?

[markus2330]

Thank you for reporting this problem!

Plugins should always return the same KeySet for the same config file, so something is severely wrong here (in ini).

depending on the mount points and the parentKey of the kdbGet invocation.

Can you elaborate? With the same mount points it also should be always the same parentKeys? Why are the parentKeys different?

[mpranj]

Can you elaborate?

Sure, but they are not the same mount point. Assume you mount default.ini to user and other.ini to user/other. Let there be two keys, user/foo and user/other/bar. Then we get different ordering meta keys for user/other/bar, depending on the invocation parentKey being either user or user/other.

The ini plugin keeps a "global variable" inside the parentKey of the invocation. The variable is incremented by the ini logic at some point. This results in having different ordering keys depending on the parentKey.

[markus2330]

Ok, now it is clear. So the INI plugin needs to be fixed to not use the global variable. (Or reset it properly in a new context.)

[mpranj]

I don't understand much of the ini plugin, but that sounds right to me.

[kodebach]

The cache always assumes that unchanged configuration files result in the same keyset, which is not true with ini.

specload might also cause problems then. Because the base configuration is queried from the application and not stored in the file associated with the mountpoint, the configuration might change if the application changes.

[kodebach]

Actually the problem for specload is different. Right now it has to be mounted with noresolver, otherwise kdbGet isn't called since the underlying file hasn't changed. Because of that it is technically compatible with mmapcache, but it will never be cached, because noresolver always reports out of date. We might need to have some sort of versioning, to know when the application has changed and specload needs to be invoked again...

[markus2330]

@kodebach actually we could just fetch the spec of the application again and compare it. (This approach would work with every noresolver plugin.) Or do you think that the number of bytes transmitted make a difference compared to executing a program?

But yes, some version schema of the spec is useful in any case.

[markus2330]

Shouldn't this issue be easily reproducibly by simply exporting all keys? (And sometimes the export would differ?)

[tom-wa]

the ordering is relative to the parentkey so it's expected behaviour that it differs when called with another parentkey/mountpoint.
not sure if it's possible to change that behaviour while still preserving the order in every case.

[markus2330]

Thank you for the answer!

@mpranj Can you elaborate why and how the parentKey gets changed?

[stale]

I mark this issue stale as it did not have any activity for one year. I'll close it in two weeks if no further activity occurs. If you want it to be alive again, ping the issue by writing a message here or create a new issue with the remainder of this issue.
Thank you for your contributions 💖

[stale]

I closed this issue now because it has been inactive for more than one year. If I closed it by mistake, please do not hesitate to reopen it or create a new issue with the remainder of this issue.
Thank you for your contributions 💖