Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ini: incompatible with mmap cache #2592

Open
mpranj opened this Issue Apr 7, 2019 · 10 comments

Comments

Projects
None yet
4 participants
@mpranj
Copy link
Member

mpranj commented Apr 7, 2019

As discovered in #2457, the current cache design and implementation is incompatible when using ini as storage.

The reason is that ini uses a special meta key internal/ini/order to preserve ordering. The problem is that one and the same key, say user/some/config can have different orderings (e.g. #3 vs. #5) depending on the mount points and the parentKey of the kdbGet invocation.

The cache always assumes that unchanged configuration files result in the same keyset, which is not true with ini. The cache can also decide to return a larger keyset, if the requested parentKey is within a larger, already cached keyset.

Due to the incompatibility, we decided in #2457 to disable the cache when ini is used.

I assume that having wrong ordering keys would result in scrambled, possibly incorrect config files. Maybe @tom-wa can clarify whether this is correct?

@markus2330 markus2330 changed the title mmap cache: incompatible with ini ini: incompatible with mmap cache Apr 7, 2019

@markus2330

This comment has been minimized.

Copy link
Contributor

markus2330 commented Apr 7, 2019

Thank you for reporting this problem!

Plugins should always return the same KeySet for the same config file, so something is severely wrong here (in ini).

depending on the mount points and the parentKey of the kdbGet invocation.

Can you elaborate? With the same mount points it also should be always the same parentKeys? Why are the parentKeys different?

@mpranj

This comment has been minimized.

Copy link
Member Author

mpranj commented Apr 7, 2019

Can you elaborate?

Sure, but they are not the same mount point. Assume you mount default.ini to user and other.ini to user/other. Let there be two keys, user/foo and user/other/bar. Then we get different ordering meta keys for user/other/bar, depending on the invocation parentKey being either user or user/other.

The ini plugin keeps a "global variable" inside the parentKey of the invocation. The variable is incremented by the ini logic at some point. This results in having different ordering keys depending on the parentKey.

mpranj added a commit to mpranj/libelektra that referenced this issue Apr 7, 2019

@markus2330

This comment has been minimized.

Copy link
Contributor

markus2330 commented Apr 7, 2019

Ok, now it is clear. So the INI plugin needs to be fixed to not use the global variable. (Or reset it properly in a new context.)

@mpranj

This comment has been minimized.

Copy link
Member Author

mpranj commented Apr 7, 2019

I don't understand much of the ini plugin, but that sounds right to me.

@kodebach

This comment has been minimized.

Copy link
Contributor

kodebach commented Apr 7, 2019

The cache always assumes that unchanged configuration files result in the same keyset, which is not true with ini.

specload might also cause problems then. Because the base configuration is queried from the application and not stored in the file associated with the mountpoint, the configuration might change if the application changes.

mpranj added a commit to mpranj/libelektra that referenced this issue Apr 7, 2019

@kodebach

This comment has been minimized.

Copy link
Contributor

kodebach commented Apr 7, 2019

Actually the problem for specload is different. Right now it has to be mounted with noresolver, otherwise kdbGet isn't called since the underlying file hasn't changed. Because of that it is technically compatible with mmapcache, but it will never be cached, because noresolver always reports out of date. We might need to have some sort of versioning, to know when the application has changed and specload needs to be invoked again...

@markus2330

This comment has been minimized.

Copy link
Contributor

markus2330 commented Apr 7, 2019

@kodebach actually we could just fetch the spec of the application again and compare it. (This approach would work with every noresolver plugin.) Or do you think that the number of bytes transmitted make a difference compared to executing a program?

But yes, some version schema of the spec is useful in any case.

@markus2330

This comment has been minimized.

Copy link
Contributor

markus2330 commented Apr 8, 2019

Shouldn't this issue be easily reproducibly by simply exporting all keys? (And sometimes the export would differ?)

@tom-wa

This comment has been minimized.

Copy link
Contributor

tom-wa commented Apr 8, 2019

the ordering is relative to the parentkey so it's expected behaviour that it differs when called with another parentkey/mountpoint.
not sure if it's possible to change that behaviour while still preserving the order in every case.

@markus2330

This comment has been minimized.

Copy link
Contributor

markus2330 commented Apr 8, 2019

Thank you for the answer!

@mpranj Can you elaborate why and how the parentKey gets changed?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.