New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ksAppendKey locks name, ksPop does not unlock name #2202

Open
mpranj opened this Issue Aug 21, 2018 · 6 comments

Comments

Projects
None yet
3 participants
@mpranj
Contributor

mpranj commented Aug 21, 2018

It makes sense that ksAppendKey() locks the key name, since it determines the position of the key in the keyset. However, I did not find an API function that clears the KEY_FLAG_RO_NAME flag.

It would seem intuitive to me, that ksPop (or lookup with KDB_O_POP) would unlock the name again. Is a key not meant to be re-used? Am I missing some important part of the API design, or would my suggestion be of interest.

The question comes from the context of writing the storage plugin tests/scenarios. I moved many tests out of my mmapstorage plugin to ctest. Currently, it tests dump and mmapstorage only.

@mpranj mpranj added the question label Aug 21, 2018

@mpranj mpranj changed the title from ksAppendKey locks name, ksPop() does not unlock name to ksAppendKey locks name, ksPop does not unlock name Aug 21, 2018

@e1528532

This comment has been minimized.

Show comment
Hide comment
@e1528532

e1528532 Aug 21, 2018

Contributor

I've encountered the same issue in the pluginprocess library and worked around by copying the relevant stuff manually after i finally understood what caused the issue :) To me this behavior was also unintuitive.

Contributor

e1528532 commented Aug 21, 2018

I've encountered the same issue in the pluginprocess library and worked around by copying the relevant stuff manually after i finally understood what caused the issue :) To me this behavior was also unintuitive.

@markus2330

This comment has been minimized.

Show comment
Hide comment
@markus2330

markus2330 Aug 21, 2018

Contributor

Yes, you are right. If the ref counter is back to its original value, unlocking a key's name should be safe. Are there unit tests checking for this?

Contributor

markus2330 commented Aug 21, 2018

Yes, you are right. If the ref counter is back to its original value, unlocking a key's name should be safe. Are there unit tests checking for this?

@mpranj

This comment has been minimized.

Show comment
Hide comment
@mpranj

mpranj Aug 21, 2018

Contributor

Are there unit tests checking for this?

I'm not sure what you mean. I encountered it when writing tests. There is however, no unit test, checking whether a key name is unlocked when the ref counter is 0 again.

Contributor

mpranj commented Aug 21, 2018

Are there unit tests checking for this?

I'm not sure what you mean. I encountered it when writing tests. There is however, no unit test, checking whether a key name is unlocked when the ref counter is 0 again.

@markus2330

This comment has been minimized.

Show comment
Hide comment
@markus2330

markus2330 Aug 21, 2018

Contributor

There is however, no unit test, checking whether a key name is unlocked when the ref counter is 0 again.

I meant if there is a unit test checking the other way round (if the key name stays locked, even after removing it from every KeySet). This would break the ABI tests then. If the test suite says nothing after changing this behavior, we are fine about that.

Another problem might be with applications manually tempting with the ref counter. One could add a key to a keyset, then decrease the ref counter, then change the name, breaking the KeySet's invariant (which now got even more important because of the OPMHP from @KurtMi). To solve this problem we would need a second ref counter which only remembers the number of KeySet's a key is in.

Contributor

markus2330 commented Aug 21, 2018

There is however, no unit test, checking whether a key name is unlocked when the ref counter is 0 again.

I meant if there is a unit test checking the other way round (if the key name stays locked, even after removing it from every KeySet). This would break the ABI tests then. If the test suite says nothing after changing this behavior, we are fine about that.

Another problem might be with applications manually tempting with the ref counter. One could add a key to a keyset, then decrease the ref counter, then change the name, breaking the KeySet's invariant (which now got even more important because of the OPMHP from @KurtMi). To solve this problem we would need a second ref counter which only remembers the number of KeySet's a key is in.

@mpranj

This comment has been minimized.

Show comment
Hide comment
@mpranj

mpranj Aug 21, 2018

Contributor

For me the problem is "solved". I work around this by clearing the flag by hand (since in the unit tests I know I hold the only reference to it). I wanted to be able to pop a key from a keyset, change the name, and then let the storage plugin store it again. In the case of mmapstorage, it's a very interesting test case since it changes a lot in the data and reveals memory handling problems.

The code change seems trivial. If it really has large implications, we can put it to future ideas or close it.

Contributor

mpranj commented Aug 21, 2018

For me the problem is "solved". I work around this by clearing the flag by hand (since in the unit tests I know I hold the only reference to it). I wanted to be able to pop a key from a keyset, change the name, and then let the storage plugin store it again. In the case of mmapstorage, it's a very interesting test case since it changes a lot in the data and reveals memory handling problems.

The code change seems trivial. If it really has large implications, we can put it to future ideas or close it.

markus2330 added a commit that referenced this issue Aug 21, 2018

@markus2330

This comment has been minimized.

Show comment
Hide comment
@markus2330

markus2330 Aug 21, 2018

Contributor

Yes, it is no problem if you clear the flag in the unit test (it is, however, not an ABI test then).

Contributor

markus2330 commented Aug 21, 2018

Yes, it is no problem if you clear the flag in the unit test (it is, however, not an ABI test then).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment