Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

highlevel: detect if spec properly mounted and unmodified #4047

Merged
merged 63 commits into from Sep 28, 2021

Conversation

qwepoizt
Copy link
Contributor

@qwepoizt qwepoizt commented Sep 15, 2021

Fixes #3998

Basics

These points need to be fulfilled for every PR:

  • Short descriptions of your changes are in the release notes
    (added as entry in doc/news/_preparation_next_release.md which
    contains _(my name)_)
    Please always add something to the release notes.
  • Details of what you changed are in commit messages
    (first line should have module: short statement syntax)
  • References to issues, e.g. close #X, are in the commit messages.
  • The buildservers are happy. If not, fix in this order:
    • add a line in doc/news/_preparation_next_release.md
    • reformat the code with scripts/dev/reformat-all
    • make all unit tests pass
    • fix all memleaks
  • The PR is rebased with current master.

If you have any troubles fulfilling these criteria, please write
about the trouble as comment in the PR. We will help you.
But we cannot accept PRs that do not fulfill the basics.

Checklist

Check relevant points but please do not remove entries.
For docu fixes, spell checking, and similar none of these points below
need to be checked.

  • I added unit tests for my code
  • I fully described what my PR does in the documentation
    (not in the PR description)
  • I fixed all affected documentation
  • I added code comments, logging, and assertions as appropriate (see Coding Guidelines)
  • I updated all meta data (e.g. README.md of plugins and METADATA.ini)
  • I mentioned every code not directly written by me in THIRD-PARTY-LICENSES

Review

Reviewers will usually check the following:

Labels

If you are already Elektra developer:

  • Add the "work in progress" label if you do not want the PR to be reviewed yet.
  • Add the "ready to merge" label if the basics are fulfilled and you also
    say that everything is ready to be merged.

@qwepoizt
Copy link
Contributor Author

Mentioning @markus2330 as discussed.

Tobias Schubert added 22 commits September 17, 2021 10:06
… "kdb spec-mount" and improve error message
See inline comments for the rationale of this algorithm.
@qwepoizt
Copy link
Contributor Author

@markus2330: I'm very happy to report, that - from my point of view - this is now ready to merge!

I've fixed the problems with arrays and some others reported by the build servers.

The 4 unsuccessful checks are caused by:

@qwepoizt qwepoizt marked this pull request as ready for review September 22, 2021 14:13
@qwepoizt qwepoizt changed the title Draft: highlevel: detect if spec properly mounted (reopened from #4027) highlevel: detect if spec properly mounted (reopened from #4027) Sep 22, 2021
@qwepoizt qwepoizt changed the title highlevel: detect if spec properly mounted (reopened from #4027) highlevel: detect if spec properly mounted and unmodified Sep 22, 2021
Copy link
Contributor

@markus2330 markus2330 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the PR!

src/tools/kdb/gen/highlevel/highlevel.cpp Outdated Show resolved Hide resolved
src/tools/kdb/gen/highlevel/highlevel.cpp Outdated Show resolved Hide resolved
tests/shell/gen/highlevel/commands.expected.c Outdated Show resolved Hide resolved
src/libs/ease/hash.c Outdated Show resolved Hide resolved
src/libs/highlevel/elektra.c Outdated Show resolved Hide resolved
Copy link
Contributor

@markus2330 markus2330 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! 2 (hopefully small) points are yet open for clarification.

tests/ctest/test_hash.c Show resolved Hide resolved
src/libs/ease/hash.c Outdated Show resolved Hide resolved
@kodebach
Copy link
Member

@qwepoizt In kdb gen we generate a keyset with cascading names and without parentKey prefix. The C++ code calls it simply spec. This keyset is used to generate the embeddedSpec() function. Here is an example result from the tests:

static KeySet * embeddedSpec (void)
{
return ksNew (14,
keyNew ("/", KEY_META, "command", "", KEY_META, "default", "", KEY_META, "gen/command/function", "commandKdb", KEY_META, "mountpoint", "tests_gen_elektra_commands.ini", KEY_META, "type", "string", KEY_END),
keyNew ("/dynamic/#", KEY_META, "args", "remaining", KEY_META, "default", "", KEY_META, "type", "string", KEY_END),
keyNew ("/get", KEY_META, "command", "get", KEY_META, "default", "", KEY_META, "gen/command/function", "commandKdbGet", KEY_META, "type", "string", KEY_END),
keyNew ("/get/keyname", KEY_META, "args", "indexed", KEY_META, "args/index", "0", KEY_META, "default", "", KEY_META, "type", "string", KEY_END),
keyNew ("/get/maxlength", KEY_META, "default", "-1", KEY_META, "opt/arg", "required", KEY_META, "opt/long", "max-length", KEY_META, "type", "long", KEY_END),
keyNew ("/get/meta", KEY_META, "command", "meta", KEY_META, "default", "", KEY_META, "gen/command/function", "commandKdbGetMeta", KEY_META, "type", "string", KEY_END),
keyNew ("/get/meta/keyname", KEY_META, "args", "indexed", KEY_META, "args/index", "0", KEY_META, "default", "", KEY_META, "type", "string", KEY_END),
keyNew ("/get/meta/metaname", KEY_META, "args", "indexed", KEY_META, "args/index", "1", KEY_META, "default", "", KEY_META, "type", "string", KEY_END),
keyNew ("/get/meta/verbose", KEY_META, "default", "0", KEY_META, "opt", "v", KEY_META, "opt/arg", "none", KEY_META, "opt/long", "verbose", KEY_META, "type", "boolean", KEY_END),
keyNew ("/get/verbose", KEY_META, "default", "0", KEY_META, "opt", "v", KEY_META, "opt/arg", "none", KEY_META, "opt/long", "verbose", KEY_META, "type", "boolean", KEY_END),
keyNew ("/printversion", KEY_META, "default", "0", KEY_META, "opt", "v", KEY_META, "opt/arg", "none", KEY_META, "opt/long", "version", KEY_META, "type", "boolean", KEY_END),
keyNew ("/setter", KEY_META, "command", "set", KEY_META, "default", "", KEY_META, "gen/command/function", "commandKdbSet", KEY_META, "type", "string", KEY_END),
keyNew ("/setter/keyname", KEY_META, "args/index", "0", KEY_META, "default", "", KEY_META, "type", "string", KEY_END),
keyNew ("/setter/value", KEY_META, "args/index", "1", KEY_META, "default", "", KEY_META, "type", "string", KEY_END),
KS_END);
;
}

So for your solution, you need to remove the parentKey prefix as well as the namespace.

However, I think there is much easier solution. You can simply duplicate the KeySet during kdb gen and add the parentKey as a prefix (including the spec namespace). Then you should have the exact same keys in kdb gen and at runtime. Since kdb gen only runs once and at compile-time, the cost of duplicating everything doesn't matter here (if it is even noticeable at all).

@qwepoizt
Copy link
Contributor Author

@markus2330 @kodebach Thank you for your hints!

I've fixed this and am waiting for the checks to run through.

Copy link
Member

@kodebach kodebach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've found two more minor things, otherwise I'd say this looks very good.

Comment on lines 72 to 85
/**
* Ignore array parents for token calculation.
* Rationale: There is a bug in the spec plugin that is triggered on changing the size of an array.
* It leads to array parents vanishing from the spec namespace and thus a different token.
* See https://github.com/ElektraInitiative/libelektra/issues/4061
*/
size_t baseNameSize = keyGetBaseNameSize (currentKey);
char * kBaseName = elektraMalloc (baseNameSize * sizeof (char));
keyGetBaseName (currentKey, kBaseName, baseNameSize);
if (strcmp (kBaseName, "#") == 0)
{
continue;
}
elektraFree (kBaseName);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this much simpler snippet work here too:

if (strcmp (keyBaseName (currentKey), "#") == 0)
{
    continue;
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well spotted, thanks! I wasn't aware this function existed. I've replaced it accordingly!

* For token calculation, only keys from the spec namespace are relevant.
* Thus, the parentKeyForTokenCalculation must be in spec namespace.
*/
Key parentKeyForTokenCalculation (parentKeyName, KEY_END);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can avoid all of this parentKeyForTokenCalculation stuff, by simply requiring that parentKeyName starts with spec:/. We already check in highlevel.cpp that it starts with spec:/ or / (and then generate the other variant). Theoretically other templates might not have this requirement, but for now kdb gen only makes sense for a spec:/ parent, so we might as well enforce this properly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your suggestion! Unfortunately, I don't have time to implement and test this change.

I propose to merge this PR and make any further changes in a new one!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, merging as is is totally fine for me.

Copy link
Member

@mpranj mpranj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you so much, great work! 👍

Thank you for the review as well.

@mpranj mpranj merged commit 45f1769 into ElektraInitiative:master Sep 28, 2021
@mpranj mpranj added this to the 0.9.8 milestone Sep 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Help message not printed when high level API with code generation is used
4 participants