Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make nix path-info --json return an object not array #9242

Merged
merged 4 commits into from Nov 6, 2023

Conversation

Ericson2314
Copy link
Member

Motivation

Before it returned a list of JSON objects with store object information, including the path in each object. Now, it maps the paths to JSON objects with the metadata sans path.

This matches how nix derivation show works.

Context

In the process of doing this, shuffle around the ValidPathInfo JSON rendering logic into what I hope is a better state:

Store::pathInfoToJSON was a rather baroque functions, being full of parameters to support both parsed derivations and nix path-info. The common core of each, a simple UnkeyedValidPathInfo::toJSON function, is factored out, but the rest of the logic is just duplicated and then specialized to its use-case (at which point it is no longer that duplicated).

This keeps the human oriented CLI logic (which is currently stable) and the core domain logic (export reference graphs with structured attrs, which is stable), separate, which I think is better.

Quite hilariously, none of our existing tests caught this change to path-info --json though they did use it. So just new tests need to be added.

Priorities

Add 👍 to pull requests you find important.

@Ericson2314 Ericson2314 added the new-cli Relating to the "nix" command label Oct 26, 2023
@github-actions github-actions bot added documentation with-tests Issues related to testing. PRs with tests have some priority store Issues and pull requests concerning the Nix store labels Oct 26, 2023
Copy link
Member

@thufschmitt thufschmitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks (although I'd rather have seen the refactoring and the functional change separately as it would have made the whole thing wayyy simpler to review). Just a small semantic suggestion and a couple of implementation notes.

src/nix/path-info.cc Outdated Show resolved Hide resolved
src/nix/path-info.cc Outdated Show resolved Hide resolved
@@ -125,4 +125,25 @@ std::string NarInfo::to_string(const Store & store) const
return res;
}

nlohmann::json NarInfo::toJSON(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this method used anywhere?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah right, I was going to write a round-trip unit test but didn't yet :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added unit tests!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's all nice to have it tested, but it's still dead code, isn't it? 🙃

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but I think it is still worth it. This is the one way to test that the JSON format covers all the fields --- and it's how I discovered the compression field was missing.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I have a few ways in mind I think we might eventually use this, but I would have that as part of the motivation. Future usability is indeed not a good enough reason to add dead code.)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we were to make a strict rule out of this, we'd have to move it into the test directory, but that just makes it harder to eventually use it. I think writing these is a good practice.

@Ericson2314
Copy link
Member Author

@thufschmitt Thanks for the review. I think I did all the things you asked for, and I also split it into two commits so refactors come before behavior changes.

We will need these for tests.
`Store::pathInfoToJSON` was a rather baroque functions, being full of
parameters to support both parsed derivations and `nix path-info`. The
common core of each, a simple `dValidPathInfo::toJSON` function, is
factored out, but the rest of the logic is just duplicated and then
specialized to its use-case (at which point it is no longer that
duplicated).

This keeps the human oriented CLI logic (which is currently unstable)
and the core domain logic (export reference graphs with structured
attrs, which is stable), separate, which I think is better.
Before it returned a list of JSON objects with store object information,
including the path in each object. Now, it maps the paths to JSON
objects with the metadata sans path.

This matches how `nix derivation show` works.

Quite hillariously, none of our existing functional tests caught this
change to `path-info --json` though they did use it. So just new
functional tests need to be added.
@thufschmitt thufschmitt merged commit 06d0d51 into NixOS:master Nov 6, 2023
8 checks passed
@Ericson2314 Ericson2314 deleted the path-info-map branch November 6, 2023 17:05

jsonObject["closureSize"] = getStoreObjectsTotalSize(store, closure);

if (auto * narInfo = dynamic_cast<const NarInfo *>(&*info)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

src/nix/path-info.cc:57:28: warning: unused variable ‘narInfo’ [-Wunused-variable]
   57 |                 if (auto * narInfo = dynamic_cast<const NarInfo *>(&*info)) {
      |                            ^~~~~~~

Was something lost along the way, or is narInfo truly redundant? Or is the whole conditional redundant?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

computeFSClosure is transitive-reflexive I think so we do not need narInfo (auto & p : closure below will get us what we need).

However we do still need the conditional because the invariant is "the transitive references are only expected to have NarInfo if the root has NarInfo".

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(The original code would just skip store objects without a NarInfo, making it possible to get a bogus closureDownloadSize silently if an internal invariant was broken.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation new-cli Relating to the "nix" command store Issues and pull requests concerning the Nix store with-tests Issues related to testing. PRs with tests have some priority
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

None yet

3 participants