Add json output to info command (both console and file) #4359

jgsogo · 2019-01-22T11:42:00Z

Changelog: Feature: Add JSON output to 'info' command
Docs: conan-io/docs#1050

Refer to the issue that supports this Pull Request. closes Support conan info --json for all outputs, and support stdout #3694
I've opened another PR in the Conan docs repo to the develop branch, documenting this one.

It digests the information of the graph and then outputs it using JSON format (all the fields) or through the console (some logic still there)

jgsogo · 2019-01-22T11:42:52Z

conans/client/command.py

@@ -475,8 +475,7 @@ def info(self, *args):
                            "specify both install-folder and any setting/option "
                            "it will raise an error.")
        parser.add_argument("-j", "--json", nargs='?', const="1", type=str,
-                            help='Only with --build-order option, return the information in a json.'
-                                 ' e.g --json=/path/to/filename.json or --json to output the json')
+                            help='Path to a json file where the information will be written')


I've intentionally removed the mention to --json (without value) will output to console, I think it is something we don't want to promote. Shall I revert it?

why not promote? isn't such text interface used as a base for unix command-line pipelining, e.g. conan --json | some_tool | some_other_tool?

We cannot make this commitment right now, there are some commands that are outputting a lot of information to the console and then generating the json data (like the info one with the profiles information), so you cannot pipe the output to the next tool.

It would be a nice-to-have, it will require to silent all the output and then just print the JSON data. Something to do, not so hard, but not yet.

More issues related to output: #4225 (what we are talking here), and two more issues also worth reading #4215, #4067

I am not sure we can just remove this despite the fact that I don't like it as it is either. We will need to discuss it

I'm talking about removing it just from the --help output.

ok, it is fine for me

conans/client/command.py

jgsogo · 2019-01-22T11:45:14Z

conans/client/conan_command_output.py

+                ref = str(conanfile)
+
+            item_data["reference"] = str(ref)
+            item_data["is_ref"] = isinstance(ref, ConanFileReference)


I need this field in the JSON to perform some logic in the console output: will output Remote: None if it is a ConanFileReference, but it won't if it isn't... same with Binary remote:

... if we do not consider breaking to remove this Remote: None I will be very pleased to get rid of them

…hose are requested

conans/client/conan_command_output.py

conans/client/command.py

conans/test/functional/editable/commands/info_on_child_test.py

conans/test/functional/command/info_command_test.py

danimtb · 2019-01-23T14:40:13Z

conans/client/command.py

@@ -475,8 +475,7 @@ def info(self, *args):
                            "specify both install-folder and any setting/option "
                            "it will raise an error.")
        parser.add_argument("-j", "--json", nargs='?', const="1", type=str,
-                            help='Only with --build-order option, return the information in a json.'
-                                 ' e.g --json=/path/to/filename.json or --json to output the json')
+                            help='Path to a json file where the information will be written')


I am not sure we can just remove this despite the fact that I don't like it as it is either. We will need to discuss it

lasote · 2019-01-28T07:41:18Z

conans/client/conan_command_output.py

+                item_data["export_folder"] = self.cache.export(ref)
+                item_data["source_folder"] = self.cache.source(ref, conanfile.short_paths)
+                if isinstance(self.cache, SimplePaths):
+                    # @todo: check if this is correct or if it must always be package_id()


You have the package_id already in the main for iteration. Why to do all this?

👍 I can remove one line, the package_id variable is already available in the for; but the block related to the build_folder and the @todo section come from #1032, it should require a deeper rationale.

The two calls to conanfile.info.package_id() yes. I understand the rest is necessary.

conans/client/conan_command_output.py

lasote · 2019-01-28T07:46:15Z

conans/client/command.py

-            else:
+            if args.json:
+                json_arg = True if args.json == "1" else args.json
+                self._outputer.json_info(deps_graph, json_arg, get_cwd(), show_paths=args.paths)


I think we could output always the paths when json output and forbid the args.paths and args.json together. It simplifies a bit everything and I would expect a json to be always complete.

For packages installed as editable access to paths raises. There are two options:

keep the argument, if the user asks for paths, it will raise.

always True and try/except the paths section, paths won't be returned and the user won't know why.

lasote · 2019-01-28T07:51:14Z

conans/client/conan_command_output.py

+        data = [str(n) for n in nodes_to_build]
+        self._handle_json_output(data, json_output, cwd)
+
+    def _grab_info_data(self, deps_graph, grab_paths):


I think this "conversion function" doesn't belong to this module. A to_json in the graph? Not very fan either...

This class is created and called from outside the conan_api, the graph shouldn't have left the api. I'm sure that this json should be the output of the conan_api::info function, but this would be a refactor (I vote for it) and not the objective of this PR.

Agree. Open the engineering issue, please.

…info-json

jgsogo added 2 commits January 22, 2019 12:25

add json output to info command (both console and file)

45ca72b

add handy function

8af3862

ghost assigned jgsogo Jan 22, 2019

ghost added the stage: review label Jan 22, 2019

jgsogo commented Jan 22, 2019

View reviewed changes

conans/client/command.py Outdated Show resolved Hide resolved

jgsogo commented Jan 22, 2019

View reviewed changes

jgsogo added 3 commits January 22, 2019 12:46

typo

46d2a93

cannot get paths from editable packages (but still need to raise if t…

102566e

…hose are requested

cannot check hashes, those are different on win/linux

7827f29

SSE4 reviewed Jan 22, 2019

View reviewed changes

conans/client/conan_command_output.py Outdated Show resolved Hide resolved

unify common code in a single function

432cf2d

jgsogo requested a review from danimtb January 22, 2019 12:50

jgsogo assigned lasote Jan 22, 2019

jgsogo added this to the 1.12 milestone Jan 22, 2019

danimtb reviewed Jan 23, 2019

View reviewed changes

jgsogo added 2 commits January 23, 2019 15:55

typo

9c8c785

output to json and graph from the same command

cf3ed89

lasote reviewed Jan 28, 2019

View reviewed changes

lasote unassigned jgsogo Jan 28, 2019

lasote requested a review from memsharded January 28, 2019 08:12

lasote assigned memsharded Jan 28, 2019

remove duplicated var

325f12d

ghost assigned jgsogo Jan 28, 2019

Merge remote-tracking branch 'conan/develop' into issue/3694-command-…

c5a9084

…info-json

lasote unassigned jgsogo Jan 28, 2019

jgsogo added 2 commits January 28, 2019 11:20

remove redundant call

50bd677

pep8

8c17b8e

ghost assigned jgsogo Jan 28, 2019

memsharded approved these changes Jan 29, 2019

View reviewed changes

lasote merged commit 8f4aea5 into conan-io:develop Jan 29, 2019

ghost removed the stage: review label Jan 29, 2019

jgsogo deleted the issue/3694-command-info-json branch January 29, 2019 11:10

This was referenced Jan 29, 2019

Documentation for JSON output of 'conan info' command conan-io/docs#1050

Merged

Running into issue with short names #4079

Closed

Variable requirement overrides and include prereleases in version ranges #3798

Closed

danimtb mentioned this pull request Feb 27, 2019

Commands returning data like --build_order sometimes get dirty output #1757

Closed

czoido mentioned this pull request Oct 11, 2019

conan info . --json writes more than just the JSON to stdout #5895

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add json output to info command (both console and file) #4359

Add json output to info command (both console and file) #4359

jgsogo commented Jan 22, 2019 •

edited by danimtb

jgsogo Jan 22, 2019

SSE4 Jan 22, 2019

jgsogo Jan 22, 2019

danimtb Jan 23, 2019

jgsogo Jan 23, 2019

danimtb Jan 23, 2019

jgsogo Jan 22, 2019

jgsogo Jan 23, 2019

danimtb Jan 23, 2019

lasote Jan 28, 2019

jgsogo Jan 28, 2019

lasote Jan 28, 2019

lasote Jan 28, 2019

jgsogo Jan 28, 2019

lasote Jan 28, 2019

jgsogo Jan 28, 2019

lasote Jan 28, 2019

Add json output to info command (both console and file) #4359

Add json output to info command (both console and file) #4359

Conversation

jgsogo commented Jan 22, 2019 • edited by danimtb

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jgsogo commented Jan 22, 2019 •

edited by danimtb