Provide Structured Data Format Options

[jdhitsolutions]

Description of the new feature / enhancement

Short of adding native PowerShell support, it would be very helpful if a command-line Winget upgrade or winget list had a format switch that would write output to a format like CSV, JSON, or even XML. That way I could run a native Winget command in a PowerShell function and more easily work with the output. Right now, parsing the native Winget output is a very, very tedious task.

Proposed technical implementation details

I'd like to run a command like this in PowerShell

winget upgrade --format json | Out-file upgrade.json

[denelon]

@jdhitsolutions we are working on PowerShell cmdlets. We are also discussing adding a switch to provide JSON output via the COM interface to help with our own ability to support PowerShell pipelining. You can see some of the work in progress on my fork.

[rkeithhill-keysight]

Since my feature request was closed as a duplicate of this one, I wanted to add my example usages here:

winget list -o csv | ConvertFrom-Csv | Where-Object Name -match Microsft | Sort-Object Name

You could also support a List format that would not truncate text e.g.:

winget list -o list

Name:      JetBrains ETW Host Service (x64) 
Id:        JetBrains.ReSharper
Version:   < 2022.2.2
Available: 2022.2.3
Source:    winget

Name: ...
...

I would imagine the default --output-format would be Table to match how winget displays by default.

[treestryder]

Seems odd, after all these years of Powershell, that any group at Microsoft would roll their own command-line parser and not pipe .Net objects to a built-in formater.

[rkeithhill-keysight]

Have you seen System.CommandLine? Or the dotnet CLI. 🤷‍♂️

[treestryder]

Isn't the "PackageManagement" Powershell module a full-fledged package manager? Why didn't they just use that?

[halr9000]

Any update on this issue? This is a serious accessibility issue for me. When you must use large terminal font sizes and lower display sizes like I do, this makes truncation an everyday problem. Listing, installing, upgrading, removing -- I always run into the dreaded "... "problem, and then have to do extra work to determine what the heck winget is trying to tell me. Sometimes, it's frustrating enough that I'll skip it and use the GUI. Which is a total fail in my book!

TIA

[denelon]

@halr9000 we've just released a native PowerShell module. It provides structured output.

The CLI was designed to provide tabular output, but clearly that fails in the face of large fonts and lower display sizes. Would another formatting mechanism work better in your opinion?

[halr9000]

@denelon cool I'll check out the module. Me and @jdhitsolutions go way back though, so if he's happy I'm sure I will be 😊

As long as it outputs objects like a native PS cmdlet would, that's what I need.

[jdhitsolutions]

I'll pick a major nit here and comment that your PowerShell commands are Crescendo wrappers around the command line tools. This is no different in concept that what many of us have already done, albeit without Crescendo. The major problem has always been creating a structured object from the winget output, which is a challenge when culture comes into the picture. Now, I have to still try this module and look at the code, but if the module is doing nothing more than parsing text, that's not much progress and it will be fragile. That's why I have always asked for structured output like JSON. It is easy enough to build a PowerShell function wrapped around winget, but if you give me an option for raw JSON data, then we can write structured, and predictable, objects to the pipeline.

[denelon]

@jdhitsolutions the initial cmdlets I wrote initially were based on Crescendo. Unfortunately, the differences between the CLI syntax and PowerShell approved nouns and verbs were a bit too much to overcome. My goal in writing them was to get feedback on the syntax, not to release them as any form of final solution.

These cmdlets are based on the COM API which is a rich object-based structure. I'm looking forward to everyone's feedback. I'm expecting the need to iterate some before publishing an initial release to the PowerShell gallery.

[jdhitsolutions]

After looking at the latest code, I see a combination of .NET and Crescendo. I'd love to see this succeed, but so far haven't been able to test it. See issue #2795.

[josh-hemphill]

@denelon

These cmdlets are based on the COM API which is a rich object-based structure.

Will the COM API be stabilizing anytime soon? At least as used by the powershell module?
A lot of the time I want structured output, it's for use in other applications, and I'd love to be able to rely on winget data without having to spawn shells. Even if it's left as mostly "undocumented", even just a few documented functions would be useful.

[denelon]

@josh-hemphill yes, it's actually what the Microsoft Store and Intune have used to integrate with.

[denelon]

We're in the process of updating the documentation for the COM API (both "in-process" and "out-of-process").

[jdhitsolutions]

After trying out the new PowerShell module, I'm going to renew my request for raw JSON output. Many command line tools have such a feature. Give me JSON or XML, and I can handle the rest in PowerShell. As it stands now, the object output from the module is more convoluted than it needs to be. I'll be posting more issues.

[jdhitsolutions]

#2796

[giggio]

After trying out the new PowerShell module, I'm going to renew my request for raw JSON output. Many command line tools have such a feature. Give me JSON or XML, and I can handle the rest in PowerShell. As it stands now, the object output from the module is more convoluted than it needs to be. I'll be posting more issues.

I second that.
I actually prefer native PowerShell objects so I can do all the work in PowerShell, but many people prefer to use GNU tools. If the cli was outputting json (or, ugh, xml), we wouldn't need PowerShell at all.

To be clear, both approaches should continue, the PowerShell module and the structured output. I'd prioritize PowerShell, which I believe is more common for Windows administration.

[halr9000]

Having caught up to this issue, what's in the new release, and the discussion in #2796, I would like to reiterate support for this issue for the following reasons.

Regardless of anyone's preferences, Winget is and will always be a console app which must be able to run outside of a PowerShell host in order to function as a universal package installer. That means native PowerShell support (which I do want) has to happen outside of the executable, which I know the team has been working on. So, I propose let's keep this issue focused on winget.exe.

Console apps which output data must always deal with the same sets of problems over and over again. What to display, how to format it, sorting, filtering -- the list is endless. This list HAS ALREADY BEEN SOLVED IN POWERSHELL. But -- that's a topic for another issue, another discussion.

Most console apps outside of Windows, or prior to PowerShell's existence which faced this problem only had two paths to go down:

1. Try to please everyone, greatly increasing the complexity of your codebase and the amount of time you have to spend on the front end stuff, not the more important core functionality. End result is something flexible, but often inconsistent and hard to learn.
2. Handle only the basics, and then provide one or more output formats, allowing for tooling and automation to happen outside of your box. Composable.

Pretty sure everyone here is already in the second camp.

If it's easy to add CSV and/or JSON (these being the most obvious formats to support),I would vote to keep this issue open and work towards that. And if you already have an internal object representation that maps well to the output for a given object, just give us that in JSON and be done. For example, half the commands will output a single package, or an array of them. That's one object to worry about; do that and you've solved 80% of the automation use cases -- maybe more.

If it's not easy, let us know. Speaking for myself, I'd then say close this issue, and let's help you get your COM-based API work for proper PowerShell support in good shape instead. (Proper means real .NET objects which avoid all the problems that @jdhitsolutions highlights in #2796 .)

[markusdd]

Wish to second this as well.

I am currently working on a rust-based TUI app which should support various package managers, including winget, and currently parsing whatever winget gives you as output is absolutely unusable, as the tables do not even have proper delimiters where you can split.
Having a --porcelain-style output that gives you a JSON, yaml, toml, csv whatever that is locale-independent and non-truncatzed is just such a obvious feature.
I do not care about Powershell at all and I do not want to spawn any shells from my application, that screams security issue right away.

I just want to be able to call winget in a subprocess from my app and get proper parseable stdout, stderr and a sane exit code.
Such a representation like a nested Map or whatever surely already exists within winget before it prints the tables, so dumping those on the stdout should be very easy to add and all save us a huge headache.

[TechStudent10]

Hey any updates? Parsing WinGet's current output is a nightmare, and I don't want to have to deal with PowerShell in my project

[brandonh-msft]

Bumping because I'm running into an issue where upgrade --all runs the Microsoft.Office upgrader which isn't (apparently) compatible with winget as it pops a blank cmd UI and just hangs. I need to, therefore, skip this particular package in the list of packages available for upgrade, but cannot parse this list because... well... this.

[denelon]

We've published the Microsoft.WinGet.Client PowerShell module which provides structured data.

For those who do not want to depend on PowerShell, WinGet has COM APIs that an application can use for integration as well.

[giggio]

We've published the Microsoft.WinGet.Client PowerShell module which provides structured data.

For those who do not want to depend on PowerShell, WinGet has COM APIs that an application can use for integration as well.

This has been discussed in this issue. The PowerShell module is incomplete and should not be the only approach. Not everyone uses PowerShell. Winget should be a good citizen in cli world, and making output difficult to parse is not doing that.
This is not that complicated, why is it being treated as such?

[jdhitsolutions]

I would still love to see winget with a format option to output JSON.

[markusdd]

agree with the takes before, very unsatisfying answer.

CLI tools with output that is not easily parseable (and hence not piepable) is very bad practice.

Powershell obejct stuff is irrelevant to almost anyone and it's incomplete, and using a COM api is shooting shotguns at flies.

And @denelon cloing this as 'completed' is a bit of a joke, because it's not solved. at all.

[TechStudent10]

agreed with all the points provided

Providing a basic JSON output for each command would be way more preferred than a PowerShell API (because no one wants to integrate PowerShell into their program)