Lock/pin pipx --list output for provisioning multiple machines?

[gvoysey]

I have four workstations I move between.

What would it look like for pipx to provide an, e.g., JSON version of pipx --list --verbose so that i could:

1. Stuff that output into a file
2. Version that file in my dotfiles repository
3. Sync all my workstation's pipx-installed packages ?

[cs01]

Sounds like a good workflow idea.

I think pipx --list --json would work, and it would just print json to stdout. I can come up with a serialization format and print it out.

[gvoysey]

sounds great! I'm not super concerned with getting into the weeds of version pinning and locking and the myriad ways there are to do that -- but i would like to preserve my injected packages, which is currently my favorite pipx subcommand :)

[itsayellow]

Continuing from some conversation in #224 ...

With regards to modifying the current metadata to be more compatible with a pipx freeze-like option: My thought on this was always that the current venv metadata is squarely for internal use for one particular system, but that this internal venv metadata could be read and translated to a more user-facing and more portable set of metadata for things like a machine-portable list of packages. I think they serve different purposes and don't necessarily need to be the same or locked together.

[pylipp]

@itsayellow Can you elaborate a bit how the current implementation would be affected? Do you think in the same direction as what @cs01 said:

I think the way to do it is first update the metadata format to not record the full path, but rather record relative paths. Then join them with the venv dir or bin dir, at runtime as necessary.

I'm interested in working on an extension but I'm not familiar with the codebase.

[itsayellow]

That comment was actually what I was replying to.

My thought is that one could leave the current venv metadata format alone. To create a pipx freeze option you could read it and then make the data more portable before writing out to some sort of file.

That's just my opinion, but I think it might be easier to keep these use-cases separate, so we can make sure each continues to function properly independently of the format of the other one.

[itsayellow]

I'm not sure you need the internal paths anyway in order to make a portable pipx-freeze type application. I would think you just need to know the packages to install, any options, and any other packages to inject. The only paths that would come into play would be if you were installing local packages (and that would be a little tricky--do you assume the local packages exist on all systems?)

[itsayellow]

In order to recreate appropriate install and inject commands to install again somewhere else, from a pipx_metadata.json you probably just need:

From the value for "main_package", record:

• "include_apps"
• "include_dependencies"
• "package_or_url"
• "pip_args"

Similarly, from each value of the "injected_packages" dict, take the following:

• "include_apps"
• "include_dependencies"
• "package_or_url"
• "pip_args"

I think that information should be sufficient, and doesn't involve any paths (unless "package_or_url" is a path to a local package.)

[pylipp]

What about the main_package.package_version field, wouldn't it be required to re-create an installation of the same version?

[itsayellow]

Currently we don't use that in installing a package, this is only information that pipx stores after it installs a package.

If you want to lock down a particular version you'd have to make that part of the PACKAGE_NAME argument to pipx install, I believe any standard pip format would work e.g. mypackage==0.1.0. In the absence of a user specifying a package in that manner, pipx currently assumes you always want the latest version.

If the user does specify a valid pip version string, I believe that will be in pipx_metadata.json in "package_or_url".

[pylipp]

If the user does specify a valid pip version string, I believe that will be in pipx_metadata.json in "package_or_url".

Gotcha; just checked one of my (pinned) installed apps, and you're assumption is right.

[itsayellow]

@gvoysey I'm going to close this now since with #660 I believe it's handled.

[gvoysey]

@itsayellow thanks very much! Looks good.

quick question -- is round-tripping supported (ie, is there something like pipx list --json > foo.json; rm -rf ~/.local/pipx/venvs; pipx --install-from foo.json) or is that yet to come?

[itsayellow]

Round-tripping (import from .json) is not yet supported unfortunately. Maybe that should be a separate issue.

[gvoysey]

If you want, i can kick that open and continue the tradition :)