New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a routine to refresh the working set #373

Closed
bb-migration opened this Issue Apr 23, 2015 · 10 comments

Comments

Projects
None yet
1 participant
@bb-migration

bb-migration commented Apr 23, 2015

Originally reported by: stonebig (Bitbucket: stonebig, GitHub: stonebig)


Hi,

I'm using this to get list of current package.

pip.get_installed_distributions()

Unfortunately:

  • it doesn't refresh if I add a package via another process,
  • I didn't find an option .refresh()
  • trying to trick by importing pip under a different name didn't work,
  • so I currently use a complex workaround (launching another separate python session).

rbtcollins pointed me to pkg_resources API, here, but the document is obscure for me.

Could you spare a few minutes to give me an example on how to obtain the wished effect ?

see pypa/pip#2695 (comment)


@bb-migration

This comment has been minimized.

bb-migration commented Apr 23, 2015

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


@stonebig What do you get if you invoke

import pkg_resources, imp
imp.reload(pkg_resources)

I haven't even looked at the code, but that might do the trick.

@bb-migration

This comment has been minimized.

bb-migration commented Apr 23, 2015

Original comment by stonebig (Bitbucket: stonebig, GitHub: stonebig):


This doesn't work (on Python 3.4 / pip 0.6.8 / setuptools 15.1, at least )

                import pip, pkg_resources, imp
                imp.reload(pkg_resources)
                pip_list = [(i.key, i.version)
                             for i in pip.get_installed_distributions()]

I'm forced to do this ugly thing, which creates a totally different python process:

                #  indirect way: we interrogate something else
                cmdx=[osp.join(self.target, 'python.exe'), '-c',
                      "import pip;print('+!+'.join(['%s@+@%s@+@' % (i.key,i.version)  for i in pip.get_installed_distributions()]))"]
                p = subprocess.Popen(cmdx, shell=True, stdout=subprocess.PIPE,
                                     cwd=self.target)
                stdout, stderr = p.communicate()
                start_at = 2 if sys.version_info >= (3,0) else 0
                pip_list = [line.split("@+@")[:2] for line in
                            ("%s" % stdout)[start_at:].split("+!+")]

(and which, I fear, may not work on a korean Windows PC computer using Python 2.7)

Maybe, without fixing the issue, there is another way not implying reading a DOS output

@bb-migration

This comment has been minimized.

bb-migration commented Apr 23, 2015

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


What if you invoke pkg_resources.find_distributions on each element in your sys.path? Does that find what you're looking for? That won't work for packages that are installed to a location that wasn't on sys.path at startup, but for pip-installed packages, it might work.

@bb-migration

This comment has been minimized.

bb-migration commented Apr 23, 2015

Original comment by stonebig (Bitbucket: stonebig, GitHub: stonebig):


I have only one distribution directory in the path (no virtualenv).

I add/remove packages via a sup-process after start-up, like:

                            '-m', 'pip', 'uninstall', package.name, '-y'],
                            cwd=this_executable_path)

```

and my `in main process GUI` pip.get_installed_distributions() doesn't see any change, whatever I try.

if `pip` had a GUI, you may encounter the same behavior.
```
@bb-migration

This comment has been minimized.

bb-migration commented Apr 23, 2015

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


@stonebig What do you make of my last suggestion? Did you try it?

C:\Users\jaraco> python
Python 3.4.3 (v3.4.3:9b73f1c3e601, Feb 24 2015, 22:44:40) [MSC v.1600 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pkg_resources
>>> import sys
>>> pkg_resources.find_distributions(sys.path[0])
<generator object find_on_path at 0x0000000003B365E8>
>>> list(pkg_resources.find_distributions(sys.path[0]))
[]
>>> list(pkg_resources.find_distributions(sys.path[1]))
[six 1.8.0 (c:\python\lib\site-packages\six-1.8.0-py3.4.egg)]
>>> list(pkg_resources.find_distributions(sys.path[2]))
[more-itertools 2.2 (c:\python\lib\site-packages\more_itertools-2.2-py3.4.egg)]

Do you see a difference after installing the package out-of-band?

@bb-migration

This comment has been minimized.

bb-migration commented Apr 23, 2015

Original comment by stonebig (Bitbucket: stonebig, GitHub: stonebig):


I'll will retry this week-end. thank you for the example: pkg_resources is a little less obscure to me.

@bb-migration

This comment has been minimized.

bb-migration commented Apr 24, 2015

Original comment by stonebig (Bitbucket: stonebig, GitHub: stonebig):


indeed,

at the same place in code

  • this doesn't refresh the package list:
                import pip, pkg_resources, imp
                imp.reload(pkg_resources)
                pip_list = [(i.key, i.version)
                             for i in pip.get_installed_distributions()]
  • but this does refresh the package list:
                import pip, pkg_resources, imp
                imp.reload(pkg_resources)
                pip_list = [(i.key, i.version) for i in pkg_resources.find_distributions(sys.path[5])]  

But then there is the is_local() test to re-implement, not a one liner.

I whish a simpler option could be added in pip.

@bb-migration

This comment has been minimized.

bb-migration commented Apr 26, 2015

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


Extract function for initializing the master working set. Fixes #373.

@bb-migration

This comment has been minimized.

bb-migration commented Apr 26, 2015

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


The 15.2 release has this functionality, but it is exposed provisionally. In fact, it's currently presented as a private function (with a leading underscore). Feel free to try it out. Because it has potentially broad effects to refresh the working set of a running environment, it may not work in all cases (such as when sys.path has changed or when sys.path hasn't been changed for relevant changes). However, for the use-case you've described, I believe you need only now invoke pkg_resources._initialize_master_working_set.

@bb-migration

This comment has been minimized.

bb-migration commented Apr 26, 2015

Original comment by stonebig (Bitbucket: stonebig, GitHub: stonebig):


My usage is a standalone "Packet management" GUI, so there is no broad effect expected.

I will shift to the new private function when setuptools 15.2+ will be in initial Python Windows installers.

Thanks again for the solution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment