Add a routine to refresh the working set

[bb-migration]

Originally reported by: stonebig (Bitbucket: stonebig, GitHub: stonebig)

---

Hi,

I'm using this to get list of current package.

pip.get_installed_distributions()

Unfortunately:

• it doesn't refresh if I add a package via another process,
• I didn't find an option .refresh()
• trying to trick by importing pip under a different name didn't work,
• so I currently use a complex workaround (launching another separate python session).

rbtcollins pointed me to pkg_resources API, here, but the document is obscure for me.

Could you spare a few minutes to give me an example on how to obtain the wished effect ?

see pypa/pip#2695 (comment)

---

• Bitbucket: https://bitbucket.org/pypa/setuptools/issue/373

[bb-migration]

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):

---

@stonebig What do you get if you invoke

import pkg_resources, imp
imp.reload(pkg_resources)

I haven't even looked at the code, but that might do the trick.

[bb-migration]

Original comment by stonebig (Bitbucket: stonebig, GitHub: stonebig):

---

This doesn't work (on Python 3.4 / pip 0.6.8 / setuptools 15.1, at least )

                import pip, pkg_resources, imp
                imp.reload(pkg_resources)
                pip_list = [(i.key, i.version)
                             for i in pip.get_installed_distributions()]

I'm forced to do this ugly thing, which creates a totally different python process:

                #  indirect way: we interrogate something else
                cmdx=[osp.join(self.target, 'python.exe'), '-c',
                      "import pip;print('+!+'.join(['%s@+@%s@+@' % (i.key,i.version)  for i in pip.get_installed_distributions()]))"]
                p = subprocess.Popen(cmdx, shell=True, stdout=subprocess.PIPE,
                                     cwd=self.target)
                stdout, stderr = p.communicate()
                start_at = 2 if sys.version_info >= (3,0) else 0
                pip_list = [line.split("@+@")[:2] for line in
                            ("%s" % stdout)[start_at:].split("+!+")]

(and which, I fear, may not work on a korean Windows PC computer using Python 2.7)

Maybe, without fixing the issue, there is another way not implying reading a DOS output

[bb-migration]

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):

---

What if you invoke pkg_resources.find_distributions on each element in your sys.path? Does that find what you're looking for? That won't work for packages that are installed to a location that wasn't on sys.path at startup, but for pip-installed packages, it might work.

[bb-migration]

Original comment by stonebig (Bitbucket: stonebig, GitHub: stonebig):

---

I have only one distribution directory in the path (no virtualenv).

I add/remove packages via a sup-process after start-up, like:

                            '-m', 'pip', 'uninstall', package.name, '-y'],
                            cwd=this_executable_path)

```

and my `in main process GUI` pip.get_installed_distributions() doesn't see any change, whatever I try.

if `pip` had a GUI, you may encounter the same behavior.
```

[bb-migration]

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):

---

@stonebig What do you make of my last suggestion? Did you try it?

C:\Users\jaraco> python
Python 3.4.3 (v3.4.3:9b73f1c3e601, Feb 24 2015, 22:44:40) [MSC v.1600 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pkg_resources
>>> import sys
>>> pkg_resources.find_distributions(sys.path[0])
<generator object find_on_path at 0x0000000003B365E8>
>>> list(pkg_resources.find_distributions(sys.path[0]))
[]
>>> list(pkg_resources.find_distributions(sys.path[1]))
[six 1.8.0 (c:\python\lib\site-packages\six-1.8.0-py3.4.egg)]
>>> list(pkg_resources.find_distributions(sys.path[2]))
[more-itertools 2.2 (c:\python\lib\site-packages\more_itertools-2.2-py3.4.egg)]

Do you see a difference after installing the package out-of-band?

[bb-migration]

Original comment by stonebig (Bitbucket: stonebig, GitHub: stonebig):

---

I'll will retry this week-end. thank you for the example: pkg_resources is a little less obscure to me.

[bb-migration]

Original comment by stonebig (Bitbucket: stonebig, GitHub: stonebig):

---

indeed,

at the same place in code

• this doesn't refresh the package list:

                import pip, pkg_resources, imp
                imp.reload(pkg_resources)
                pip_list = [(i.key, i.version)
                             for i in pip.get_installed_distributions()]

• but this does refresh the package list:

                import pip, pkg_resources, imp
                imp.reload(pkg_resources)
                pip_list = [(i.key, i.version) for i in pkg_resources.find_distributions(sys.path[5])]  

But then there is the is_local() test to re-implement, not a one liner.

I whish a simpler option could be added in pip.

[bb-migration]

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):

---

Extract function for initializing the master working set. Fixes #373.

[bb-migration]

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):

---

The 15.2 release has this functionality, but it is exposed provisionally. In fact, it's currently presented as a private function (with a leading underscore). Feel free to try it out. Because it has potentially broad effects to refresh the working set of a running environment, it may not work in all cases (such as when sys.path has changed or when sys.path hasn't been changed for relevant changes). However, for the use-case you've described, I believe you need only now invoke pkg_resources._initialize_master_working_set.

[bb-migration]

Original comment by stonebig (Bitbucket: stonebig, GitHub: stonebig):

---

My usage is a standalone "Packet management" GUI, so there is no broad effect expected.

I will shift to the new private function when setuptools 15.2+ will be in initial Python Windows installers.

Thanks again for the solution.