Detect if package is installed editable

[kasium]

Hey,

Not sure if this is the right place and/or right package to ask, but you need to start somewhere 😄 .

I have the following case: A script in a distributable should behave differently if it's started by the distributable or by anther project just importing the distributable.

For instance, the distributable has the name foo.
If I developing foo currently and I execute foo.py it should return 1. If another project depends on foo and executes foo.py after installing foo over pip it should return 2.

Since there seems to be no reliable way, for now I use the assumption that current project means editable install. With that I can use

 distribution = pkg_resources.get_distribution("foo")
 is_foo= "dist-info" not in distribution.egg_info

So if foo is installed editable, the egg_info has no dist-info. But that's very hacky.

Therefore, how can I improve this?

1. do you have anything in mind which would solve the below issue in the first place
2. if not, do you think you can expose the direct_url.json of PEP 660 somehow? It would help to detect if it's an editable install w/o finding/parsing the file on my own

[jaraco]

Since direct_url.json is meant to be a file in the metadata directory, I'd expect you to could use importlib_metadata.read_text('direct_url.json'). Give that a try and let me know if that works in your environment.

[kasium]

@jaraco yes that you work. But this means that I must rely that this file is created,whuch means that the packaging tool must follow the PEP.

1. do you think there is a more general approach?
2. I still need to check for the files existence and need to parse its content. Could maybe importlib-metadata add a function to do that?

[jaraco]

@jaraco yes that you work. But this means that I must rely that this file is created,whuch means that the packaging tool must follow the PEP.

Yes, that's right. The good news is that many/most tools now support the PEP.

1. do you think there is a more general approach?

Possibly, though I suspect not a robust one. Probably it would require coming up with some inference heuristic.

2. I still need to check for the files existence and need to parse its content. Could maybe importlib-metadata add a function to do that?

Yes, perhaps. If this behavior would prove generally useful, it could be exposed here. Can you describe more about why this behavior is useful and why it would be useful for others? My instinct is it probably could belong here, but I'd not wish to adopt the behavior if it only has narrow usefulness.

Presuming we do wish to move forward, it would be worthwhile to have a draft/demo routine that implements the logic. Would you be willing to research the details and devise an implementation?

[kasium]

Thanks a lot for your comment.

I read now the full PEP 660 and PEP 610 and I think importlib-metadata could include a parser for direct_url.json which parses the file into an object which can be received over the distribution object.

Since this is more general than my initial requirement all use cases listed in PEP 610 could benefit from this feature. In short, its then possible to get e.g. the file link and even git information over a well known API.

As a way forward I would propose a new field part of the distribution called direct_url_origin. Behind the scenes a (possible lazy) parses reads the JSON file and publishes all fields of PEP 610.

What do you think?

[jaraco]

That all sounds reasonable. I'd want to think carefully about the new field. direct_url_origin doesn't communicate to me what it is. I see PEP 610 uses the phrase Direct URL Origin in the title, but those words expand to "Direct Uniform Resource Locator Origin" and I still struggle to parse a meaning from that, especially when the primary field of that Direct URL Origin is a URL. I'm thinking maybe just origin for the property. I'm not sure Direct is meaningful in the name, and URL is a property, so presumably dist.origin.url would return the URL. I would like for the parsed object to expose the fields as attributes instead of or in addition to items (i.e. dist.origin.url and not just dist.origin['url']).

Those are my thoughts. I'm encouraged and look forward to your implementation.

[kasium]

Thanks for the ideas. Sounds like a great idea. I'll give a PR a try in the next 1/2 weeks

[kasium]

@jaraco I have some issues getting the direct_url.json file path. I thought I can use Distribution._path, but for editable installs it returns currently the egg folder.

# install importlib_metadata itself with 'pip install -e .'
import importlib_metadata
x = importlib_metadata.distribution("importlib_metadata")
x._path
# PosixPath('importlib_metadata.egg-info')

However, by using the following, I can see that also the dist-info path was found

import importlib_metadata
x = list(importlib_metadata.Distribution.discover(name="importlib_metadata"))
# x[0]._path -> PosixPath('importlib_metadata.egg-info')
# x[1]._path -> PosixPath('importlib_metadata/.venv/lib64/python3.7/site-packages/importlib_metadata-0.1.dev1283+g5573f98.dist-info')

If a user would use the default way with importlib_metadata.distribution, I'm not able to locale the JSON file. Any ideas?

[jaraco]

You don't want to use Distribution._path, because that's an implementation detail, and if you rely on it, you'll not be supporting Distributions of other forms. Instead, why not use Distribution.read_text?

I can see that it doesn't work for egg-info. Only dist-info seems to be supplying the "direct_url.json" file. That means you won't be able to supply a meaningful origin for editable installs (which rely on egg-info).

You've observed above that your environment has two distributions for importlib_metadata. That'll happen if you have it installed but you're also developing it: running tests, installing editable, or simply building the project will often generate a .egg-info in the current working directory. The presence of that directory when on sys.path will give it precedence over the installed version (hence why .distribution('importlib_metadata') returns that egg-info.

So the fact that you get None from dist.read_text('direct_url.json') is correct - there's no such file. You'll have to figure out how to handle the .origin property in that case. My instinct is .origin should be None in that case.

[kasium]

Moved to pypa/packaging#701

[jaraco]

I still think importlib_metadata should probably provide a convenience method to load the metadata file. I can see I pushed 5898112. Would that be helpful for the packaging implementation?