Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data files are unreadable when beets is installed as an egg #3264

Open
arcresu opened this issue May 11, 2019 · 3 comments

Comments

Projects
None yet
3 participants
@arcresu
Copy link
Member

commented May 11, 2019

Following up from a related Discourse thread, we noticed that beets isn't happy when it's installed as an egg. This is currently the default way beets is installed if you run setup.py install (not sure if #2083 changes this story).

Problem

In egg mode, the whole beets and beetsplug directory trees remain bundled in a zipfile and are not extracted. This causes problems when we try and read data files (aka package resources) like config_default.yaml, because we currently build up the path to such files relative to the executing module's source file and open() the resulting path. Obviously this doesn't work of the file is inside a zipfile.

  • Symptom: ConfigErrors like configuration error: verbose not found.
    Caused by the failure to open config_default.yaml (aside: it would have been clearer if we got a confit.ConfigReadError rather than missing default values, but the loading of the default config file is inside a if os.path.isfile(filename) guard so it just silently skips the file when it doesn't exist even though beets depends on the default values being set).
    Needs to be fixed in confuse.

  • Symptom: FileNotFoundErrors related to files like lastgenre/genres.txt.
    Caused by failure to open other data files needed by plugins.
    Needs to be fixed in beets.

Solutions

  • Explicitly state that we don't support running beets as an egg. This seems unfortunate if it's the result of setup.py install.
  • Try and fix the way data files are accessed. There is pkg_resources from setuptools that has helpers to transparently handle this. We could refactor these data file accesses into a helper for plugins.

@sampsyo sampsyo added the bug label May 12, 2019

@sampsyo

This comment has been minimized.

Copy link
Member

commented May 12, 2019

Indeed; I agree those are basically the two options! In my view, Python Eggs are a relic of the past—they served an important role for their time, but most of the problems they solve these days are better solved by the PEP 427 Wheel format. Here's a quick comparison between Egg and Wheel in the official docs.

That said, it does seem like python setup.py install installs an egg by default, which is too bad. I am wary of introducing a runtime dependency on setuptools, via its pkg_resources module—we currently only need it for installation, which seems like the Way It Should Be. (That is, if you were to install beets from a Wheel, you could do so without ever needing setuptools.)

It looks like you can instruct setuptools to avoid installing as an egg using the zip_safe option. Maybe we should try setting zip_safe=False and see if that forces python setup.py install to use an ordinary source installation?

@arcresu

This comment has been minimized.

Copy link
Member Author

commented May 12, 2019

It looks like it's probably not worth supporting a runtime beets egg for the reasons you mention (wheels replacing eggs and the runtime dependency on setuptools). The Hacking page currently includes an egg URL in the local setup instructions that might need changing.

I'm happy to play around with zip_safe and move us off the egg-by-default setup but I wonder if @jackwilsdon has any thoughts since his proposed changes in #2083 involve egg-related stuff.

Whatever we do, I think Confuse might need a little attention related to this scenario. I'll have a look into that separately and move that discussion to its repo.

@arogl

This comment has been minimized.

Copy link
Contributor

commented May 13, 2019

adding

zip_safe=False,

to line 68 of setup.py installs the files to a directory rather than an egg file.

I'll open a PR.

The command

pip install -e git+https://github.com/beetbox/beets.git#egg=beets

works as expected the "#egg=beets" is a required parameter

arogl added a commit to arogl/beets that referenced this issue May 13, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.