Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Packaging issues #1

Closed
ganto opened this issue Nov 6, 2016 · 8 comments
Closed

Packaging issues #1

ganto opened this issue Nov 6, 2016 · 8 comments
Assignees
Labels

Comments

@ganto
Copy link

ganto commented Nov 6, 2016

I wrote a Gentoo ebuild for yaml4rst. However, I'm experiencing multiple issues with the default setup.py shipped with this project:

  • Tests are installed by default. This results in an error when using the distutils-r1 eclass:
Installing yaml4rst script to /var/tmp/portage/dev-python/yaml4rst-0.1.1/image/_python3.4/usr/lib/python-exec/python3.4
 * ERROR: dev-python/yaml4rst-0.1.1::linuxmonk-overlay failed (install phase):
 *   Package installs 'tests' package which is forbidden and likely a bug in the build system.

I could fix this error with the following patch:

diff --git a/setup.py b/setup.py
index 6cb58bb..2f2b702 100755
--- a/setup.py
+++ b/setup.py
@@ -51,7 +51,7 @@ setup(
         'Topic :: Text Processing',
     ),
     keywords='YML YAML RST reStructuresText Ansible DebOps linting docs documentation',
-    packages=find_packages(),
+    packages=find_packages(exclude=('tests',)),
     install_requires=[
         # Debian packages: python-yaml python3-yaml python-jinja2 python3-jinja2
         'PyYAML',
  • However, then the DebOps template won't be installed. You can find the build output here. With the following patch, everything seems to be installed properly:
commit 4bc0d95b94bf3894546824d16e7b1ce0c7158149
Author: Reto Gantenbein <reto.gantenbein@linuxmonk.ch>
Date:   Wed Oct 26 10:17:27 2016 +0200

    Fix packaging

diff --git a/setup.py b/setup.py
index 6cb58bb..a842c2e 100755
--- a/setup.py
+++ b/setup.py
@@ -3,7 +3,7 @@

 import re
 import os
-from setuptools import setup, find_packages
+from setuptools import setup

 here = os.path.abspath(os.path.dirname(__file__))

@@ -51,7 +51,8 @@ setup(
         'Topic :: Text Processing',
     ),
     keywords='YML YAML RST reStructuresText Ansible DebOps linting docs documentation',
-    packages=find_packages(),
+    packages=['yaml4rst'],
+    package_data={'yaml4rst': ['templates/debops/ansible/*']},
     install_requires=[
         # Debian packages: python-yaml python3-yaml python-jinja2 python3-jinja2
         'PyYAML',

I don't know setuptools so well to know what exactly is misbehaving here. Just wanted to let you know my issues.

Btw. I'm using the currently Gentoo stable dev-python/setuptools-20.6.7.

@ypid ypid added the bug label Nov 6, 2016
@ypid ypid self-assigned this Nov 6, 2016
@ypid
Copy link
Owner

ypid commented Nov 6, 2016

Thanks very much! If all issues where like this I would have not much to do anymore 😉 I will check that tomorrow and make a new release.

@ganto
Copy link
Author

ganto commented Nov 6, 2016

That's the nice thing with free software. You don't always have to do everything yourself 😃

ypid added a commit that referenced this issue Nov 17, 2016
After reading thought https://packaging.python.org and
https://docs.python.org/3/distutils/ again, I decided to continue using
the `find_packages` helper function because this make copy and paste
easier when I decide to reuse this little project as a template for the
next Python project. But ensure that Python packages like tests and docs
are not installed by excluding them.
It is directly used from the example given here: https://packaging.python.org/distributing/#packages

Also add `package_data` to ensure that the Jinja2 templates of the
package are included.

Now only the "yaml4rst" package is installed which can be quickly
verified by `pip3 uninstall yaml4rst`:

```
Uninstalling yaml4rst:
  /usr/local/lib/python3.5/dist-packages/yaml4rst-0.1.1.dist-info/DESCRIPTION.rst
  /usr/local/lib/python3.5/dist-packages/yaml4rst-0.1.1.dist-info/METADATA
  /usr/local/lib/python3.5/dist-packages/yaml4rst-0.1.1.dist-info/RECORD
  /usr/local/lib/python3.5/dist-packages/yaml4rst-0.1.1.dist-info/WHEEL
  /usr/local/lib/python3.5/dist-packages/yaml4rst-0.1.1.dist-info/entry_points.txt
  /usr/local/lib/python3.5/dist-packages/yaml4rst-0.1.1.dist-info/metadata.json
  /usr/local/lib/python3.5/dist-packages/yaml4rst-0.1.1.dist-info/top_level.txt
  /usr/local/lib/python3.5/dist-packages/yaml4rst/__init__.py
  /usr/local/lib/python3.5/dist-packages/yaml4rst/__pycache__/__init__.cpython-35.pyc
  /usr/local/lib/python3.5/dist-packages/yaml4rst/__pycache__/_meta.cpython-35.pyc
  /usr/local/lib/python3.5/dist-packages/yaml4rst/__pycache__/cli.cpython-35.pyc
  /usr/local/lib/python3.5/dist-packages/yaml4rst/__pycache__/defaults.cpython-35.pyc
  /usr/local/lib/python3.5/dist-packages/yaml4rst/__pycache__/helpers.cpython-35.pyc
  /usr/local/lib/python3.5/dist-packages/yaml4rst/__pycache__/reformatter.cpython-35.pyc
  /usr/local/lib/python3.5/dist-packages/yaml4rst/_meta.py
  /usr/local/lib/python3.5/dist-packages/yaml4rst/cli.py
  /usr/local/lib/python3.5/dist-packages/yaml4rst/defaults.py
  /usr/local/lib/python3.5/dist-packages/yaml4rst/helpers.py
  /usr/local/lib/python3.5/dist-packages/yaml4rst/reformatter.py
```

Related to: #1
Thanks to: @ganto
@ypid
Copy link
Owner

ypid commented Nov 17, 2016

@ganto I checked the docs again and fixed the packaging based on our suggestions. Thanks again! Should be fixed in master. Can you retry before I make the release?

@ganto
Copy link
Author

ganto commented Nov 17, 2016 via email

@ganto
Copy link
Author

ganto commented Nov 18, 2016

I tested the new setup.py with Gentoo. The good news is, that the change in the find_packages() makes the installation error disappear (as already experienced before).

However, the template file is still missing 😞 Btw. I also cannot identify it in your pip3 output of the commit message 😏

ypid added a commit that referenced this issue Nov 18, 2016
There are quite a few approaches to try to make setuptools more smart things
like `include_package_data` or writing a small function which returns
the files.
But for once, lets not over-engineer it here and just list the paths
manually.

Fixes the issue of the missing template as verifiable by:

unzip -v dist/yaml4rst-0.1.1-py3-none-any.whl
```
Archive:  dist/yaml4rst-0.1.1-py3-none-any.whl
 Length   Method    Size  Cmpr    Date    Time   CRC-32   Name
--------  ------  ------- ---- ---------- ----- --------  ----
    2467  Defl:N      755  69% 2016-11-05 17:55 405770e3  yaml4rst/helpers.py
     499  Defl:N      314  37% 2016-11-05 17:55 7c0ad5d7  yaml4rst/defaults.py
   34464  Defl:N     7289  79% 2016-11-05 18:57 dc492b1c  yaml4rst/reformatter.py
     233  Defl:N      150  36% 2016-11-05 17:55 b126375d  yaml4rst/__init__.py
     957  Defl:N      545  43% 2016-11-05 19:04 dab9ef7a  yaml4rst/_meta.py
    4758  Defl:N     1601  66% 2016-11-18 00:03 bf1b38d7  yaml4rst/cli.py
     266  Defl:N      176  34% 2016-11-05 17:55 9f7fe2b5  yaml4rst/templates/debops/ansible/defaults_header.j2
    4580  Defl:N     1826  60% 2016-11-18 20:53 244ff541  yaml4rst-0.1.1.dist-info/DESCRIPTION.rst
      48  Defl:N       43  10% 2016-11-18 20:53 04447e33  yaml4rst-0.1.1.dist-info/entry_points.txt
    1516  Defl:N      725  52% 2016-11-18 20:53 85946fec  yaml4rst-0.1.1.dist-info/metadata.json
       9  Defl:N       11 -22% 2016-11-18 20:53 5bb9d6b5  yaml4rst-0.1.1.dist-info/top_level.txt
      92  Defl:N       92   0% 2016-11-18 20:53 4311f75b  yaml4rst-0.1.1.dist-info/WHEEL
    6041  Defl:N     2265  63% 2016-11-18 20:53 315b7eb5  yaml4rst-0.1.1.dist-info/METADATA
    1165  Defl:N      707  39% 2016-11-18 20:53 9df2d361  yaml4rst-0.1.1.dist-info/RECORD
--------          -------  ---                            -------
   57095            16499  71%                            14 files
```

Related to: #1
@ypid
Copy link
Owner

ypid commented Nov 18, 2016

Thanks for testing. You are right, I missed that. I was aware that package_data does not recurse into dirs but forgot to test it and did not notice the missing template. Should be fixed now.

@ganto
Copy link
Author

ganto commented Nov 18, 2016

ok, looks good now 👍

@ypid
Copy link
Owner

ypid commented Nov 18, 2016

All right 😉 I made a new release. Feel free to report any further issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants