New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metadata and options are now could be set in configuration files #862

Merged
merged 21 commits into from Dec 8, 2016

Conversation

Projects
None yet
4 participants
@idlesign
Member

idlesign commented Nov 26, 2016

Hi,

It is useful to be able to have metadata and other options declared in configuration files (e.g. setup.cfg). That not only allows some automation, but also reduces bolerplate code in some cases.
See also #394.

This chage allows us to have something like this in our config files:

[metadata]
name = some_name
version = attr: some_package.VERSION
description = Some description
long_description = file: README
keywords = one, two
license = BSD 3-Clause License

[metadata.classifiers]
Framework :: Django
Programming Language :: Python :: 3.5

[options]
zip_safe = False
include_package_data = True
packages = find:
scripts = 
  bin/one.py
  bin/two.py

[options.package_data]
* = *.txt, *.rst
hello = *.msg

[options.extras_require]
pdf = ReportLab>=1.2; RXP
rest = docutils>=0.3; pack ==1.1, ==1.3

Currently every metadata and other options are supported (or I'd like to believe that). More examples in test_config.py.
What I'm concerned with though is whether we need to sandbox file: directive which is supported by some options for security considerations (shouldn't it be impossible to read data from outside).

If overall design and implementation is fine I could update the documentation.

Below are supported ways of value definition for various options.

Terms used

  • list-comma: dangling list or comma-separated values string
  • list-semi: dangling list or semicolon-separated values string
  • bool: True is 1, yes, true
  • dict: list-comma where keys from values are separated by =
  • attr: value could be read from module attibute
  • file: value could be read from a file
  • section: values could be read from a dedicated section

Metadata

  • version: string or attr (iterables, callables are supported; others - through str())
  • platforms (alias: platform): list-comma
  • keywords: list-comma
  • provides: list-comma
  • requires: list-comma
  • obsoletes: list-comma
  • classifiers (alias: classifier): file, section, or list-comma
  • license: file or string
  • description (alias: summary): file or string
  • long_description: file or string
  • author_email (alias: author-email): string
  • url (alias: home-page): string

Options

  • zip_safe: bool
  • setup_requires: list-semi
  • install_requires: list-semi
  • extras_require: section
  • entry_points: file, section
  • use_2to3: bool
  • use_2to3_fixers: list-comma
  • use_2to3_exclude_fixers: list-comma
  • convert_2to3_doctests: list-comma
  • scripts: list-comma
  • eager_resources: list-comma
  • dependency_links: list-comma
  • tests_require: list-semi
  • include_package_data: bool
  • packages: list-comma or find:
  • package_dir: dict
  • package_data: section
  • exclude_package_data: section
  • namespace_packages: list-comma
@idlesign

This comment has been minimized.

Member

idlesign commented Nov 26, 2016

Fails on unrelated test_easy_install.py, and test_sandbox.py.

@idlesign

This comment has been minimized.

Member

idlesign commented Dec 1, 2016

@jaraco What I can do for this issue to be reviewed?

@jaraco

This comment has been minimized.

Member

jaraco commented Dec 3, 2016

I'm trying to re-start the Travis build now that the test failures have been fixed in master, but it's not restarting readily. I restarted the appveyor tests, and the Python 2.7 build is now passing, but not the Python 3.

@jaraco

This comment has been minimized.

Member

jaraco commented Dec 3, 2016

The Travis builds are running now and are showing the same - test failures on Python 3.

@jaraco

This comment has been minimized.

Member

jaraco commented Dec 3, 2016

I like this a lot. The implementation is well-factored and well-thought-out.

I'd like @ncoghlan's thoughts - does this feature fit well into the plans for packaging metadata? Does it conflict or align with the goals of PEP 426 or similar PEPs?

A few aspects I'll want to consider before acceptance:

  • Fix test failures on Python 3.
  • Update docs (as you've offered to do), probably following additional concurrence on the implementation. An important question to answer in the docs: What is the precedence when a value is defined both in setup.cfg and setup.py imperatively? I'd expect setup.py values to take precedence.
  • Use from __future__ import absolute_import, unicode_literals and maybe others in the new modules for better future compatibility.
  • Assess this behavior in light of pbr. IIRC, pbr also uses setup.cfg to store declarative metadata. Does this feature interfere with that functionality?
  • Run the integration tests. There's a tests/test_pypi.py which can be run manually. Let's run those tests as well (which may answer the question about pbr).
@idlesign

This comment has been minimized.

Member

idlesign commented Dec 3, 2016

Py3 failures seem to be connected with pbr indeed, since those 'author_email': 'openstack-dev@lists.openstack.org must have come from there.

I'll peep into pbr to deside how to handle those cases. Yet most probably the only option is not to raise Unknown distribution option exception and to skip unknown keys silenlty.

I'd expect setup.py values to take precedence.

Yes, that's how it's currently done.

@idlesign

This comment has been minimized.

Member

idlesign commented Dec 3, 2016

Skimmed through pbr docs. There seems to be another option: we can handle pbr notation of metadata section. And it's easy enough.

Yet I'm not sure whether it'll the right thing to do, since keys used there do not fully correspond with setup() argument names (classifier vs classifiers; author-email vs author_email; etc.) that we are all used to.

So I propose that we skip unknown keys, and handle both forms for those clashing (it seems to be keywords only).

What do you think?

@idlesign

This comment has been minimized.

Member

idlesign commented Dec 4, 2016

Items from #862 (comment) seem to be covered except docs update, which I'll do after @ncoghlan review.

There's one more thing I'm concerned with: it's seems that it'll the the right thing to expose even minimal API, so that automation tools could easily get already parsed and preprocessed data from configs without ado.

@ncoghlan

This comment has been minimized.

Member

ncoghlan commented Dec 4, 2016

@jaraco When we decided to go in the pyproject.toml direction for the build system independent toolchain bootstrapping, that left setup.cfg free for setuptools/d2to1/pbr to use as you wish.

@idlesign

This comment has been minimized.

Member

idlesign commented Dec 4, 2016

Now with PEP-518, pbr and d2to1 in mind, I think it will be wise not only to handle metadata keys introduced by the latter two, but also to keep closer to .toml. It should simplify the task of transition for users.

@idlesign

This comment has been minimized.

Member

idlesign commented Dec 5, 2016

@jaraco Seems to be quite complete. Docs require proof reading, I guess.

@jaraco jaraco merged commit 3b50de3 into pypa:master Dec 8, 2016

2 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

jaraco added a commit that referenced this pull request Dec 8, 2016

@anthrotype

This comment has been minimized.

Contributor

anthrotype commented Dec 8, 2016

Thanks for working on this! 👍

@idlesign idlesign deleted the idlesign:feat/setupcfg_handling branch Dec 10, 2016

@vaab vaab referenced this pull request Feb 15, 2017

Merged

Updated packaging proposition #67

@lauralorenz lauralorenz referenced this pull request Feb 22, 2017

Merged

7 fix versioning #11

@lrntct lrntct referenced this pull request Jan 6, 2018

Closed

Uploading broken #258

@brainwane brainwane referenced this pull request Mar 1, 2018

Open

setup.cfg vs setup.py #29

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment