Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot setup log_format in setup.cfg, because of InterpolationMissingOptionError` #3062

Closed
prokher opened this issue Dec 25, 2017 · 19 comments
Closed
Labels
plugin: logging related to the logging builtin plugin type: bug problem that needs to be addressed

Comments

@prokher
Copy link

prokher commented Dec 25, 2017

I would like to setup log_format in the project-wise configuration file setup.cfg. I do it like this:

[tool:pytest]
log_format = %(filename)s:%(lineno)d %(name)s%(levelname)s%(message)s

Given this, any invocation of some setup.py or simply pip install -r requirements.txt in the directory where setup.cfg resides, gives me a error:

configparser.InterpolationMissingOptionError: Bad value substitution: option 'log_format' in section 'tool:pytest' contains an interpolation key 'filename' which is not a valid option name. Raw value: '%(filename)s:%(lineno)d %(name)s%(levelname)s%(message)s'

As far as I understand, the reason is that %-formatting is used in config files for string interpolation. It is recommended to screen % with another % to avoid interpolation. Well, if I do this then pytest-catchlog stops interpreting this string as format and simply output it as is.

It looks like, pytest-catchlog shall adopt the rules of writing setup.cfg files. If there is another solution or workaround, please let me know.
Bug was originally posted to the pytest-catchlog issue tracker here , but once this plugin was merged into PyTest I have to report the bug is still here.

P.S. Sorry for ignoring all the steps listed in the contributing guidelines, but this bug was reproduced and confirmed successfully in the original bug report.

@pytestbot
Copy link
Contributor

GitMate.io thinks the contributors most likely able to help are @nicoddemus, and @RonnyPfannschmidt.

@nicoddemus
Copy link
Member

Thanks @prokher for moving the bug report to here (you can close the original issue if you would like).

@thisch, from that thread you mention that there's no workaround for it in Python 2.X?

Also seems strange that by using double % then pytest-catchlog outputs the format string verbatim.

@twmr
Copy link
Contributor

twmr commented Dec 28, 2017

According to https://bugs.python.org/issue20754l#msg299292 the distutils "bug", which causes "pip install ..." to fail, will not be fixed in cpython 2.x.

Also seems strange that by using double % then pytest-catchlog outputs the format string verbatim.

We could add support for both single % (used inside pytest.ini) and double % (used inside setup.cfg) in the logging plugin.

@prokher
Copy link
Author

prokher commented Dec 29, 2017

We could add support for both single % (used inside pytest.ini) and double % (used inside setup.cfg) in the logging plugin.

I suppose, this will solve the original issue.

@nicoddemus
Copy link
Member

Thanks. @thisch should we still consider this a bug or is this an improvement?

@twmr
Copy link
Contributor

twmr commented Jan 8, 2018

What would be the difference? If we mark it as a bug we could fix it in 3.3.x, if not we would have to wait till 3.4, right?

@nicoddemus
Copy link
Member

Correct, but we also should consider if existing suites might break after fixing this; this could warrant releasing it into 3.4 even if it is a bug fix.

@RonnyPfannschmidt
Copy link
Member

well, this is mainly another example of why it is bad to naively hook into configfiles of other tools that aren't meant to be used that way - i propose documenting setup.cfg support as intentionally incomplete due to python version differences in how the file is supposed to be parsed

@prokher
Copy link
Author

prokher commented Jan 10, 2018

@nicoddemus

we also should consider if existing suites might break after fixing this

How is that possible?

@RonnyPfannschmidt

well, this is mainly another example of why it is bad to naively hook into configfiles of other tools that aren't meant to be used that way

I must disagree. In our projects we put all the relevant settings into setup.cfg and that is quite convenient. The only exception for now is PyTest. I mean, it looks quite natural to keep all the settings in the setup.cfg.

@RonnyPfannschmidt
Copy link
Member

@prokher feel free to disagree,however different tools disagree on how to parse what configfile and bam there is a massive mess

while it may feel natural, tecnically its strictly wrong and works mostly by accident, this issue simply shows that there are cases where that "happy it works" breaks down for all the right reasons

@prokher
Copy link
Author

prokher commented Jan 10, 2018

@RonnyPfannschmidt It is not necessary wrong. For instance it will not be wrong if PyTest would use configparser to parse setup.cfg file. I agree that the solution will not solve problem for Python 2.x, since there is no option for screening % symbols yet, but, come on, now it does not work in both 2.x and 3.x which is clearly worse.

I do not want to induce (or participate in) philosophical discussions about what is good and what is wrong, but currently many tools use sections of setup.cfg files for their own parameters, including PyTest. I simply think that this feature shall work as intended.

@RonnyPfannschmidt
Copy link
Member

@prokher then py.test would have different config syntaxes with different behaviours for different files - i'm not quite sure how that makes anything better

@RonnyPfannschmidt
Copy link
Member

from my pov it was a mistake to ever allow the parsing of setup.cfg since it uses a different semantics than pytest.ini

@BeyondEvil
Copy link

So is there a workaround for 3.6? I've hit this issue and I'm confused as how to fix it. For the record we use a setup.cfg file where we put all our configs (for pytest and other tools).

@RonnyPfannschmidt
Copy link
Member

it is impossible to fix as of now - we shoudl start deprecating using setup.cfg

@BeyondEvil
Copy link

@RonnyPfannschmidt So what I'm hearing is that in order to fix this, I'll need to move out the pytest-specific stuff from our setup.cfg and put them into pytest.ini instead?

@RonnyPfannschmidt
Copy link
Member

@BeyondEvil correct

scolby33 added a commit to scolby33/OCSPdash that referenced this issue Jul 2, 2018
I wish it could go in setup.cfg, but that's impossible due to the string interpolation (see pytest-dev/pytest#3062 for a similar problem)

I don't like it being in tox.ini because it becomes confusing where things are.
@wimglenn
Copy link
Member

My related issue over on pip pypa/pip#5182

@RonnyPfannschmidt
Copy link
Member

im closing this one as "wontfix"

ronnix added a commit to betagouv/zam that referenced this issue Feb 27, 2019
ronnix added a commit to betagouv/zam that referenced this issue Feb 27, 2019
autorebase bot pushed a commit to betagouv/zam that referenced this issue Feb 27, 2019
gusutabopb added a commit to gusutabopb/aioinflux that referenced this issue Mar 9, 2019
linw1995 added a commit to linw1995/data_extractor that referenced this issue Oct 12, 2019
Don't move pytest settings into setup.cfg

pytest-dev/pytest#3062
linw1995 added a commit to linw1995/data_extractor that referenced this issue Oct 12, 2019
* Add pre-commit

* Fix pre-commit check error

* Replace pytest.ini with setup.cfg

* Remove MANIFEST.in

* Fix .travis.yml error

* Fix .travis.yml error

.travis.yml currently not support merging multi maps

* Fix setup.cfg error

Don't move pytest settings into setup.cfg

pytest-dev/pytest#3062
landonb pushed a commit to landonb/nark that referenced this issue Apr 8, 2020
- Running flake8 via tox, e.g., `tox -e flake8` crashes.
  Setuptools uses configparser to read the config, which
  throws configparser.InterpolationMissingOptionError because
  of the percent (%) symbols in the format string.
  - Ref:
    pytest-dev/pytest#3062
    pypa/pip#5182
landonb pushed a commit to landonb/dob that referenced this issue Apr 8, 2020
- Running flake8 via tox, e.g., `tox -e flake8` crashes.
  Setuptools uses configparser to read the config, which
  throws configparser.InterpolationMissingOptionError because
  of the percent (%) symbols in the format string.
  - Ref:
    pytest-dev/pytest#3062
    pypa/pip#5182
landonb pushed a commit to landonb/dob-viewer that referenced this issue Apr 8, 2020
- Running flake8 via tox, e.g., `tox -e flake8` crashes.
  Setuptools uses configparser to read the config, which
  throws configparser.InterpolationMissingOptionError because
  of the percent (%) symbols in the format string.
  - Ref:
    pytest-dev/pytest#3062
    pypa/pip#5182
landonb pushed a commit to landonb/dob-prompt that referenced this issue Apr 8, 2020
- Running flake8 via tox, e.g., `tox -e flake8` crashes.
  Setuptools uses configparser to read the config, which
  throws configparser.InterpolationMissingOptionError because
  of the percent (%) symbols in the format string.
  - Ref:
    pytest-dev/pytest#3062
    pypa/pip#5182
landonb added a commit to landonb/dob-bright that referenced this issue Apr 8, 2020
- Running flake8 via tox, e.g., `tox -e flake8` crashes.
  Setuptools uses configparser to read the config, which
  throws configparser.InterpolationMissingOptionError because
  of the percent (%) symbols in the format string.
  - Ref:
    pytest-dev/pytest#3062
    pypa/pip#5182
landonb pushed a commit to landonb/config-decorator that referenced this issue Apr 10, 2020
- Running flake8 via tox, e.g., `tox -e flake8` crashes.
  Setuptools uses configparser to read the config, which
  throws configparser.InterpolationMissingOptionError because
  of the percent (%) symbols in the format string.
  - Ref:
    pytest-dev/pytest#3062
    pypa/pip#5182
landonb pushed a commit to landonb/human-friendly_pedantic-timedelta that referenced this issue Apr 16, 2020
- Running flake8 via tox, e.g., `tox -e flake8` crashes.
  Setuptools uses configparser to read the config, which
  throws configparser.InterpolationMissingOptionError because
  of the percent (%) symbols in the format string.
  - Ref:
    pytest-dev/pytest#3062
    pypa/pip#5182
landonb added a commit to landonb/pep440-version-compare-cli that referenced this issue Apr 26, 2020
- Running flake8 via tox, e.g., `tox -e flake8` crashes.
  Setuptools uses configparser to read the config, which
  throws configparser.InterpolationMissingOptionError because
  of the percent (%) symbols in the format string.
  - Ref:
    pytest-dev/pytest#3062
    pypa/pip#5182
landonb added a commit to landonb/easy-as-pypi that referenced this issue Apr 26, 2020
- Running flake8 via tox, e.g., `tox -e flake8` crashes.
  Setuptools uses configparser to read the config, which
  throws configparser.InterpolationMissingOptionError because
  of the percent (%) symbols in the format string.
  - Ref:
    pytest-dev/pytest#3062
    pypa/pip#5182
xflr6 added a commit to glottolog/treedb that referenced this issue Jul 31, 2021
sondrelg added a commit to snok/django-guid that referenced this issue Dec 13, 2021
Turns out Pytest doesn't support log-format specification in setup.cfg declarations. See pytest-dev/pytest#3062.
sondrelg added a commit to snok/django-guid that referenced this issue Dec 13, 2021
Turns out Pytest doesn't support log-format specification in setup.cfg declarations. See pytest-dev/pytest#3062.
sondrelg added a commit to snok/django-guid that referenced this issue Dec 13, 2021
Turns out Pytest doesn't support log-format specification in setup.cfg declarations. See pytest-dev/pytest#3062.
sondrelg added a commit to snok/django-guid that referenced this issue Dec 13, 2021
Turns out Pytest doesn't support log-format specification in setup.cfg declarations. See pytest-dev/pytest#3062.
sondrelg added a commit to snok/django-guid that referenced this issue Dec 13, 2021
Turns out Pytest doesn't support log-format specification in setup.cfg declarations. See pytest-dev/pytest#3062.
sondrelg added a commit to snok/django-guid that referenced this issue Dec 13, 2021
Turns out Pytest doesn't support log-format specification in setup.cfg declarations. See pytest-dev/pytest#3062.
sondrelg added a commit to snok/django-guid that referenced this issue Dec 13, 2021
Turns out Pytest doesn't support log-format specification in setup.cfg declarations. See pytest-dev/pytest#3062.
sondrelg added a commit to snok/django-guid that referenced this issue Dec 13, 2021
Turns out Pytest doesn't support log-format specification in setup.cfg declarations. See pytest-dev/pytest#3062.
aucampia added a commit to aucampia/rdflib that referenced this issue Apr 18, 2022
pytest was using config from `tox.ini` preferentially and ignoring config
from `setup.cfg`, as a side-effect doctests were not running on
code/docstrings in `rdflib/`.

The reason why some pytest config was in `tox.ini` instead of `setup.cfg`
was because of these issues:
- pypa/pip#5182
- pytest-dev/pytest#3062

As a compromise to fix this I have opted for moving all pytest config to
`pyproject.toml`:
- https://docs.pytest.org/en/stable/reference/customize.html#pyproject-toml

This seems sensible as `pyproject.toml` is standarized by PEPs and
eventually most things from `setup.cfg` will end up in there anyway.

Also:
- remove the pytest ignore on `test/translate_algebra` as tests in there
  have been running and passing for some time.
- fixed path to test data in `rdflib/plugins/parsers/nquads.py`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
plugin: logging related to the logging builtin plugin type: bug problem that needs to be addressed
Projects
None yet
Development

No branches or pull requests

7 participants