Add support for native TOML configuration

[bluetech]

Fixes #13743.

Add support for using native TOML configuration, while maintaining full backwards compatibility with the existing INI-based configuration system.

In pyproject.toml, the native configuration is under [pytest.tool]. Also add support for pytest.toml/.pytest.toml files, under [pytest] table (similar to pytest.ini).

--override-ini always uses "ini" mode for compatibility.

[nicoddemus]

Gave it a first pass, this is looking great! The ability of using [tool.pytest] or a pytest.toml file is definitely appealing from the user's perspective.

However, it seems the only difference between ini_options and the native TOML support is the type checking versus string coercion:

# pyproject.toml
[tool.pytest]
test_int_validation = 5
test_paths = ["src", "lib"]

#[tool.pytest.ini_options]
#test_int_validation = "5"
#test_paths = ["src", "lib"]

# conftest
def pytest_addoption(parser):
    parser.addini("test_int_validation", "Test int validation", type="int", default=0)
    parser.addini("test_paths", "Test paths config", type="paths")

# test_foo.py
def test(request):
    val = request.config.getini("test_int_validation")
    print(f"{val} ({type(val)})")
    val2 = request.config.getini("test_paths")
    print(f"{val2} ({type(val2)})")

From plugin author's POV, both [tool.pytest] and [tool.pytest.ini_options] yield exactly the same configuration values (as they should):

5 (<class 'int'>)
[WindowsPath('e:/projects/pytest/.tmp/toml-config/src'), WindowsPath('e:/projects/pytest/.tmp/toml-config/lib')] (<class 'list'>)

The type-coercion for ini options means that test_int_validation in ini_options can be either "5" or 5.

Unless I'm missing something, [tool.pytest.ini_options] and [tool.pytest] are identical, except the latter does type validation instead of coercion.

To be clear, this is all good and definitely we want to introduce this in pytest.

I'm just wondering why we just not did this in the first place, rather than introducing pytest.ini_options. I was under the impression that @RonnyPfannschmidt had other plans regarding TOML configuration -- perhaps letting plugins define their options in their own tables or something like that. That said, I believe this implementation is more grounded and probably what we should have done from the start, so thanks a lot @bluetech for tackling this.

[RonnyPfannschmidt]

I hope to have cot.config.ingest a reasonable wip by the end of the year

[bluetech]

@nicoddemus My impression was that we wanted to get some toml support and so went with the "quick" ini_options support. But it was a long time ago, I'm probably misremembering.

In any case, we can wait for @RonnyPfannschmidt solution. But I wanted to finish the PR while it's still fresh in my head, maybe Ronny could reuse some of it. The changes since the initial posting:

• Added pytest.toml/.pytest.toml support
• Added a changelog
• Went over the docs
• Tested on a project of mine which uses a jungle of pyproject.toml and pytest.ini and converted it to native toml.

[nicoddemus]

In any case, we can wait for @RonnyPfannschmidt solution.

I think we should go ahead with your support, I believe @RonnyPfannschmidt's solution can be built on top of it later (please correct me if I'm wrong Ronn).

It would be nice to release this new support in 9.0. 👍

[RonnyPfannschmidt]

My solution would completely replace ini/toml/argpatse settings

Its inspired by typedsetting

[RonnyPfannschmidt]

But i believe its fine to go Ahead

[nicoddemus]

My solution would completely replace ini/toml/argpatse settings

Sounds interesting, but I wonder how it would work with backward compatibility in mind.

[RonnyPfannschmidt]

@nicoddemus i'm exploring that part still - but the basic goal is to provide a backward compatible layer - once tests pass pytest-alikeness i can propose upstream changes

[bluetech]

Rebased, still missing some coverage.

I also noticed about this code here:

pytest/src/_pytest/config/findpaths.py

Lines 70 to 72 in 0fb7cae

	# "pytest.ini" files are always the source of configuration, even if empty.
	if filepath.name == "pytest.ini":
	return {}

It allows an empty pytest.ini to still be considered the config file. I think it also should check for .pytest.ini. I think this point was forgotten when .pytest.ini support was added (#9988), not intentionally omitted.

[bluetech]

OK, this should be ready now.

[nicoddemus]

LGTM!