Get test coverage back up to 100%

[sydney-runkle]

Test coverage is currently at 96%, ideally we'd be at 100% 😄.

If you're a first time contributor -- this is a great issue to work on. Writing tests is one of the fastest ways to get comfortable with pydantic internals 😄.

[Kludex]

It can help to set a strictness on files: nedbat/coveragepy#691 - There's a script on this issue that can be used for it.

My suggestion is to avoid regressions.

[alexmojaki]

A note from working on this a bit: in order to effectively cover these lines:

pydantic/pydantic/_internal/_typing_extra.py

Lines 170 to 172 in b7d9281

	# if f_back is None, it's the global module namespace and we don't need to include it here
	if frame.f_back is None:
	return None

pydantic/pydantic/_internal/_model_construction.py

Lines 587 to 588 in b7d9281

	if d is None:
	return None

You need to create a model class in the initial stack frame, i.e. in the actual __main__ module. This might be possible while still using pytest and the coverage CLI by invoking a subprocess but it'll probably be messy. The simplest approach might be to run a script separately from the other tests, call the coverage API directly from within the script, and then construct the model.

[vbmendes]

I'm at PyCon US 2024 and am working on increasing the test coverage of the fields module.

[vbmendes]

Now, I will work on the test coverage of the types module.

[AlessandroMiola]

Hi @sydney-runkle, folks,
I am trying to add a tiny case to increase coverage, but I've stopped because I do see¹ a huge difference in coverage of

Name                                              Stmts   Miss Branch BrPart   Cover
------------------------------------------------------------------------------------
pydantic\mypy.py                                    593    587    315      0   0.88%
------------------------------------------------------------------------------------
TOTAL                                              9255    854   4086    189  89.58%

compared to coverage shown at https://smokeshow.helpmanual.io/1z3s4a3o5l1i1b365u0t/ (90.97%).
Can anyone reproduce or explain the issue?

Thank you :)

pydantic version: 2.8.0a1
        pydantic-core version: 2.19.0
          pydantic-core build: profile=release pgo=true
               python version: 3.12.4 (tags/v3.12.4:8e8a4ba, Jun  6 2024, 19:30:16) [MSC v.1940 64 bit (AMD64)]
                     platform: Windows-11-10.0.22631-SP0
             related packages: mypy-1.1.1 pydantic-extra-types-2.8.0 pydantic-settings-2.2.1 typing_extensions-4.10.0
                       commit: c08dacde

EDIT: if I got it right, the coverage-combine's CI step is doing the trick. make test runs the tests unrelated to mypy (all skipped at first), make test-mypy runs the mypy ones (which are indeed slower) and coverage reports are eventually combined. So one would get the proof that the total coverage has increased after the push. Is this somewhat correct? Thank you!

Footnotes

1. both running pdm run coverage report locally and make all (thus trying to reproduce the CI checks) ↩

[sydney-runkle]

if I got it right, the coverage-combine's CI step is doing the trick. make test runs the tests unrelated to mypy (all skipped at first), make test-mypy runs the mypy ones (which are indeed slower) and coverage reports are eventually combined. So one would get the proof that the total coverage has increased after the push. Is this somewhat correct? Thank you!

Yep, you have to run the mypy tests separately. You should be able to do that locally as well.

As a side note, you might want to work on test coverage for tests that aren't mypy oriented, as they can get pretty difficult to write pretty quickly. You're welcome to take on the challenge though, if you'd like!

Feel free to ping me if you have any other questions.

[sydney-runkle]

Relevant opportunity for tests: #9183 (comment)

Idea for improving coverage and CI: #9740 (comment)