Skip to content

Option to rerun all tests in file with last-failed #13390

Not planned
@bsipocz

Description

@bsipocz
Contributor

What's the problem this feature will solve?

I do use --last-failed to rerun flaky tests. This works perfectly fine in formal testing of libraries, however when I use the same machinery for jupyter notebook-based tutorials I need to rerun all previous cells, too as tests are not standalone.

(I use the nbval plugin to collect the cells as tests)

My other option would be to use pytest-rerunfailes, but that plugin reruns the failures straight away, and most of my cases are flaky as they query a remote server. So the approach of --last-failed would be preferred, aka to have a second test tun that only runs the problematic ones.

Describe the solution you'd like

Have an option to be used along with --last-failed that would collect all the tests from the files contained a failed test

Alternative Solutions

Additional context

Activity

Tusenka

Tusenka commented on Apr 27, 2025

@Tusenka
RonnyPfannschmidt

RonnyPfannschmidt commented on Apr 27, 2025

@RonnyPfannschmidt
Member

The specific use case creates the need for test dependencies as the jupyter plugin isn't using subtests

At first glance it doesn't correctly genralize

Test items per design are independent and pytest core shouldn't have to handle plugins that don't respect that

nicoddemus

nicoddemus commented on Jul 1, 2025

@nicoddemus
Member

I understand the need for this feature, but as @RonnyPfannschmidt mentioned, tests should ideally be independent.

Implementing this workaround in pytest core would likely add more code bloat to an area that's already quite complex—especially with the lastfailed and piecewise plugins providing comparable functionality.

A dedicated plugin for rerunning notebook-related test failures might ultimately deliver a better user experience.

I will close this for now, but thanks for the report regardless @bsipocz!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      Participants

      @RonnyPfannschmidt@nicoddemus@bsipocz@Tusenka

      Issue actions

        Option to rerun all tests in file with last-failed · Issue #13390 · pytest-dev/pytest