[bugfix] Fix multiple test retries for tests with fixtures

[vkarak]

Closes #3595.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.25%. Comparing base (e47ed38) to head (7a9d316).
⚠️ Report is 2 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3612      +/-   ##
==========================================
- Coverage   91.37%   91.25%   -0.13%     
==========================================
  Files          62       62              
  Lines       13583    13584       +1     
==========================================
- Hits        12412    12396      -16     
- Misses       1171     1188      +17     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ekouts]

LGTM!! I also tested it and it seems fine. Thanks @vkarak !!