Higher test latency than pytest-watch

[brylie-wolt]

We are migrating from pytest-watch to pytest-watcher and noticed considerable latency when making code changes.

I tried to debug the pytest-watcher code. In fact, both of them use the same library to track files change events - watchdog. And it seems that pytest-watcher catches such events almost immediately, but most of the time it waits for `subprocess.run' (to run tests)

@RB387

How could the test latency be reduced?

[brylie-wolt]

@andreoliwa, could you provide any additional context to help troubleshoot this latency?

[olzhasar]

There is a delay option that you can use to control latency. Default value is 0.5 seconds.

You can use it like this:

ptw --delay 0.1 .

[olzhasar]

@brylie-wolt Does the provided information resolve your issue?

Btw, the reason why this delay exists in the first place, is because oftentimes one can have linters / code formatters configured that run after the actual file save happens

[brylie-wolt]

I'll look into the advice you provided to see if it can resolve the issue, as it's not something I've encountered firsthand. I opened this issue when investigating the slowness reported by a colleague.

Your explanation of the delay was helpful and informative. Just so you know, we're using pre-commit for linters and other tools.

[olzhasar]

@brylie-wolt
The latest version now uses a delay of 0.2 by default, and the loop delay (the interval between trigger monitoring events) has also been reduced to 0.1.
With that in mind, it should now be pretty fast by default without the need to provide additional arguments.

FYI

[andreoliwa]

could you provide any additional context to help troubleshoot this latency?

Sorry about not replying earlier.

I will try the latest version.

[andreoliwa]

@olzhasar the latest version works better with the new shorter default for the delay.

The latency problem I had before is due to a different behaviour in pytest-watcher.
I was expecting it to run immediately on poetry run ptw, like joeyespo/pytest-watch does.

I realise now that it's an intentional behaviour: #1
I am used to pytest-watch's behaviour, but I can get used to this one.

As described in the issue, I need to change something in the code to trigger tests.
E.g. add a space and save the file.
Because PyCharm doesn't save a file when I press Cmd+S if nothing has changed.

[olzhasar]

@andreoliwa @brylie-wolt Do you guys think that auto-run is a better default behavior? I am personally used to the current way of things, but I don't have a strong preference here

[brylie-wolt]

I don't have a strong opinion, but default behavior shouldn't surprise people. So, consider user expectations when adding default behavior. Developers expect tests to run without delay since delays slow the development process and increase CI costs.

[olzhasar]

@brylie-wolt I'm talking about running tests on startup, not the delay.
As was mentioned earlier, short delay is needed to properly handle multiple save events.

[andreoliwa]

Do you guys think that auto-run is a better default behavior?

Not really, I also don't have a strong opinion here.

It might be worthy to add something to the README, to manage expectations.
Things like:

• auto-run behaviour is different from pytest-watch's behaviour;
• this is not a fork but a rewrite (right?).

When I first saw the project, I thought it was a drop-in replacement for pytest-watch.

[olzhasar]

@andreoliwa good points, thank you!

[olzhasar]

@andreoliwa Updated the README file accordingly.
Let me know if you have any other suggestions regarding README

[andreoliwa]

Thank you for the prompt responses. 🙏🏻

The README looks great.

I also like the new verbose logging from v0.3.2:

pytest-watcher version 0.3.2
Runner command: pytest --testmon
Waiting for file changes in /Users/(...)
[ptw] /Users/(...).py modified

I'll switch my repos to use pytest-watcher and you got my star. 🌟 😄

[olzhasar]

@andreoliwa Thank you! Appreciate your feedback!