Add end to end test for SCM and file decorations #2013
Conversation
| expect(expectedScmItemLabels.sort()).toStrictEqual( | ||
| dvcTreeItemLabels.sort() | ||
| ) | ||
| }).timeout(60000) |
There was a problem hiding this comment.
Isn't this timeout too long?
There was a problem hiding this comment.
The SCM state relies on dvc status. Whenever an experiment is running dvc status will fail. There is an exponential back off in place for retries which means that after the experiment finishes there can be a significant lag before the SCM view is updated with the correct details. Locally this is less of a problem but in the CI running an experiment takes longer. Leading to a longer lag in the subsequent update. I'd rather that this test is reliable with a longer timeout than flaky with a shorter one.
Does that make sense? WDYT?
There was a problem hiding this comment.
It does for me 👍
I know this is a can of worms, but is there a way to only dvc status after all the runs? Like using a subscription or something like that?
There was a problem hiding this comment.
Q: can it be done as a global setting - one minute timeout per test?
There was a problem hiding this comment.
probably it applies to all tests that run DVC internally?
There was a problem hiding this comment.
I know this is a can of worms, but is there a way to only
dvc statusafter all the runs? Like using a subscription or something like that?
That would mean we would have to introduce stubs into this test suite, making them less e2e. I would rather test the extension as is. We should be able to improve the situation once we get data:status.
Q: can it be done as a global setting - one minute timeout per test?
This was actually already set to 1m
|
Code Climate has analyzed commit dbbba1e and detected 0 issues on this pull request. The test coverage on the diff in this pull request is 100.0% (85% is the threshold). This pull request will bring the total coverage in the repository to 96.6% (0.0% change). View more on Code Climate. |
3 participants
2/2
main<- #2002 <- thisThis PR adds an end to end test for the SCM view and our custom file decorations.