test weekly (not daily)

[casperdcl]

reduces number of test comments on the commit

[0x2b3bfa0]

I assume that the discussion label has been added on purpose. 😉

If we merge this, it should probably be only as a temporary measure to reduce notifications until we improve acceptance tests to create and use ephemeral repositories. Merging this will delay DVC updates on containers for a whole week (worst case scenario).

[casperdcl]

Could have a webhook/trigger (get DVC releases to instantly trigger this workflow)

[0x2b3bfa0]

That would be awesome, but may require some workarounds for iterative/setup-dvc#9 (comment) / iterative/dvc#5542.

[casperdcl]

That's a common issue - easily solved with a sleep 600 if triggered by a DVC release event.

[0x2b3bfa0]

The use of arbitrary delays to fix timing issues on an otherwise event-driven system is one of my pet peeves. Nevertheless, yes, it should work; builds take ~8 minutes... until they don't.

[0x2b3bfa0]

The purpose of running [acceptance] tests on a daily basis is just to test if CML still works as expected in conjunction with all the external infrastructure that it uses. There wouldn't be any use in running bit-perfect, local, deterministic tests on the main branch every 24 hours, 🙉 but running end to end tests that involve three different external services might be useful.

CML is a no-service that, paradoxically, needs status monitoring. 😉

[0x2b3bfa0]

Anyhow, given the currrent status of testing, I don't care if we throttle them a bit. If you want this to be merged, I'll press the green button at your command. 💣 💥 🍄 😄

[DavidGOrtega]

Anyhow, given the currrent status of testing, I don't care if we throttle them a bit. If you want this to be merged, I'll press the green button at your command. 💣 💥 🍄 😄

Late now is mine MUAHAHAHAHAHAHA