Unit tests are running slow / CI takes a long time.

[alxmrs]

I observe that our CI takes a long time to complete. It is early in the lifetime of this project for our unit tests to be slow (certainly, the codebase is small enough). Having fast tests and an expedient CI system increases the pace of development & bugfixes.

This bug involves investigating why our unit tests takes a long time to run. This may be related to #53.

[CillianFn]

@alxmrs This is caused by these two tests and the exponential backoff here

I'm thinking of overriding some of the defaults for the tests only, sound good?

Edit: I feel like ValueError shouldn't be triggering retry_filter=retry.retry_on_server_errors_and_timeout_filter

This might be what we're after...

[alxmrs]

Great investigation! It makes sense that retries are causing the delay.

I looked at Apache Beam's filter definition, and I don't understand why a value error would cause retries. I wonder if the underlying GCS File IO modules are wrapping the exceptions. Does your fix from the edit save time in practice?

[CillianFn]

@alxmrs Seems to be back in business 🐢 🐇
https://github.com/google/weather-tools/runs/4860959961?check_suite_focus=true

[alxmrs]

Thanks Cillian! Marking as closed.