Join GitHub today
Add integration tests for cloud storage #1246
In order to facilitate onboarding of new storage backends in the future (e.g. Azure Blob Storage), this pull request refactors the existing tests for the local storage backend to also run for cloud storage backends.
The integration tests are disabled by default. For Google Cloud Storage, the integration tests can be enabled by setting the environment variables
This pull request also fixes two edge-cases in the storage implementation that were uncovered by the tests:
Furthermore, the pull request also:
All the changes described above can be found in d16b5ac. The commit depends on 42e40d6 which introduces more parameterization to the test settings such that the storage backend used during the tests can be configured via environment variables.
The pull request also contains a number of additional minor fixes:
As discussed per OK Slack. This seems to be working when using the interoperability storage API. The one test that was failing then is now passing, maybe a flake. It would be nice to make this working using a service account, but I think for now this is fine.
The CI failed due to
The test could be fixed by comparing date-time values with a certain tolerance threshold (e.g. one second) but that's outside of the scope of this PR.