Already Exists: Dataset apache-beam-testing:python_bq_file_loads_NNN

[damccorm]

It looks like there are too many collisions in the dataset name.

There don't seem to be many changes to these tests or the implementation in several months except for the migration to nosetest to pytest

We seem to be generating a dataset name based upon the second at which the test is run with a relatively large random value appended.

Note that this same setup code is copied in multiple locations such as BQ Write IT.

Imported from Jira BEAM-12766. Original Jira may contain additional context.
Reported by: lcwik.

[damccorm]

It doesn't seem like this has caused a failure in the retained jenkins history, and I'm not able to get a repro even starting 26 concurrent runs (via 50 instances of calling pytest -v apache_beam/io/gcp/bigquery_file_loads_test.py::BigQueryFileLoadsIT --test-pipeline-options='--runner=TestDataflowRunner --project=<my gcp project> --temp_location=gs://<my bucket> --sdk_location=dist/apache-beam-2.43.0.dev0.tar.gz --region=us-central1'). More than 26 concurrent runs caused jobs to fail because my project was out of quota. I tried this twice I also don't see anything wrong with the logic itself. I'm inclined to close this and reopen if it reoccurs.

Note that this same setup code is copied in multiple locations such as BQ Write IT.

It is worth noting that these tests have different prefixes, so they won't actually conflict with each other. We'd just need any fix we make to hit all of them.