Failing test(s): TestAccFirestoreDatabase_firestoreCmekDatabaseExample (Beta)

[SarahFrench]

Impacted tests

• TestAccFirestoreDatabase_firestoreCmekDatabaseExample
• TestAccFirestoreDatabase_firestoreCmekDatabaseInDatastoreModeExample FIXED

Affected Resource(s)

• google_firestore_database

Failure rates

• 100% since 2024-03-04

Message(s)

=== RUN   TestAccFirestoreDatabase_firestoreCmekDatabaseInDatastoreModeExample
=== PAUSE TestAccFirestoreDatabase_firestoreCmekDatabaseInDatastoreModeExample
=== CONT  TestAccFirestoreDatabase_firestoreCmekDatabaseInDatastoreModeExample
    vcr_utils.go:152: Step 1/2 error: Error running apply: exit status 1
        Error: Error creating Database: googleapi: Error 400: This project is not eligible to create CMEK databases.
          with google_firestore_database.database,
          on terraform_plugin_test.tf line 6, in resource "google_firestore_database" "database":
           6: resource "google_firestore_database" "database" {
--- FAIL: TestAccFirestoreDatabase_firestoreCmekDatabaseInDatastoreModeExample (19.16s)
FAIL

Nightly build test history

• TestAccFirestoreDatabase_firestoreCmekDatabaseExample - Beta
• TestAccFirestoreDatabase_firestoreCmekDatabaseInDatastoreModeExample - Beta FIXED

b/339909748

[melinath]

Not sure what would make us "eligible" - maybe it's an allowlist? Forwarding.

[jinyangtang]

Hi, I have granted beta nightly project ci-test-project-nightly-beta the permission to create CMEK firestore database, the next run should succeed.

[SarahFrench]

I can see that TestAccFirestoreDatabase_firestoreCmekDatabaseInDatastoreModeExample passed last night 🎉

The TestAccFirestoreDatabase_firestoreCmekDatabaseExample test was previously affected by the "This project is not eligible..." error, which is now solved, but now it is failing due to a different problem. When the test attempts to delete the database at the end of the test this happens:

------- Stdout: -------
=== RUN   TestAccFirestoreDatabase_firestoreCmekDatabaseExample
=== PAUSE TestAccFirestoreDatabase_firestoreCmekDatabaseExample
=== CONT  TestAccFirestoreDatabase_firestoreCmekDatabaseExample
    testing_new.go:85: Error running post-test destroy, there may be dangling resources: exit status 1
        Error: Error when reading or editing Database: googleapi: Error 409: There are concurrent database changes, please try again.
--- FAIL: TestAccFirestoreDatabase_firestoreCmekDatabaseExample (40.02s)
FAIL

I'll update the issue description

[SarahFrench]

It may be just a bad luck, or this newly-exposed error might be flakey. We'll see with time

[melinath]

@SarahFrench what would you think about closing this ticket and then we'll open a new one with the new failure if it fails consistently?