just setup fails as Firestore database already exists

[gregoryfoster]

Describe the Bug

During initial setup of an instance, running just setup [infrastructure_slug] [gcp_region] resulted in a fatal error indicating the Firestore database already existed.

ERROR: (gcloud.firestore.databases.create) Resource in projects [infrastructure_slug] is the subject of a conflict: Database already exists. Please use another database_id
error: Recipe `setup` failed on line 76 with exit code 1

Commenting out that line, and the preceding lines that had been executed once already, enabled the setup script to continue without error. I'm not sure where that Justfile is in this project, perhaps it's pulled in from elsewhere? But it's in the generated instance directory under SETUP/Justfile line 76.

gcloud firestore databases create --location=us-west1

Expected Behavior

The best I can figure, maybe there was a change in the preceding command which creates that resource?

gcloud app create --region=us-west1

This one didn't make sense to me.

Reproduction

Following the standard installation procedure.

Environment

Any additional information about your environment.

• OS Version: [e.g. macOS 11.3.1]
• Cookiecutter Version: [e.g. 0.5.0]

[evamaxfield]

I'm not sure where that Justfile is in this project, perhaps it's pulled in from elsewhere? But it's in the generated instance directory under SETUP/Justfile line 76

It lives in cdp-backend here: https://github.com/CouncilDataProject/cdp-backend/blob/main/cdp_backend/infrastructure/Justfile

We used to use Pulumi and tried Terraform to have "versioned infrastructure as code" but due to the fact that both of those are dependent on the upstream Google team adding their resources as modules to those libraries / ecosystems, we always found we had to do really really hacky things to get them to work.

Instead we switched to the Justfile setup we have now so that we can literally ship all of the files we need to with the cdp-backend library which is versioned and already handles distribution of updates + it's directly tied to the backend library.

---

Can I ask how you discovered this issue? Were you trying to setup an entirely new CDP instance? Or were you re-running commands for an old instance you wanted to clean or something?

[gregoryfoster]

Thanks for the pointer to the Justfile and understood on the reasoning.

Can I ask how you discovered this issue? Were you trying to setup an entirely new CDP instance? Or were you re-running commands for an old instance you wanted to clean or something?

Entirely new CDP instance. However, this was my second attempt to set one up. The earlier version had a different GCP project name (due to the unique suffix) and I didn't make it through installation of the CDP backend, so nothing had been setup on Firebase to my knowledge.

FWIW, I previously setup a completely separate Firebase project (no relation to CDP) on the same account, so that's one potentially unique aspect of my configuration.

[evamaxfield]

Having multiple firebase setups in the same account is totally fine. We use a single GCP account for all of the instances that we maintain.

I think this is the second attempt to set on up issue.

If you go to the firebase console is everything actually setup already even though the command failed?

If that is the case, there might be a potential solution to this but sort of annoying one.

Just has a mechanism for "allow command to fail but keep moving on" -- we could make a second just command like just retry-setup which just has all of the same commands as just setup but marks them as "allowed fail" you can do this with:

-some command here (prepending the - to the command)

That might work but honestly there is just a whole host of issues that can happen in deployment sadly :(

[gregoryfoster]

I don't think we have enough to go on here, but it may be helpful to have it documented in the future. Closing for now.