add app engine flex version

[megan07]

Added flexible app version resource.

Fixes hashicorp/terraform-provider-google#5025

This uses the same API as google_app_engine_standard_app_version, but they have different schemas (some fields are flex-only, etc) so we have decided to split them into two different resources.

Release Note Template for Downstream PRs (will be copied)

appengine: added new resource `google_appengine_flexible_app_version"

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 7 files changed, 4795 insertions(+), 7 deletions(-))
Terraform Beta: Diff ( 7 files changed, 4795 insertions(+), 7 deletions(-))
TF Conversion: Diff ( 1 file changed, 1608 insertions(+))

[c2thorn]

This sweeper should sweep Standard/Flexible indiscriminately right? Might be worth renaming to reflect that. Mostly so I don't forget and add a sweeper for standard in the future...

[c2thorn]

Just looked at the sweeper for now

[c2thorn]

These lines are in the sweeper template because the we won't know what the URL will look like until generation. Fortunately we with a handwritten sweeper we can just replace these lines with:

	listUrl := "https://appengine.googleapis.com/v1/apps/" + config.Project + "/services"

[c2thorn]

It is on my TODO list to handle id's in the sweeper generation, so resources like these would be covered. For now, I think it is important to keep the distinction between id & name, especially since both are present in the api.yaml

Suggested change

	log.Printf("[INFO][SWEEPER_LOG] %s resource name was nil", resourceName)
	log.Printf("[INFO][SWEEPER_LOG] %s resource id was nil", resourceName)

[c2thorn]

id is all we need

Suggested change

	name := GetResourceNameFromSelfLink(obj["id"].(string))
	id := obj["id"].(string)

[c2thorn]

Suggested change

	if !strings.HasPrefix(name, "tf-test") {
	if !strings.HasPrefix(id, "tf-test") {

[c2thorn]

See: my explanation on the list URL code.
We can replace these lines with:

deleteUrl := "https://appengine.googleapis.com/v1/apps/" + config.Project + "/services/" + id

[c2thorn]

Suggested change

	log.Printf("[INFO][SWEEPER_LOG] Sent delete request for %s resource: %s", resourceName, d.Get("service"))
	log.Printf("[INFO][SWEEPER_LOG] Sent delete request for %s resource: %s", resourceName, id)

[chrisst]

This is a Big resource (capital B) so I can't promise much more than spot checking some of the fields to see if they make sense. But I have some overall thoughts as well:

It appears to be a true superset of the StandardAppVersion resource and they share the same endpoints. It is probably good to add a comment in the api.yaml indicating the relationship as I imagine that any bug fix on one resource will need to be added to the other. This is effectively "cross linking" the resources for developers, but should also be done in the public docs as well.

The surface area of this resource is huge, but you only have a single test. I would add a couple more testable examples, as this will improve test coverage and also help our users figure out how it should be used. For example handlers appears to be the most important field of the resource and there's no go example of how to use it. Also there should be some hand written update tests otherwise the update method is completely untested. Extra tests will be extra important with large resources like this as reviewers are unlikely to catch the small stuff like fields that should be required or defaulted.

[chrisst]

I think it's worth explaining how this resource is different from the standard app engine resource and cross linking the other resource in the doc will go a long way to helping users find the right one. Linking to https://cloud.google.com/appengine/docs/the-appengine-environments would be a good call.

[chrisst]

isn't this the default already?

[chrisst]

I'm not sure if this is the most helpful "overview" link. This appears to be more about the admin api and less about the resource at large. Would https://cloud.google.com/appengine/docs/flexible be more appropriate?

[chrisst]

I'm curious if both livenessCheck and readinessCheck need to be defaulted from the API. Since they are set to at_least_one_of 1 will always be defined in the config, are you seeing default values coming back from the non-defined field? Is this something where we could just ignore what was coming back from the API for the non-intended block instead of marking them both computed?

Optional/Computed doesn't always work the same way for all fields so I personally try to avoid it.

[megan07]

Yes, there are default values coming back. I will just set the defaults in api.yaml, which might be a better option.

[chrisst]

Are there default betaSettings that come from the api?

[megan07]

Yes, there are, use_deployment_manager, no_appserver_affinity, and has_docker_image that I've seen.

[megan07]

This is confusing - those values are returned in the response, but if I do an update (which is just a POST for a new version), it says those values (no_appserver_affinity and use_deployment_manager) aren't allowed to be set in the flexible environment. Which makes me think we should ignore_read these, however, if there aren't conflicts with other fields, we probably want to write them to state.

[chrisst]

Can it be set by the user at all? If they can't be sent in a Create or Update then I would just remove the field completely. We can always add them back in as fully computed if a user needs this as an output. Growing a resource is much easier than shrinking it.

[chrisst]

A lot of these enums have a 0 value (default value) of SOMETHING_UNSPECIFIED. But the API almost always rejects a request that contains one of these values. AFAIK it's primarily used for internal error handling of uninitialized objects. 99% of the time we should not expose these values to Terraform users.

[chrisst]

TODO?

[megan07]

Currently ExactlyOneOf won't' work for arrays (or fields whose parent's are arrays). We're hoping to eventually make that possible, so I added this in places where I want to update when we have that ability.

[chrisst]

Is this a user supplied value? If yes is there a better description of how it should be used?

[chrisst]

copy/pasted from entrypoint?

[chrisst]

This feels like a lot of ignore read values. Some like deployment make more sense to not care about, but instanceClass seems like something Terraform would want to correct. Adding something to the description to indicate that upstream changes are ignored might be helpful for some of them if there is a reason that's not super intuitive. For Terraform specific behavior like this you can override description here in the terraform.yaml.

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 10 files changed, 4016 insertions(+), 14 deletions(-))
Terraform Beta: Diff ( 10 files changed, 4016 insertions(+), 14 deletions(-))
TF Conversion: Diff ( 1 file changed, 1262 insertions(+))

[megan07]

I ended up removing a few fields - I'm super confused, and hope that's fine, just going to note them here:

• handlers and libraries - after testing with both nodejs and python, I could send it in the request, but it would not be included in the response. the best i could conclude is that it's not an option with the flex environment, as it's included in the api.yaml documentation for the standard version but not for flexible
• error_handlers - i didn't test this one, i can, but it seemed to follow the same pattern as handlers
• threadsafe - i tested with both nodejs and python and in both cases, if i set threadsafe = false, it would fail, but if i didn't set it, it would default to true, it is also not included in the api.yaml documentation for the flexible environment, for any of the runtimes.

I also wrote an update test, but it failed due to 27e7aa7#r382746058 so I took it out and we can decide maybe decide how to handle that?

[khanali21]

@megan07 Thank you very much for picking this up, I was going to start working on it. I will keep watch on this. If you need help let me know.

[chrisst]

Can you elaborate on why the update test was failing? I can't tell from the linked commit.

[chrisst]

Can it be set by the user at all? If they can't be sent in a Create or Update then I would just remove the field completely. We can always add them back in as fully computed if a user needs this as an output. Growing a resource is much easier than shrinking it.

[chrisst]

Does this have to be a zip file? I would prefer adding this as a file or folder so that git can do file by file diffs, especially if this ever needs updating.

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 10 files changed, 4016 insertions(+), 14 deletions(-))
Terraform Beta: Diff ( 10 files changed, 4016 insertions(+), 14 deletions(-))
TF Conversion: Diff ( 1 file changed, 1262 insertions(+))

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 13 files changed, 4086 insertions(+), 14 deletions(-))
Terraform Beta: Diff ( 13 files changed, 4086 insertions(+), 14 deletions(-))
TF Conversion: Diff ( 1 file changed, 1262 insertions(+))

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 13 files changed, 4099 insertions(+), 14 deletions(-))
Terraform Beta: Diff ( 13 files changed, 4099 insertions(+), 14 deletions(-))
TF Conversion: Diff ( 1 file changed, 1262 insertions(+))

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 12 files changed, 4099 insertions(+), 14 deletions(-))
Terraform Beta: Diff ( 12 files changed, 4099 insertions(+), 14 deletions(-))
TF Conversion: Diff ( 1 file changed, 1262 insertions(+))

[chrisst]

All my comments are test related at this point, nothing critical.

[chrisst]

If these are things that can be updated by other tools after create you should override the description here indicating that Terraform won't detect drift.

[chrisst]

It would be nice to have more of these fields being updated if possible. Especially anything where the empty case makes sense. ie: if it makes sense to set a liveness_check.path and then remove it later by passing an empty string it should be set in the initial create and then empty string in a second update.

[chrisst]

nit: I have a slight preference for these to be 2 different example blocks instead of 2 app_versions in a single example. If possible I like to have each example be a single use case so that the example can have a title block like "NodeJs Example with scaling" and a user doesn't have to parse both whole resource blocks to understand the difference between the two. As pointed out by a user aws s3 bucket does a better job of this than many of our resources.

[megan07]

I'm totally fine with that, I copied this from the standard app (again), and it appears both versions are the same. I'm guessing the example of it was just having 2 versions to the same service? But if that doesn't seem like a great thing to show in an example, I'm happy to remove v2.

[chrisst]

The only difference I saw was noop_on_destroy = true vs delete_service_on_destroy = true which doesn't seem like it's worth all of the wasted real estate on the docs page.

[c2thorn]

Sweeper looks good to me

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 12 files changed, 4117 insertions(+), 14 deletions(-))
Terraform Beta: Diff ( 12 files changed, 4117 insertions(+), 14 deletions(-))
TF Conversion: Diff ( 1 file changed, 1262 insertions(+))