Terraform Plugin SDK v2 Upgrade #289

adamcstephens · 2023-04-06T01:25:34Z

I ran the upgrade migration tool, and then addressed all visible errors.

https://developer.hashicorp.com/terraform/plugin/sdkv2/guides/v2-upgrade-guide
https://github.com/hashicorp/tf-sdk-migrator#tf-sdk-migrator-v2upgrade-migrate-from-sdkv1-to-sdkv2

This is pretty similar to #243, without some of the other changes. Also marking #285 closed with this, since it takes x/net to 0.8.0

Closes #243
Closes #285

adamcstephens · 2023-04-06T14:14:20Z

Thanks for approving the tests. I'll see if I can diagnose what's going wrong, but they run successfully locally. :/

adamcstephens · 2023-04-06T14:31:14Z

Indirect dependencies needed to be updated: hashicorp/terraform-plugin-sdk#821

jtopjian · 2023-04-06T15:31:18Z

@adamcstephens Thanks for working on this.

If you'd like to open a small PR that only includes an update to the README or some other documentation, I can merge that which will allow your other PRs to run without approval.

jtopjian · 2023-04-10T16:07:17Z

@adamcstephens I think you can now amend the PRs and the tests will run automatically. If that doesn't work, try opening a new PR.

Let me know if you have any questions or need anything investigated. It might take me a day or two, but I'll get to it 🙂

adamcstephens · 2023-04-10T20:03:16Z

Hi @jtopjian, thanks for the ping. I've actually been working on this, but running into some issues with the tests. I also am familiarizing myself with the codebase, along with the terraform provider docs, given that I've never worked on one before. I did learn over the weekend that even sdk v2 is no longer even recommended and they are pushing people towards the Terraform Plugin Framework now. Either way, getting to v2 should allow us to upgrade the LXD client library to take advantage of newer capabilities, and prepare for a possible migration to framework in the future.

One of the things I'm struggling with in the tests is our provider testing. Specifically, we define a provider block in our configurations, such as

terraform-provider-lxd/lxd/provider_test.go

Lines 237 to 238 in 0c6b4eb

    
           provider "lxd" { 
        
           }

This causes a duplicate provider conflict as the acceptance testing framework appears to bring its own provider initialization. I'm still exploring solutions, but I'm wondering if these tests are actually valuable? I also wonder if we should even support the multi-remote functionality at all, given that Terraform has native support for multiple provider blocks.

jtopjian · 2023-04-12T05:49:56Z

This causes a duplicate provider conflict as the acceptance testing framework appears to bring its own provider initialization. I'm still exploring solutions, but I'm wondering if these tests are actually valuable?

They were valuable at one time. If you want to try removing these tests to see if the other tests pass, it's worth a try. But there's a lot of logic in the provider configuration for connecting to an LXD remote, so I would caution about modifying anything related to that.

I also wonder if we should even support the multi-remote functionality at all, given that Terraform has native support for multiple provider blocks.

This support was added before LXD supported clustering and before Terraform allowed multiple provider declarations of the same provider, so it's a bit of a historical feature. I would say that removing this functionality would require a major version increment, which is possible to do, but my advice would be to try to upgrade all of the internal libraries first and then start making front-facing architecture changes like this.

adamcstephens · 2023-04-12T14:22:22Z

I would say that removing this functionality would require a major version increment, which is possible to do, but my advice would be to try to upgrade all of the internal libraries first and then start making front-facing architecture changes like this.

Yes, I 100% agree with this. I wouldn't want to make a change to this functionality without a clear deprecation and a major version change. For now I'm fully focused on getting the tests to pass and getting the dependencies up to date.

Most of the test issues are small, and some are actually fixed in #243 in what I thought were unrelated changes.

The one I've been unable to track down yet is a failure in the publish image tests, which is causing a panic.

=== RUN   TestAccPublishImage_aliases                                                                                                                         
panic: Invalid address to set: []string{"architecture"}

https://github.com/adamcstephens/terraform-provider-lxd/blob/v2upgrade/lxd/resource_lxd_publish_image.go#L242-L244

jtopjian · 2023-04-12T15:11:14Z

The one I've been unable to track down yet is a failure in the publish image tests, which is causing a panic.

I'm quite certain that is because fingerprint, architecture, and created_at aren't defined in that resource's schema at all. Terraform must have been silently disregarding that in the past.

adamcstephens · 2023-04-14T12:36:34Z

All acceptance tests are now passing. Let me know if you have any feedback, want me to squash commits, etc.

jtopjian

@adamcstephens Thanks so much for your work on this. In general, this all looks really good and definitely a long-needed update.

I do have a few comments. Please let me know if you have any questions or would like to discuss them.

lxd/provider.go

jtopjian · 2023-04-16T19:05:11Z

lxd/resource_lxd_container.go

@@ -160,6 +160,7 @@ func resourceLxdContainer() *schema.Resource {
 						"mode": {
 							Type:     schema.TypeString,
 							Optional: true,
+							Default:  "0755",


Is this needed for the SDK update? If not, let's remove it. Plus, 0755 could be insecure for a default mode.

Changes to the file mode were required as the related tests were throwing a type panic.

panic: mode: '' expected type 'string', got unconvertible type 'int', value: '0'

0755 is already the default, but it's buried in containerUploadFile()

terraform-provider-lxd/lxd/shared.go

Line 232 in 0c6b4eb

mode := os.FileMode(0755)

I could probably fix the panic without moving the default if you'd prefer, but it seemed better practice to have the default be in the schema and not the code.

Furthermore, this file mode is used for any directory creations as well, so 0755 could be an appropriate default for directories.

I could probably fix the panic without moving the default if you'd prefer, but it seemed better practice to have the default be in the schema and not the code.

Yeah, that makes sense.

Furthermore, this file mode is used for any directory creations as well, so 0755 could be an appropriate default for directories.

I'm pretty sure this was reviewed in the past because I would have questioned 0755 whenever it comes up. It's okay for now, but I do recommend looking at 0750 in the future. The issue isn't so much about directories, but files where anyone can see the contents of potentially sensitive files that are copied to the container. The end responsibility is on the user, though - they can always change the mode from the default.

I'd be happy to revisit this later. We should probably split the two modes and have two attributes, one for the file and one for any directories that need to be created. Then we can have a more restrictive default for the file itself.

lxd/resource_lxd_container.go

lxd/resource_lxd_container_file.go

lxd/resource_lxd_publish_image.go

https://developer.hashicorp.com/terraform/plugin/sdkv2/guides/v2-upgrade-guide#removal-of-helper-schema-schema-removed

https://developer.hashicorp.com/terraform/plugin/sdkv2/guides/v2-upgrade-guide#removal-of-helper-schema-resourcedata-setpartial

https://developer.hashicorp.com/terraform/plugin/sdkv2/guides/v2-upgrade-guide#removal-of-the-terraform-resourceprovider-interface

go: warning: cloud.google.com/go/storage@v1.27.0: retracted by module author: due to googleapis/google-cloud-go#6857 go: to switch to the latest unretracted version, run: go get cloud.google.com/go/storage@latest

open bug about require sections: golang/go#56471

the lxd client api works in ints, but mode is a string for our users

jtopjian

This looks good to me. Really nice work - thank you very much!

jtopjian · 2023-04-16T20:50:17Z

@adamcstephens Thank you again. Based on this work, I think you've done a great job. If you're interested in acting as a maintainer, I can set you up with the appropriate privileges in this repository. You can reach out to ask for help, history, advice, etc but also be able to take an active role here. Let me know.

adamcstephens · 2023-04-16T23:09:46Z

@jtopjian I am definitely still interested in helping as a maintainer.

run tf-sdk-migrator v2upgrade

aaf6fbf

adamcstephens force-pushed the v2upgrade branch from 1823f10 to cd09eea Compare April 6, 2023 01:31

adamcstephens mentioned this pull request Apr 6, 2023

Upgrade lxd client library: 20191225 -> 20230415 #290

Merged

adamcstephens force-pushed the v2upgrade branch from dc33e29 to c17d3d7 Compare April 14, 2023 03:46

jtopjian requested changes Apr 16, 2023

View reviewed changes

adamcstephens force-pushed the v2upgrade branch from c17d3d7 to 6c84512 Compare April 16, 2023 20:06

This comment was marked as duplicate.

Sign in to view

adamcstephens added 9 commits April 16, 2023 16:21

schema.Removed removed in v2, change to deprecated

7e3a60b

https://developer.hashicorp.com/terraform/plugin/sdkv2/guides/v2-upgrade-guide#removal-of-helper-schema-schema-removed

SetPartial was removed in v2

000afff

https://developer.hashicorp.com/terraform/plugin/sdkv2/guides/v2-upgrade-guide#removal-of-helper-schema-resourcedata-setpartial

ResourceProvider removed in v2

3a29d57

https://developer.hashicorp.com/terraform/plugin/sdkv2/guides/v2-upgrade-guide#removal-of-the-terraform-resourceprovider-interface

go/storage@v1.27.0 was retracted

f514267

go: warning: cloud.google.com/go/storage@v1.27.0: retracted by module author: due to googleapis/google-cloud-go#6857 go: to switch to the latest unretracted version, run: go get cloud.google.com/go/storage@latest

update plugin/v2 deps and manually merge require sections

f14ed8b

open bug about require sections: golang/go#56471

publish image needs fields for computed resources

20e9951

set default for file mode. ensure proper string-int conversion

cb49fa7

the lxd client api works in ints, but mode is a string for our users

use set-specific attribute test function

c3521d7

simplify provider tests

ef67b01

adamcstephens force-pushed the v2upgrade branch from 6c84512 to ef67b01 Compare April 16, 2023 20:22

jtopjian approved these changes Apr 16, 2023

View reviewed changes

jtopjian merged commit c28d1e1 into terraform-lxd:master Apr 16, 2023
2 checks passed

adamcstephens deleted the v2upgrade branch April 16, 2023 23:07

adamcstephens added the chore label Jun 8, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Terraform Plugin SDK v2 Upgrade #289

Terraform Plugin SDK v2 Upgrade #289

adamcstephens commented Apr 6, 2023 •

edited

adamcstephens commented Apr 6, 2023

adamcstephens commented Apr 6, 2023

jtopjian commented Apr 6, 2023

jtopjian commented Apr 10, 2023

adamcstephens commented Apr 10, 2023 •

edited

jtopjian commented Apr 12, 2023

adamcstephens commented Apr 12, 2023

jtopjian commented Apr 12, 2023

adamcstephens commented Apr 14, 2023

jtopjian left a comment

jtopjian Apr 16, 2023

adamcstephens Apr 16, 2023

jtopjian Apr 16, 2023

adamcstephens Apr 16, 2023

This comment was marked as duplicate.

jtopjian left a comment

jtopjian commented Apr 16, 2023

adamcstephens commented Apr 16, 2023

Terraform Plugin SDK v2 Upgrade #289

Terraform Plugin SDK v2 Upgrade #289

Conversation

adamcstephens commented Apr 6, 2023 • edited

adamcstephens commented Apr 6, 2023

adamcstephens commented Apr 6, 2023

jtopjian commented Apr 6, 2023

jtopjian commented Apr 10, 2023

adamcstephens commented Apr 10, 2023 • edited

jtopjian commented Apr 12, 2023

adamcstephens commented Apr 12, 2023

jtopjian commented Apr 12, 2023

adamcstephens commented Apr 14, 2023

jtopjian left a comment

Choose a reason for hiding this comment

jtopjian Apr 16, 2023

Choose a reason for hiding this comment

adamcstephens Apr 16, 2023

Choose a reason for hiding this comment

jtopjian Apr 16, 2023

Choose a reason for hiding this comment

adamcstephens Apr 16, 2023

Choose a reason for hiding this comment

This comment was marked as duplicate.

jtopjian left a comment

Choose a reason for hiding this comment

jtopjian commented Apr 16, 2023

adamcstephens commented Apr 16, 2023

adamcstephens commented Apr 6, 2023 •

edited

adamcstephens commented Apr 10, 2023 •

edited