gRPC client library similar to hashicorp/terraform-plugin-go

[magodo]

Terraform Version

Terraform v1.3.6
on linux_amd64

Use Cases

I have the same requirement as the author of hashicorp/terraform-plugin-go#64 (which is closed and points me here) that:

I want to create a separate binary which will be able to use Terraform plugins. This binary will do the import of the existing resource to Terraform state

Currently, the only way to do this is to use terraform-exec, which under the hood calls terraform binary. My use case will require me to import multiple resources at the same time. Since bulk import (#22219) is still not available yet, I just launch multiple terraform instances, with multiple provider instances. It makes the CPU throttling quickly.

So I'd like to have a GRPC client binding similar as the server binding that is done in the https://github.com/hashicorp/terraform-plugin-go. I assume the code to be split out from terraform resides in the internal/plugin(6).

Attempted Solutions

I've tried to create a repo https://github.com/magodo/terraform-client-go to do the basic client implementation, which still requires to duplicate the pb and the toproto/fromproto implementation from https://github.com/hashicorp/terraform-plugin-go. Whilst, this is still far from enough to be usable.

Proposal

Create a new repo or extend https://github.com/hashicorp/terraform-plugin-go to include the gRPC client implementations for protocol 5 and 6. Ideally, also provide an interface to be implemented by both protocol's client just as internal/providers/Interface does.

References

• Programmatic interface to Terraform operations #22595
• Implement Terraform plugin client terraform-plugin-go#64

[apparentlymart]

Hi @magodo,

For one of my side projects I used the available documentation and protobuf schema to build a separate client library myself. That's not an official HashiCorp project and is limited only to the parts of the protocol I needed for that particular project, but it might be a useful starting point for integrating your own software with Terraform plugins. I did that just the same way you did: I copied the protobuf schema. Generating RPC stubs from that schema is the expected way to make a client for a gRPC service, and so I imagine any such implementation would do that same thing.

I'm not sure if there will be an official form of this because maintaining that would of course have an opportunity cost, taking time from other work. But a third-party implementation like mine can make use of the fact that plugins tend to be backward compatible with older versions of Terraform to feel reasonably confident that they'll only need to update a custom client relatively infrequently, particularly if only using a small part of the protocol for some specific purpose.

However, we can use this issue to represent the request and see if there's sufficient demand for it to justify prioritizing it over other work. Thanks!