Ability to attach debugger to provider

[joshuarowe-salesforce]

Current Terraform Version

Terraform v0.11.8

Use-cases

When developing a terraform provider, sometimes the easiest way to understand the behavior of the provider is to attach a debugger to it. Unfortunately, the Terraform model of executing providers makes this a bit difficult.

Attempted Solutions

I started by replacing my provider with a shell script that invoked "dlv" in a headless mode. I then found I needed to filter output from dlv itself. This is ok, though it would've been simpler if Terraform just looked for lines beginning with a particular prefix on stdout. A little "grep" magic and this works.

While actually debugging the provider, terraform's use of the go-plugin library enables a keepalive function that causes terraform to quit while in the middle of debugging. This can be disabled in the source code by editing mux.go's DefaultConfig() routine and recompiling. This works, but isn't ideal.

Proposal

I propose either a "-debug", "-run-in-debugger", or an environment variable to the Terraform .exe. This flag should ultimately control the parameters passed to mux.Client(...). I also propose shipping a script that executes dlv that can be used in place of an actual provider; "ln -s" the script to the provider name suffices.

The call chain to mux.Client is a bit vexing in its cross-project-ness and the need to maintain compatibility, especially since . I recommend one of these approaches:

1. Add a "EnableKeepalive bool" or "DisableTimeouts" to go-plugin/client.go ClientConfig struct:

• Set it from terraform/plugin/client.go based on a global variable defined in either "debug.go" or some other ".go" file (newly added "debugger.go" or "debugger-settings.go" file?)
• Add a yamux.Config member to the go-plugin/Client struct
  • Initialize it from the yamux.DefaultConfig() and set the keepalive parameters from the
• Add a new NewRPCClient-like method that accepts a yamux.Config, have the existing NewRPCClient method delegate to it.
• Flags add to the ordinary user burden when viewing the help output, but it does make it obvious to developers how to enter this mode. Additionally, the flag could disable the panic-wrapping behavior.

2. Add a "YAMUX_DISABLE_TIMEOUTS" environment variable and have DefaultConfig pay attention to it.

• Not very clean, but effective.

3. Provide a way of invoking a provider directly through "dlv".

• I recommend against this because in theory providers can be written in languages that aren't "go".

4. Add yamux.Config as an object in the ClientConfig struct:

• Pro: This enables transparency of the yamux config values without requiring changes through the go-plugin scheme.
• Con: This exposes the use of yamux as the underlying library.

5. Add the yamux.Config values as fields in the ClientConfig struct, either all or a subset.

• Pro: This enables all the yamux config values.

I recommend approach 4 or 5. I am coding 5, using an extra "ConnectionConfig" struct and field in the ClientConfig object.

References

See related tickets:

#16305

[karoldrazek]

I've tried your approach with script, grep etc. However, it seems that terraform is spawning multiple instances of terraform provider binary. How did you overcome that? @joshuarowe-salesforce

[apparentlymart]

A while back the team responsible for the provider development experience implemented some features related to debugging, including support for "Debugger-based Debugging".

It ended up being a more SDK/framework-heavy feature than a Terraform CLI/Core feature because it requires cooperation with the plugin to be able to start up enough to complete the plugin handshake while still giving the opportunity to add breakpoints, etc.

The Terraform CLI/Core part of it is just the TF_REATTACH_PROVIDERS environment variable that tells Terraform to expect to find a provider already running (under your debugger), rather than starting up its own process.

Thanks for this request, and sorry that we apparently missed it while announcing the introduction of debugger support!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.