Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump gomaasclient & gomaasapi to avoid "net/http: cannot rewind body after connection loss" on terraform plan when TLS is enabled #44

Closed
tnorris opened this issue Feb 1, 2023 · 2 comments · Fixed by #47

Comments

@tnorris
Copy link

tnorris commented Feb 1, 2023

Howdy,

I am trying to use the MAAS terraform provider to spin up 8 machines and 8 instances. After enabling TLS on MAAS, the provider will sometimes throw a net/http error during plan; it looks like this:

module.maas-glue.maas_machine.fasrc_cluster_group["holyoke2c04311"]: Refreshing state... [id=bn7q4w]
module.maas-glue.maas_machine.fasrc_cluster_group["holyoke2c04312"]: Refreshing state... [id=cdhtnc]
module.maas-glue.maas_machine.fasrc_cluster_group["holyoke2a24216"]: Refreshing state... [id=q3rhqq]
module.maas-glue.maas_machine.fasrc_cluster_group["holyoke2c04316"]: Refreshing state... [id=pfqtsp]
module.maas-glue.maas_machine.fasrc_cluster_group["holyoke2c04314"]: Refreshing state... [id=dmtf4s]
module.maas-glue.maas_machine.fasrc_cluster_group["holyoke2c04310"]: Refreshing state... [id=pba8tg]
module.maas-glue.maas_machine.fasrc_cluster_group["holyokea24214"]: Refreshing state... [id=ww7fpr]
╷
│ Error: Get "https://holy-maas-region01.rc.fas.harvard.edu:5443/MAAS/api/2.0/machines/cdhtnc/": net/http: cannot rewind body after connection loss
│
│   with module.maas-glue.maas_machine.fasrc_cluster_group["holyoke2c04312"],
│   on ../modules/maas-glue/main.tf line 79, in resource "maas_machine" "fasrc_cluster_group":
│   79: resource "maas_machine" "fasrc_cluster_group" {
│

Ideally whatever is between Terraform and MaaS isn't hiccuping, but I was able to hack around this with a good ole retry loop (n.b.: I have no idea what I'm doing with golang):

diff --git i/maas/resource_maas_instance.go w/maas/resource_maas_instance.go
index d01b128..5d10a0b 100644
--- i/maas/resource_maas_instance.go
+++ w/maas/resource_maas_instance.go
@@ -3,6 +3,7 @@ package maas
 import (
 	"context"
 	"fmt"
+	"github.com/hashicorp/terraform-plugin-log/tflog"

 	"github.com/hashicorp/terraform-plugin-sdk/v2/diag"
 	"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
@@ -219,9 +220,23 @@ func resourceInstanceRead(ctx context.Context, d *schema.ResourceData, m interfa
 	client := m.(*client.Client)

 	// Get MAAS machine
-	machine, err := client.Machine.Get(d.Id())
-	if err != nil {
-		return diag.FromErr(err)
+	machine := *new(*entity.Machine)
+	err := *new(error)
+
+	failures := 0
+
+	for {
+		machine, err = client.Machine.Get(d.Id())
+		if err != nil {
+			failures++
+			tflog.Error(ctx, "Retrying a machine get")
+			if failures > 5 {
+				return diag.FromErr(err)
+
+			}
+			continue
+		}
+		break
 	}
 	// Set Terraform state
 	ipAddresses := make([]string, len(machine.IPAddresses))

The only other encounter with this issue I could find was on a deleted bug in launchpad

I suspect there's A Better Way to retry, but I went for the quick and dirty.

@tnorris
Copy link
Author

tnorris commented Feb 15, 2023

Or maybe this isn't retries, it's an empty response from MaaS? https://github.com/juju/gomaasapi/pull/98/files

@tnorris
Copy link
Author

tnorris commented Feb 15, 2023

This can be fixed by bumping gomaasapi in gomaasclient, and then by bumping gomaasapi in terraform-provider-maas

	github.com/juju/gomaasapi/v2 v2.0.1 // indirect

@tnorris tnorris changed the title Add retries to avoid "net/http: cannot rewind body after connection loss" on terraform plan Bump gomaasclient & gomaasapi to avoid "net/http: cannot rewind body after connection loss" on terraform plan when TLS is enabled Feb 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant