Skip to content
This repository has been archived by the owner on Feb 5, 2020. It is now read-only.

Adapt installer to use Terraform 0.10.* #1841

Merged
merged 14 commits into from Sep 7, 2017
Merged

Conversation

alexsomesan
Copy link
Contributor

These changes adapt the installer for the slightly different workflows introduced by Terraform 0.10.x

Notable changes:

  • provider plugins are now retrieved by Terraform itself as standalone binaries
  • custom provider plugins are discovered from $(BUILD_DIR)/terraform.d/plugins (as opposed to the old .terraformrc)
  • the terraform-provider-matchbox that we use for bare-metal installation is fetched from it's github releases and placed next to the installer binary at BUILD time (all other steps assume it to be there).
  • any other provider plugins placed next to the installer binary will be detected and used during Terraform runs - this is how patched providers can now be published.

Possibly the release scripts need to be adapted, I'll leave that for @Quentin-M to decide.

/cc: @sym3tri

@alexsomesan
Copy link
Contributor Author

Close / re-open to trigger tests

@alexsomesan alexsomesan closed this Sep 5, 2017
@alexsomesan alexsomesan reopened this Sep 5, 2017
Copy link
Contributor

@squat squat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! just a few small nits

@@ -175,12 +177,27 @@ func newExecutorFromApplyHandlerInput(input *TerraformApplyHandlerInput) (*terra
}
exPath := filepath.Join(binaryPath, "clusters", clusterName+time.Now().Format("_2006-01-02_15-04-05"))

// Publish custom providers to execution environment
clusterPluginDir := filepath.Join(exPath, "terraform.d/plugins")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

consider making this a var at the top of the file so it is clear it is a canonical value.

Makefile Outdated
TF_CMD = TERRAFORM_CONFIG=$(TF_RC) terraform
TF_CMD = terraform

PROVIDER_MATCHBOX_VER = v0.2.2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wish we didn't need to spec this so many places. We should make an internal not about all the places to bump this so it's easier when the time comes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a bit misleading. The only relevant place for this version is actually here in the Makefile.

The release scripts have been relocated to their own repo and the versions in here are to be removed. I'm waiting for @Quentin-M to give the green light on that.
In the meantime I just updated everywhere for consistency.

Makefile Outdated
TF_CMD = TERRAFORM_CONFIG=$(TF_RC) terraform
TF_CMD = terraform

PROVIDER_MATCHBOX_VER = v0.2.2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/VER/VERSION/ we can be explicit in configuration like this and it is more consistent with our other files.

"${REPOSITORY_ROOT}/examples"
)

export PROVIDER_MATCHBOX_VER=v0.2.2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/VER/VERSION/ here as well

@alexsomesan
Copy link
Contributor Author

@squat I fixed the nits (hopefully). PTAL.

squat
squat previously approved these changes Sep 6, 2017
Copy link
Contributor

@squat squat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@s-urbaniak s-urbaniak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great, thanks a lot for the effort!!! I have just a few nits.

// Wait for its termination.
select {
case <-done:
case <-time.After(30 * time.Second):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not a show-stopper, but raising awareness, that this could be a potential flake. Looking at the code for this test, terraform init downloads provider.local and provider.template for the tested main.tf file, hence we are dependent on the network.

In practice I did see terraform init failures locally due to temporarily unreachable endpoints, so I am predicting we will see it here too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This UI only code for now.
I agree that we need to do smarter things here (like retries) once this get on the critical path of any automation.

@@ -18,13 +18,15 @@ export TECTONIC_RELEASE_TOP_DIR="$TECTONIC_RELEASE_DIR/tectonic"
export INSTALLER_RELEASE_DIR="$TECTONIC_RELEASE_TOP_DIR/tectonic-installer"

export TERRAFORM_BIN_TMP_DIR="$TMP_DIR/terraform-bin"
export TERRAFORM_BIN_VERSION=0.9.6
export TERRAFORM_BIN_VERSION=0.10.3
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

super small nit: terraform is at 0.10.4 already.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bumped.

"${REPOSITORY_ROOT}/examples"
)

export PROVIDER_MATCHBOX_VERSION=v0.2.2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: we are declaring this version twice. here and here. can we have one place for this var?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These release scripts have been moved out into their own repo so this version will soon be removed and is not in use anymore. I've just updated for consistency, but this is dead code already. Just waiting for Quentin to give the go-ahead to remove these.

@@ -30,7 +30,7 @@ BIN_DIR="$ROOT/bin_test"

MATCHBOX_VERSION=v0.6.1
KUBECTL_VERSION=v1.6.4
TERRAFORM_VERSION=0.9.6
TERRAFORM_VERSION=0.10.3
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same argument as above. can we unify this and maintain one place used for the tf version? it is also declared here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This version of the smoke test is in the refactoring pipeline to move to RSpec already.
The refactored version will use different configuration conventions, so I guess it's not worth refactoring now.

s-urbaniak
s-urbaniak previously approved these changes Sep 7, 2017
@alexsomesan alexsomesan dismissed stale reviews from s-urbaniak and squat via 53064a4 September 7, 2017 08:54
Copy link
Contributor

@s-urbaniak s-urbaniak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@alexsomesan alexsomesan merged commit 16babc9 into coreos:master Sep 7, 2017
# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
# yarn lockfile v1


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did someone run yarn in the wrong directory, then check this file in?

"lockfileEntries": {},
"files": [],
"artifacts": {}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto here. It looks like someone ran yarn install in the installer dir instead of frontend/.

@alexsomesan
Copy link
Contributor Author

alexsomesan commented Sep 13, 2017 via email

@rowleyaj rowleyaj mentioned this pull request Oct 12, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants