Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Openshift 4.1 UPI installation failed on VMware (terraform version - v0.11.14) #23640

Closed
arunabhabanerjee opened this issue Dec 11, 2019 · 3 comments
Labels
provider/ignition v0.11 Issues (primarily bugs) reported against v0.11 releases

Comments

@arunabhabanerjee
Copy link

Terraform Version

Terraform v0.11.14

  • provider.external v1.2.0
  • provider.ignition v1.2.0
  • provider.null v2.1.2
  • provider.template v2.1.2
  • provider.vsphere v1.13.0

-->

...

Terraform Configuration Files

``// ID identifying the cluster to create. Use your username so that resources created can be tracked back to you.
cluster_id = "test"

// Domain of the cluster. This should be "${cluster_id}.${base_domain}".
cluster_domain = "test.lab.local"

// Base domain from which the cluster domain is a subdomain.
base_domain = "lab.loacal"

// Name of the vSphere server. The dev cluster is on "vcsa.vmware.devcluster.openshift.com".
vsphere_server = "10.55.138.36"

// User on the vSphere server.
vsphere_user = "administrator@vsphere.local"

// Password of the user on the vSphere server.
vsphere_password = "password"

// Name of the vSphere cluster. The dev cluster is "devel".
vsphere_cluster = "dev"

// Name of the vSphere data center. The dev cluster is "dc1".
vsphere_datacenter = "BTM"

// Name of the vSphere data store to use for the VMs. The dev cluster uses "nvme-ds1".
vsphere_datastore = "datastore2"

// Name of the VM template to clone to create VMs for the cluster. The dev cluster has a template named "rhcos-latest".
vm_template = "rhcos-latest"

// The machine_cidr where IP addresses will be assigned for cluster nodes.
// Additionally, IPAM will assign IPs based on the network ID.
machine_cidr = "10.55.24.0/21"

// The number of control plane VMs to create. Default is 3.
control_plane_count = 3

// The number of compute VMs to create. Default is 3.
compute_count = 3

// URL of the bootstrap ignition. This needs to be publicly accessible so that the bootstrap machine can pull the ignition.
bootstrap_ignition_url = "http://10.55.1.5:8080/ignition/bootstrap.ign"

// Ignition config for the control plane machines. You should copy the contents of the master.ign generated by the installer.
control_plane_ignition = <<END_OF_MASTER_IGNITION
{"ignition":{"config":{"append":[{"source":"https://api-int.test.lab.local:22623/config/master","verification":{}}]},"security":{"tls":{"certificateAuthorities":[{"source":"data:text/plain;charset=utf-8;base64,LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURFRENDQWZpZ0F3SUJBZ0lJUU5BQXRDbmhUVkl3RFFZSktvWklodmNOQVFFTEJRQXdKakVTTUJBR0ExVUUKQ3hNSmIzQmxibk5vYVdaME1SQXdEZ1lEVlFRREV3ZHliMjkwTFdOaE1CNFhEVEU1TVRJd016RTBNVFF6T1ZvWApEVEk1TVRFek1ERTBNVFF6T1Zvd0pqRVNNQkFHQTFVRUN4TUpiM0JsYm5Ob2FXWjBNUkF3RGdZRFZRUURFd2R5CmIyOTBMV05oTUlJQklqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FROEFNSUlCQ2dLQ0FRRUF1Q3FDSy8xYUtOOTAKQUFTamxQTTZxaFY2SHlPWGtKaFZKYm1YQzErbUthMG9CbnNiUGN4WE14NkFROXhSMSt0djJveFl1Q0lBRlNNUwplb3pIa2cyS2VhU25mQTJYdHVkRGZnaklTNUdsQ3lPRTJ4ZXZrZ1dzU0VVQXV3TmlTTXRRWS9BTWt1YnMyK3VZCmRVQUxUMHhsZWkyUWhoaFN6OXN3allNNitnaUh5K2JZMDZjZmxkbjF0NG5aRGZFeUlENFNaR25WeDBmVzNNZ3QKdVRPU200Q1E0WWpUZDlQd1pKL0RUckpkNG9sVjZHQlNoSXFBSHh0elBiaWZwMjZaTnY5MjRrN2trMFhKUDI5Lwp3MEErWDRPVG1tdnY0RVQ2SVJISDZlMFp6TW9BUkpReVF1ekJMUFhqZ0VGMFBtRDAvS2ZBSFlEc25Rdklza3JICldJa3A3enc5a3dJREFRQUJvMEl3UURBT0JnTlZIUThCQWY4RUJBTUNBcVF3RHdZRFZSMFRBUUgvQkFVd0F3RUIKL3pBZEJnTlZIUTRFRmdRVTI4TEhRTUlMZkNhREVBUGhhcWg0MmpVZjZvUXdEUVlKS29aSWh2Y05BUUVMQlFBRApnZ0VCQUFuQ2dCME9saTl4cmszZkVUK2hCOE9hQWtuRncra2NGVHdGMDNaYS9HeDkydW8wcjJZNlI2bDB2Q1NuCml2VHhoRzBMeDhaTlNYMURlSkFRbDY2YjkrTHF2QVhqdDNqUFpnT1VzRnp5NVBYUnl4emRFMVNPNGRGQ2dZdUcKUHZTYTNObkJaU3JlV0N5Wkp2SWt5S3VjajZRZWhiN0ROUmlYSjJ4WkJSajUya2NGQ29LN1JQODlkNGI4ZlRkNQpjV0dhVDJYcDBsbmJyb0dRWUp2dklmVXZ1N21kdzJ4dUVxMGFFUTFkK00xUnB6eWVxbVRJNTBUcHcxMnBjbnVYCkJnNzI2NVZ3SUxhdVhTWWRxbklnTHJjZU1SejUxbnpaSFdMMHRBMjJMdWRJNE9uNkt0NlJLaXNTQlRsT0ZlYTAKTTQyOFBsVWkrTm9ENm0zUnpmVDhreXFTWS9rPQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==","verification":{}}]}},"timeouts":{},"version":"2.2.0"},"networkd":{},"passwd":{},"storage":{},"systemd":{}}
END_OF_MASTER_IGNITION

// Ignition config for the compute machines. You should copy the contents of the worker.ign generated by the installer.
compute_ignition = <<END_OF_WORKER_IGNITION
{"ignition":{"config":{"append":[{"source":"https://api-int.test.lab.local:22623/config/worker","verification":{}}]},"security":{"tls":{"certificateAuthorities":[{"source":"data:text/plain;charset=utf-8;base64,LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURFRENDQWZpZ0F3SUJBZ0lJUU5BQXRDbmhUVkl3RFFZSktvWklodmNOQVFFTEJRQXdKakVTTUJBR0ExVUUKQ3hNSmIzQmxibk5vYVdaME1SQXdEZ1lEVlFRREV3ZHliMjkwTFdOaE1CNFhEVEU1TVRJd016RTBNVFF6T1ZvWApEVEk1TVRFek1ERTBNVFF6T1Zvd0pqRVNNQkFHQTFVRUN4TUpiM0JsYm5Ob2FXWjBNUkF3RGdZRFZRUURFd2R5CmIyOTBMV05oTUlJQklqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FROEFNSUlCQ2dLQ0FRRUF1Q3FDSy8xYUtOOTAKQUFTamxQTTZxaFY2SHlPWGtKaFZKYm1YQzErbUthMG9CbnNiUGN4WE14NkFROXhSMSt0djJveFl1Q0lBRlNNUwplb3pIa2cyS2VhU25mQTJYdHVkRGZnaklTNUdsQ3lPRTJ4ZXZrZ1dzU0VVQXV3TmlTTXRRWS9BTWt1YnMyK3VZCmRVQUxUMHhsZWkyUWhoaFN6OXN3allNNitnaUh5K2JZMDZjZmxkbjF0NG5aRGZFeUlENFNaR25WeDBmVzNNZ3QKdVRPU200Q1E0WWpUZDlQd1pKL0RUckpkNG9sVjZHQlNoSXFBSHh0elBiaWZwMjZaTnY5MjRrN2trMFhKUDI5Lwp3MEErWDRPVG1tdnY0RVQ2SVJISDZlMFp6TW9BUkpReVF1ekJMUFhqZ0VGMFBtRDAvS2ZBSFlEc25Rdklza3JICldJa3A3enc5a3dJREFRQUJvMEl3UURBT0JnTlZIUThCQWY4RUJBTUNBcVF3RHdZRFZSMFRBUUgvQkFVd0F3RUIKL3pBZEJnTlZIUTRFRmdRVTI4TEhRTUlMZkNhREVBUGhhcWg0MmpVZjZvUXdEUVlKS29aSWh2Y05BUUVMQlFBRApnZ0VCQUFuQ2dCME9saTl4cmszZkVUK2hCOE9hQWtuRncra2NGVHdGMDNaYS9HeDkydW8wcjJZNlI2bDB2Q1NuCml2VHhoRzBMeDhaTlNYMURlSkFRbDY2YjkrTHF2QVhqdDNqUFpnT1VzRnp5NVBYUnl4emRFMVNPNGRGQ2dZdUcKUHZTYTNObkJaU3JlV0N5Wkp2SWt5S3VjajZRZWhiN0ROUmlYSjJ4WkJSajUya2NGQ29LN1JQODlkNGI4ZlRkNQpjV0dhVDJYcDBsbmJyb0dRWUp2dklmVXZ1N21kdzJ4dUVxMGFFUTFkK00xUnB6eWVxbVRJNTBUcHcxMnBjbnVYCkJnNzI2NVZ3SUxhdVhTWWRxbklnTHJjZU1SejUxbnpaSFdMMHRBMjJMdWRJNE9uNkt0NlJLaXNTQlRsT0ZlYTAKTTQyOFBsVWkrTm9ENm0zUnpmVDhreXFTWS9rPQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==","verification":{}}]}},"timeouts":{},"version":"2.2.0"},"networkd":{},"passwd":{},"storage":{},"systemd":{}}
END_OF_WORKER_IGNITION

// Set ipam and ipam_token if you want to use the IPAM server to reserve IP
// addresses for the VMs.

// Address or hostname of the IPAM server from which to reserve IP addresses for the cluster machines.
// ipam = "139.178.89.254"

// Token to use to authenticate with the IPAM server.
// ipam_token = "TOKEN_FOR_THE_IPAM_SERVER"

// Set bootstrap_ip, control_plane_ip, and compute_ip if you want to use static
// IPs reserved someone else, rather than the IPAM server.

// The IP address to assign to the bootstrap VM.
bootstrap_ip = "10.55.1.10"

// The IP addresses to assign to the control plane VMs. The length of this list
// must match the value of control_plane_count.
control_plane_ips = ["10.55.1.11", "10.55.1.12", "10.55.1.13"]

// The IP addresses to assign to the compute VMs. The length of this list must
// match the value of compute_count.
compute_ips = ["10.55.1.14", "10.55.1.15", "10.55.1.16"]
root:/home/arunabha/openshift/installer/upi/vsphere>
`hcl
...


### Debug Output
<!--
Full debug output can be obtained by running Terraform with the environment variable `TF_LOG=trace`. Please create a GitHub Gist containing the debug output. Please do _not_ paste the debug output in the issue, since debug output is long.

Debug output may contain sensitive information. Please review it before posting publicly, and if you are concerned feel free to encrypt the files using the HashiCorp security public key.
-->

### Crash Output
<!--
# terraform apply -auto-approve
data.ignition_systemd_unit.restart[1]: Refreshing state...
data.ignition_systemd_unit.restart[2]: Refreshing state...
data.ignition_systemd_unit.restart: Refreshing state...
data.ignition_file.hostname: Refreshing state...
data.ignition_systemd_unit.restart[0]: Refreshing state...
data.ignition_file.hostname[2]: Refreshing state...
data.ignition_systemd_unit.restart[0]: Refreshing state...
data.ignition_file.hostname[2]: Refreshing state...
data.ignition_file.hostname[1]: Refreshing state...
data.ignition_file.hostname[1]: Refreshing state...
data.ignition_file.hostname[0]: Refreshing state...
data.ignition_systemd_unit.restart[2]: Refreshing state...
data.ignition_file.hostname[0]: Refreshing state...
data.ignition_systemd_unit.restart[1]: Refreshing state...
data.ignition_file.static_ip[1]: Refreshing state...
data.ignition_file.static_ip[2]: Refreshing state...
data.ignition_file.static_ip[0]: Refreshing state...
data.ignition_file.static_ip[2]: Refreshing state...
data.ignition_file.static_ip: Refreshing state...
data.ignition_file.static_ip[0]: Refreshing state...
data.ignition_config.ign[1]: Refreshing state...
data.ignition_config.ign[2]: Refreshing state...
data.ignition_file.static_ip[1]: Refreshing state...
data.ignition_config.ign[0]: Refreshing state...
data.ignition_config.ign: Refreshing state...
data.ignition_config.ign[2]: Refreshing state...
data.ignition_config.ign[0]: Refreshing state...
data.ignition_config.ign[1]: Refreshing state...
data.vsphere_datacenter.dc: Refreshing state...
data.vsphere_datastore.datastore: Refreshing state...
data.vsphere_virtual_machine.template: Refreshing state...
data.vsphere_network.network: Refreshing state...
data.vsphere_network.network: Refreshing state...
data.vsphere_compute_cluster.compute_cluster: Refreshing state...
data.vsphere_virtual_machine.template: Refreshing state...
data.vsphere_network.network: Refreshing state...
data.vsphere_datastore.datastore: Refreshing state...
data.vsphere_virtual_machine.template: Refreshing state...
data.vsphere_datastore.datastore: Refreshing state...

Error: Error refreshing state: 3 errors occurred:
        * module.control_plane.data.ignition_config.ign: 3 errors occurred:
        * module.control_plane.data.ignition_config.ign[0]: data.ignition_config.ign.0: No valid JSON found, make sure you're using .rendered and not .id: invalid character 'c' after top-level value
        * module.control_plane.data.ignition_config.ign[1]: data.ignition_config.ign.1: No valid JSON found, make sure you're using .rendered and not .id: invalid character 'b' in literal false (expecting 'a')
        * module.control_plane.data.ignition_config.ign[2]: data.ignition_config.ign.2: No valid JSON found, make sure you're using .rendered and not .id: invalid character 'b' after top-level value


        * module.bootstrap.data.ignition_config.ign: 1 error occurred:
        * module.bootstrap.data.ignition_config.ign: data.ignition_config.ign: No valid JSON found, make sure you're using .rendered and not .id: invalid character 'b' after top-level value


        * module.compute.data.ignition_config.ign: 3 errors occurred:
        * module.compute.data.ignition_config.ign[0]: data.ignition_config.ign.0: No valid JSON found, make sure you're using .rendered and not .id: invalid character 'c' after top-level value
        * module.compute.data.ignition_config.ign[1]: data.ignition_config.ign.1: No valid JSON found, make sure you're using .rendered and not .id: invalid character 'e' after top-level value
        * module.compute.data.ignition_config.ign[2]: data.ignition_config.ign.2: No valid JSON found, make sure you're using .rendered and not .id: invalid character 'c' looking for beginning of value

-->

### Expected Behavior
<!--
terraform should able to deploy VM
-->

### Actual Behavior
<!--
terraform is unable to deply VM
-->

### Steps to Reproduce
<!--
Please list the full steps required to reproduce the issue, for example:
1. `terraform init`
2. `terraform apply -auto-approve`
-->

### Additional Context
<!--
Are there anything atypical about your situation that we should know? For example: is Terraform running in a wrapper script or in a CI system? Are you passing any unusual command line options or environment variables to opt-in to non-default behavior?
-->

### References
<!--
Are there any other GitHub issues (open or closed) or Pull Requests that should be linked here? For example:

- #6017

-->
@jchraibi
Copy link

I had the same issue, try forcing the ignition provider to version 1.1.0 by adding this block to the top of main.tf

provider "ignition" {
version = "1.1.0"
}

@ghost
Copy link

ghost commented Dec 12, 2019

This issue has been automatically migrated to hashicorp/terraform-provider-ignition#65 because it looks like an issue with that provider. If you believe this is not an issue with the provider, please reply to hashicorp/terraform-provider-ignition#65.

@ghost ghost closed this as completed Dec 12, 2019
@ghost
Copy link

ghost commented Mar 28, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@ghost ghost locked and limited conversation to collaborators Mar 28, 2020
This issue was closed.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
provider/ignition v0.11 Issues (primarily bugs) reported against v0.11 releases
Projects
None yet
Development

No branches or pull requests

3 participants