Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

0.6.16 - azurerm - unable to obtain public IP #6634

Closed
ghost opened this issue May 12, 2016 · 36 comments
Closed

0.6.16 - azurerm - unable to obtain public IP #6634

ghost opened this issue May 12, 2016 · 36 comments

Comments

@ghost
Copy link

@ghost ghost commented May 12, 2016

Terraform Version

0.6.16

Affected Resource(s)

azurerm_public_ip

Terraform Configuration Files

resource "azurerm_public_ip" "ap-service-discovery-server-0" {
    name = "${var.node_type}-0-public-ip"
    location = "${var.location}"
    resource_group_name = "${var.resource_group_name}"
    public_ip_address_allocation = "dynamic"
}

resource "azurerm_network_interface" "ap-service-discovery-server-0" {
    name = "${var.node_type}-0-nic"
    location = "${var.location}"
    resource_group_name = "${var.resource_group_name}"
    network_security_group_id = "${azurerm_network_security_group.service_discovery_server.id}"

    ip_configuration {
        name = "${var.node_type}-0-ip-config"
        subnet_id = "${element(split(",", var.subnet_id), 0)}"
        private_ip_address_allocation = "dynamic"
        public_ip_address_id = "${azurerm_public_ip.ap-service-discovery-server-0.id}"
    }
}

resource "azurerm_virtual_machine" "ap-service-discovery-server-0" {
    name = "${var.node_type}-0"
    resource_group_name = "${var.resource_group_name}"
    location = "${var.location}"
    vm_size = "${var.instance_type}"
    network_interface_ids = ["${azurerm_network_interface.ap-service-discovery-server-0.id}"]
    availability_set_id = "${azurerm_availability_set.ap-service-discovery-availability-set.id}"


    storage_os_disk {
        name = "${var.node_type}-0-osdisk"
        #Source VHD as reference
        image_uri = "${var.source_vhd_path}"
        #Destination vhd to create
        vhd_uri = "${var.vhd_storage_base_uri}${var.node_type}-0.vhd"
        create_option = "fromImage"
        os_type = "linux"
    }

    os_profile {
        computer_name = "${var.node_type}-0"
        admin_username = "${var.ssh_username}"
        admin_password = "${var.ssh_password}"
        # Custom data must be base64 and lt 87380 chars
        custom_data = "${base64encode(template_file.user_data_0.rendered)}"
    }

    os_profile_linux_config {
       disable_password_authentication = true
        ssh_keys {
          path = "/home/${var.ssh_username}/.ssh/authorized_keys"
          key_data = "${file("${var.ssh_key_path}")}"
        }
    }
}

output "public_ips" { value = "${azurerm_public_ip.ap-service-discovery-server-0.ip_address}" }

Expected Behavior

The public ip should be retrievable as per the docs.

Actual Behavior

  • Resource 'azurerm_public_ip.ap-service-discovery-server-0' does not have attribute 'ip_address' for variable 'azurerm_public_ip.ap-service-discovery-server-0.ip_address'

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

  1. create a vm, with a public IP, then try and access that public ip from another resource or for module output.
@stack72

This comment has been minimized.

Copy link
Contributor

@stack72 stack72 commented May 12, 2016

Hi @tasquith

Am looking at this right now - can you post your full config (minus secrets) - I only see partial config above as it's missing RGs and Availability Sets etc

P.

@ghost

This comment has been minimized.

Copy link
Author

@ghost ghost commented May 12, 2016

Thanks @stack72,

Our approach makes heavy use of modules and tfvar files - so posting the full config is a bit of a task!

I'll get you a streamlined config that you should just be able to apply.

Cheers,

Tom

@ghost

This comment has been minimized.

Copy link
Author

@ghost ghost commented May 12, 2016

I've written a noddy example where the issue still occurs. I couldn't get it as an output, so I've just referenced the field in a tag for another RG.

resource "azurerm_resource_group" "test" {
  name     = "acctestrg"
  location = "North Europe"
}

resource "azurerm_virtual_network" "test" {
  name                = "acctvn"
  address_space       = ["10.0.0.0/16"]
  location            = "North Europe"
  resource_group_name = "${azurerm_resource_group.test.name}"
}

resource "azurerm_subnet" "test" {
  name                 = "acctsub"
  resource_group_name  = "${azurerm_resource_group.test.name}"
  virtual_network_name = "${azurerm_virtual_network.test.name}"
  address_prefix       = "10.0.2.0/24"
}

resource "azurerm_public_ip" "test" {
    name = "public-ip"
    location = "North Europe"
    resource_group_name = "${azurerm_resource_group.test.name}"
    public_ip_address_allocation = "dynamic"
}

resource "azurerm_network_interface" "test" {
  name                = "acctni"
  location            = "North Europe"
  resource_group_name = "${azurerm_resource_group.test.name}"

  ip_configuration {
    name                          = "testconfiguration1"
    subnet_id                     = "${azurerm_subnet.test.id}"
    private_ip_address_allocation = "dynamic"
  }
}

resource "azurerm_storage_account" "test" {
  name                = "hashicorpaccsa"
  resource_group_name = "${azurerm_resource_group.test.name}"
  location            = "northeurope"
  account_type        = "Standard_LRS"

  tags {
    environment = "staging"
  }
}

resource "azurerm_storage_container" "test" {
  name                  = "vhds"
  resource_group_name   = "${azurerm_resource_group.test.name}"
  storage_account_name  = "${azurerm_storage_account.test.name}"
  container_access_type = "private"
}

resource "azurerm_virtual_machine" "test" {
  name                  = "acctvm"
  location              = "North Europe"
  resource_group_name   = "${azurerm_resource_group.test.name}"
  network_interface_ids = ["${azurerm_network_interface.test.id}"]
  vm_size               = "Standard_A0"

  storage_image_reference {
    publisher = "Canonical"
    offer     = "UbuntuServer"
    sku       = "14.04.2-LTS"
    version   = "latest"
  }

  storage_os_disk {
    name          = "myosdisk1"
    vhd_uri       = "${azurerm_storage_account.test.primary_blob_endpoint}${azurerm_storage_container.test.name}/myosdisk1.vhd"
    caching       = "ReadWrite"
    create_option = "FromImage"
  }

  os_profile {
    computer_name  = "hostname"
    admin_username = "testadmin"
    admin_password = "Password1234!"
  }

  os_profile_linux_config {
    disable_password_authentication = false
  }
}


resource "azurerm_resource_group" "somewheretoputtheip" {
  name     = "testResourceGroup1"
  location = "North Europe"

  tags {
    environment = "${azurerm_public_ip.test.ip_address}"
  }
}
@stack72

This comment has been minimized.

Copy link
Contributor

@stack72 stack72 commented May 12, 2016

Hi @tasquith

So I have been able to run through this and reproduce as you see. The issue, that i can see seems to be the response from the azure api:

{
    "name": "public-ip",
    "id": "/subscriptions/34ca515c-4629-458e-bf7c-738d77e0d0ea/resourceGroups/acctestrg/providers/Microsoft.Network/publicIPAddresses/public-ip",
    "etag": "W/\"e4661937-a339-4590-9c05-740ed985c4ee\"",
    "type": "Microsoft.Network/publicIPAddresses",
    "location": "northeurope",
    "tags": {},
    "properties": {
        "provisioningState": "Succeeded",
        "resourceGuid": "e21ada16-28c8-4113-b648-d4ab99f37b13",
        "publicIPAllocationMethod": "Dynamic",
        "idleTimeoutInMinutes": 4
    }
}

As you can see, the API doesn't return an IP for a Dynamic allocation.

https://azure.microsoft.com/en-gb/documentation/articles/virtual-network-ip-addresses-overview-arm/

This article says as following:

Allocation method

There are two methods in which an IP address is allocated to a public IP resource - dynamic or static. The default allocation method is dynamic, where an IP address is not allocated at the time of its creation. Instead, the public IP address is allocated when you start (or create) the associated resource (like a VM or load balancer). The IP address is released when you stop (or delete) the resource. This causes the IP address to change when you stop and start a resource.

Thoughts?

P.

@ghost

This comment has been minimized.

Copy link
Author

@ghost ghost commented May 12, 2016

Thanks @stack72,

That is a pain, but its understandable! I'm trying to use that IP to then create a DNS entry with DNSimple, so thats a blocker. I'll try with assigning them statically and see how I get on!

Cheers,

Tom

@stack72

This comment has been minimized.

Copy link
Contributor

@stack72 stack72 commented May 12, 2016

sorry @tasquith

I am trying to work out where the public ip is then actually assigned but I cannot find it in the API calls - I have the full trace of the entire tree from charles proxy to try and see what the requests are doing

P.

@ghost

This comment has been minimized.

Copy link
Author

@ghost ghost commented May 12, 2016

No worries! Using statically assigned IPs is a workaround, so I can still carry on :)

Thanks for the help!

Tom

@stack72 stack72 added enhancement and removed bug labels May 17, 2016
@stack72

This comment has been minimized.

Copy link
Contributor

@stack72 stack72 commented May 17, 2016

@tasquith

I am going to change this from a bug to an enhancement - this is not a bug in the provider as it works as expected. I will have to try and raise this with MSFT

Paul

@ghost

This comment has been minimized.

Copy link
Author

@ghost ghost commented May 17, 2016

@stack72

No worries, Cheers Paul

@igable

This comment has been minimized.

Copy link
Contributor

@igable igable commented May 31, 2016

@tasquith, @stack72 , There is another reasonable workaround that you can use for this problem, especially when needing more IPs then the max allowed static IPs:

resource "azurerm_public_ip" "client" {
    count = 2
    name = "client${count.index}"
    location = "West US"
    resource_group_name = "${azurerm_resource_group.somegroup.name}"
    # below results in a dns name that looks like:
    #client01-48e35078
    domain_name_label = "${format("client%02d-%.8s",count.index,  uuid())}"
    public_ip_address_allocation = "dynamic"

    lifecycle {
        # ignore because we are generating with uuid()
        ignore_changes = ["domain_name_label"]
    }
}

Then you can access the FQDN which microsoft does return. You can then use that FQDN in a provisioner connection block like this.:

    connection {
        user = "ubuntu"
        host = "${element(azurerm_public_ip.client.*.fqdn, count.index)}"
        timeout ="15m"
    }
@stack72

This comment has been minimized.

Copy link
Contributor

@stack72 stack72 commented Jun 28, 2016

@tasquith just a FYI, the new version of the SDK (#7379) will allow this to be fixed

Just waiting on that to be merged and then will add the public IP part

@ghost

This comment has been minimized.

Copy link
Author

@ghost ghost commented Jun 29, 2016

@stack72

Cheers Paul,

Will this be part of 0.7?

@stack72

This comment has been minimized.

Copy link
Contributor

@stack72 stack72 commented Jun 29, 2016

yes :)

@ghost

This comment has been minimized.

Copy link
Author

@ghost ghost commented Jul 25, 2016

@stack72 - Did the new version of the SDK fix this?

@jgoldschrafe

This comment has been minimized.

Copy link
Contributor

@jgoldschrafe jgoldschrafe commented Aug 9, 2016

I can confirm this issue still exists on v0.7.0.

@xynova

This comment has been minimized.

Copy link

@xynova xynova commented Sep 27, 2016

I do get an ip address back but it is not the same one displays in the portal and cannot be used to access the vm either :/ (v0.7.4)

@pmcatominey

This comment has been minimized.

Copy link
Contributor

@pmcatominey pmcatominey commented Nov 18, 2016

I wonder if a data source would be a solution for this, it would be able to read the IP after the assigned resource (VM etc.) has started.

@moritzheiber

This comment has been minimized.

Copy link

@moritzheiber moritzheiber commented Jan 20, 2017

Any progress on this? I guess the workaround is to assign a static public IP, yet that'll put you in a corner pretty soon 😉

@nickithewatt

This comment has been minimized.

Copy link
Contributor

@nickithewatt nickithewatt commented Jan 27, 2017

Still an issue in 0.8.4 :(

@sozercan

This comment has been minimized.

Copy link
Contributor

@sozercan sozercan commented Mar 9, 2017

still an issue in 0.8.8

@githubvick

This comment has been minimized.

Copy link

@githubvick githubvick commented Apr 20, 2017

Not fixed in v0.9.3 either. This should be really useful if fixed.

I mean is there no way in which we can get the (Dynamic) public IP address of a newly created VM in Azure?

Update:

One workaround is by running a separate provisioner and getting the IP Address

resource "null_resource" "ansible-provision" {


  provisioner "local-exec" {
    command =  "echo \"[masters]\n ansible_ssh_host=${azurerm_public_ip.helloterraformips.ip_address} \" >> /home/osboxes/ansible-kube/ansible/inventory/testinv"
  }

@githubvick

This comment has been minimized.

Copy link

@githubvick githubvick commented Apr 21, 2017

The above worked only once, I dont know how. The workaround I'm proceeding with is running the provisioner separately after the VM is created as it is able to detect on the second run.

@longmuir

This comment has been minimized.

Copy link

@longmuir longmuir commented May 15, 2017

I'm still seeing this in 0.9.5 as well.

Public IP addresses aren't returned as outputs (but are if you run 'terraform apply' a second time, which seems to be a current work around for outputs). It also causes problems with a file provisioner, though I'm using the workaround @igable described above.

@rcarun rcarun self-assigned this Jun 2, 2017
@rcarun

This comment has been minimized.

Copy link

@rcarun rcarun commented Jun 2, 2017

Hi Longmuir, Just wanted to clarify that this is not a blocker for you? How painful is the workaround in your mind? We were considering making IP a Data Source and dynamically returning the address as an output. Will this solve your problem? Please let us know.

@abhijeetgaiha

This comment has been minimized.

Copy link
Contributor

@abhijeetgaiha abhijeetgaiha commented Jun 7, 2017

A dynamic public IP is allocated by Azure once the VM that is assigned the public IP has been created and has booted. You can now use the public IP data source to get the IP address for dynamic IPs. To ensure that the data source is processed by terraform after the VM has been created, use the resource_group_name property of the VM as an input to the public IP data source:

data "azurerm_public_ip" "datasourceip" { name = "myPublicIP" resource_group_name = "${azurerm_virtual_machine.myVirtualMachine.resource_group_name}" }

The IP address can then be accessed using the ip_address property:
${data.azurerm_public_ip.datasourceip.ip_address}

@aleksap

This comment has been minimized.

Copy link

@aleksap aleksap commented Jul 10, 2017

Was anyone able to get this work?

I still have issues and can't get provisioner to connect to the VM, neither via ip or fqdn.. can't get correct output with any of those two

@abhijeetgaiha

This comment has been minimized.

Copy link
Contributor

@abhijeetgaiha abhijeetgaiha commented Jul 10, 2017

@aleksap please see my last comment. If you have any further queries, I'd need happy to look into it.

@aleksap

This comment has been minimized.

Copy link

@aleksap aleksap commented Jul 10, 2017

@abhijeetgaiha

Thanks for the reply.

I have tried your solution but I had some issues.
I am using modules so I have add your data resource in my network module.
Then I have created output:

output "datasourceip" {
 value = "${data.azurerm_public_ip.datasourceip.ip_address}"
  }

I would call that output from my template via

public_ip = "${module.network.datasourceip}

But it fails..
Am I doing this right or am I missing something?

Thanks,
Alex

@abhijeetgaiha

This comment has been minimized.

Copy link
Contributor

@abhijeetgaiha abhijeetgaiha commented Jul 11, 2017

@aleksap By fails, do you mean you get an error message or an empty value for the IP address?
Also, please keep in mind that the dynamic IP for a VM isn't allocated until the VM is booted. To ensure that the VM is booted, I suggest using the resource group name property of the VM when configuring the IP data source.

@aleksap

This comment has been minimized.

Copy link

@aleksap aleksap commented Jul 11, 2017

@abhijeetgaiha ,

Error that I am getting is:

  • output 'datasource': unknown resource 'data.azurerm_public_ip.datasourceip' referenced in variable data.azurerm_public_ip.datasourceip.ip_address

As I have mentioned above, I have resource data in network module, I have correctly configured output.. I don't see a reason why it would throw me this error.

Thanks,
Alex

@abhijeetgaiha

This comment has been minimized.

Copy link
Contributor

@abhijeetgaiha abhijeetgaiha commented Jul 13, 2017

@aleksap I'm looking into this. It would help if you could provide your config files.

@aleksap

This comment has been minimized.

Copy link

@aleksap aleksap commented Jul 13, 2017

@abhijeetgaiha ,

Here is my network module:


# create virtual network
resource "azurerm_virtual_network" "network" {
  name                = "${format("%s-vnet", var.name)}"
  address_space       = ["${var.address_space}"]
  location            = "${var.location}"
  resource_group_name = "${var.resource_group_name}"
}

# create subnets
resource "azurerm_subnet" "subnets" {
  count                = "${length(var.subnet_cidrs)}"
  name                 = "${format("%s-subnet-%02d", var.name, count.index+1)}"
  resource_group_name  = "${var.resource_group_name}"
  virtual_network_name = "${azurerm_virtual_network.network.name}"
  address_prefix       = "${var.subnet_cidrs[count.index]}"
}

#create public ip (ip_type: static or dynamic)
resource "azurerm_public_ip" "public_ip" {
  name                         = "${format("%s-public_ip-%02d", var.name, count.index+1)}"
  location                     = "${var.location}"
  resource_group_name          = "${var.resource_group_name}"
  #count                        = "${var.vm_count}"
  domain_name_label            = "${var.domain_name}"
  public_ip_address_allocation = "${var.public_ip_type}"
}


# create nic
resource "azurerm_network_interface" "network" {
  #count               = "${length(var.subnet_cidrs)}"
  name                = "${format("%s-network-interface%02d", var.name, count.index+1)}"
  location            = "${var.location}"
  resource_group_name = "${var.resource_group_name}"

  ip_configuration {
    name                          = "${format("%s-public_ip", var.name)}"
    subnet_id                     = "${element(azurerm_subnet.subnets.*.id, count.index)}"
    private_ip_address_allocation = "${var.private_ip_type}"
    public_ip_address_id          = "${azurerm_public_ip.public_ip.id}"
  }

data "azurerm_public_ip" "datasourceip" {
  name                          = "testing"
  resource_group_name           = "${var.resource_group_name}"
}
 }

Here are all the outputs that I have tried:

output "id" {
value = ["${azurerm_network_interface.network.*.id}"]
}

output "pip" {
value = "${element(azurerm_public_ip.public_ip.fqdn)}"
 }

 output "pubip" {
 value = "${element(azurerm_public_ip.public_ip.ip_address)}"
  }

 output "datasource" {
 value = "${data.azurerm_public_ip.datasourceip.ip_address}"
  }

Output "id" works fine when it's called from template file.
Output pip or pubip don't give me either ip_address or fqdn.. they give me random value, for ex:

module.vm.azurerm_virtual_machine.vm (chef): Connecting to remote host via SSH... module.vm.azurerm_virtual_machine.vm (chef): Host: 74D93920-ED26-11E3-AC10-0800200C9A66 module.vm.azurerm_virtual_machine.vm (chef): User: cq module.vm.azurerm_virtual_machine.vm (chef): Password: true module.vm.azurerm_virtual_machine.vm (chef): Private key: false module.vm.azurerm_virtual_machine.vm (chef): SSH Agent: false

When output "datasource" is called I get error:
output 'datasource': unknown resource 'data.azurerm_public_ip.datasourceip' referenced in variable data.azurerm_public_ip.datasourceip.ip_address

@jeromeza

This comment has been minimized.

Copy link

@jeromeza jeromeza commented Aug 7, 2017

I'm also seeing this issue in Terraform v0.10.0.

Same issue:

Error applying plan:

1 error(s) occurred:

  • azurerm_virtual_machine.prod_vm1: 1 error(s) occurred:

  • Resource 'azurerm_public_ip.prod_pubip' does not have attribute 'ip_address' for variable 'azurerm_public_ip.prod_pubip.ip_address'

Terraform does not automatically rollback in the face of errors.
Instead, your Terraform state file has been partially updated with
any resources that successfully completed. Please address the error
above and apply again to incrementally change your infrastructure.

@mikenorgate

This comment has been minimized.

Copy link

@mikenorgate mikenorgate commented Sep 14, 2017

I have managed to work around this issue using the approach from @abhijeetgaiha with one extra addition

Make sure that where you use the data source has a dependency on the virtual machine

Here is my working config:

resource "azurerm_public_ip" "test_host_public_ip" {
    name                              = "test-ip"
    public_ip_address_allocation      = "dynamic"
    ...
}

resource "azurerm_virtual_machine" "test_host" {
    name                              = "test-vm"
    ...
}

data "azurerm_public_ip" "datasourceip" { 
    name = "${azurerm_public_ip.test_host_public_ip.name}" 
    resource_group_name = "${azurerm_virtual_machine.test_host.resource_group_name}" 
}

output "test_ip_addresses" {
    value = "${data.azurerm_public_ip.datasourceip.ip_address}"

    depends_on = ["azurerm_virtual_machine.test_host"]
}
@m4h3

This comment has been minimized.

Copy link

@m4h3 m4h3 commented Sep 19, 2017

@mikenorgate I've tried your solution and it doesn't work for me.
After a terraform refresh i'm getting the public ip, but the dependency on the vm isn't strong enough apparently for the apply part.
I'm not sure how depend works, does it wait for resource to be fully created ? Does azure provide a dynamic IP address immediately after the VM creation? It seems to race from my point of view.

Deploying on West-Europe.

@rahulkp220

This comment has been minimized.

Copy link

@rahulkp220 rahulkp220 commented Dec 6, 2017

@mikenorgate depends_on = ["azurerm_virtual_machine.test_host"] this value should be inside data , not output.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.