terraform crashes when trying to use "custom_field_filter" #17

diamconoa · 2020-06-17T12:36:09Z

Terraform Version

Terraform v0.12.26 and v.0.12.25
provider-phpipam v1.1

Terraform Configuration Files

https://github.com/diamconoa/terra-crash/blob/master/main.tf
https://github.com/diamconoa/terra-crash/blob/master/variables.tf

Crash Output

https://github.com/diamconoa/terra-crash/blob/master/crash.log

Expected Behavior

terraform apply with output from search request

Actual Behavior

terraform crashed

Steps to Reproduce

terraform init
terraform apply

Also note that I'm not using the documented syntax for "custom_field_filter", when copying an example from the documentation I get the following error:

Error: Unsupported block type

  on main.tf line 10, in data "phpipam_addresses" "address_search":
  10:   custom_field_filter {

Blocks of type "custom_field_filter" are not expected here. Did you mean to
define argument "custom_field_filter"? If so, use the equals sign to assign it
a value.

The text was updated successfully, but these errors were encountered:

lord-kyron · 2020-06-22T09:03:29Z

Hi @diamconoa ,
Yes, unfortunately the documentation needs to be updated with latest terraform 0.12 syntax requirements.
Actually it works, you just only need to changer the way defining it. The difference is one "=" (dash)
Old syntax
custom_field_filter { CustomTestAddresses = ".*terraform.*" }
New syntax:
custom_field_filter = { CustomTestAddresses = ".*terraform.*" }
I've already tested this with latest terraform 0.12.26 and it is working for me (no crashes)

diamconoa · 2020-06-24T09:26:48Z

Thanks for the reply @lord-kyron!
Unfortunately I cannot work around the crash. I've tried to recompile the plug-in from scratch on several different servers (ubuntu and rhel7), I've tried the ready-made version from releases section on github. I've also tried to run it on a new clean phpipam installation and still no luck.

This code is enough to get a crash:

provider "phpipam" {
  app_id   = "xxxx"
  endpoint = "xxxxx"
  password = "xxxxxx"
  username = "xxxxxxx"
}

data "phpipam_addresses" "address" {
  subnet_id         = 7
  custom_field_filter = { CustomTestAddresses = ".*terraform.*" }
}

output "search_result" {
  value = data.phpipam_addresses.address
}

I was thinking that perhaps the name of the custom field is important but no combination of name/value makes any difference. I would be really grateful if you could perhaps point me to a possible solution to this.

Also if you can suggest another way to simply get all registered addresses in a subnet without a filter it would be good enough for me. Right now I manage to get addresses only if description matches exactly. Otherwise I get an empty list.

pavel-z1 · 2020-06-30T19:19:56Z

Hi @diamconoa , inside PHPIPAM API configuration should be enabled option Nest custom fields
If this option was not enabled previously, enable it and try again.

Please provide here results.

diamconoa · 2020-07-01T08:35:36Z

Hi @pavel-z1,
Yes, this option was disabled, enabling it haven't solved the problem unfortunately.

panic: runtime error: invalid memory address or nil pointer dereference
2020-07-01T10:25:46.932+0200 [DEBUG] plugin.terraform-provider-phpipam: [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0xd499c4]
2020-07-01T10:25:46.932+0200 [DEBUG] plugin.terraform-provider-phpipam:
2020-07-01T10:25:46.932+0200 [DEBUG] plugin.terraform-provider-phpipam: goroutine 13 [running]:
2020-07-01T10:25:46.932+0200 [DEBUG] plugin.terraform-provider-phpipam: github.com/pavel-z1/phpipam-sdk-go/phpipam/client.(*Client).GetCustomFields(0xc000010260, 0xb, 0xfa7218, 0x9, 0xc000674690, 0x1, 0x0)
2020-07-01T10:25:46.932+0200 [DEBUG] plugin.terraform-provider-phpipam:         /home/rado/go/pkg/mod/github.com/pavel-z1/phpipam-sdk-go@v0.0.0-20200516074937-3a7bd299a171/phpipam/client/client.go:103 +0x64
2020-07-01T10:25:46.932+0200 [DEBUG] plugin.terraform-provider-phpipam: github.com/pavel-z1/phpipam-sdk-go/controllers/addresses.(*Controller).GetAddressCustomFields(...)
2020-07-01T10:25:46.932+0200 [DEBUG] plugin.terraform-provider-phpipam:         /home/rado/go/pkg/mod/github.com/pavel-z1/phpipam-sdk-go@v0.0.0-20200516074937-3a7bd299a171/controllers/addresses/addresses.go:125
2020-07-01T10:25:46.932+0200 [DEBUG] plugin.terraform-provider-phpipam: github.com/lord-kyron/terraform-provider-phpipam/plugin/providers/phpipam.addressSearchInSubnet(0xc0002360e0, 0xdcaf40, 0xc00067b080, 0x40dce6, 0xc000222840, 0x30, 0x30, 0xf60d20)
2020-07-01T10:25:46.932+0200 [DEBUG] plugin.terraform-provider-phpipam:         /home/rado/Projects/terraform-provider-phpipam/plugin/providers/phpipam/address_structure.go:243 +0x572
2020-07-01T10:25:46.932+0200 [DEBUG] plugin.terraform-provider-phpipam: github.com/lord-kyron/terraform-provider-phpipam/plugin/providers/phpipam.dataSourcePHPIPAMAddressesRead(0xc0002360e0, 0xdcaf40, 0xc00067b080, 0xc0002360e0, 0x0)
2020-07-01T10:25:46.932+0200 [DEBUG] plugin.terraform-provider-phpipam:         /home/rado/Projects/terraform-provider-phpipam/plugin/providers/phpipam/data_source_phpipam_addresses.go:36 +0x5d
2020-07-01T10:25:46.932+0200 [DEBUG] plugin.terraform-provider-phpipam: github.com/hashicorp/terraform-plugin-sdk/helper/schema.(*Resource).ReadDataApply(0xc0003f6a20, 0xc00055a460, 0xdcaf40, 0xc00067b080, 0xc0004ee980, 0x1, 0x0)
2020-07-01T10:25:46.932+0200 [DEBUG] plugin.terraform-provider-phpipam:         /home/rado/go/pkg/mod/github.com/hashicorp/terraform-plugin-sdk@v1.12.0/helper/schema/resource.go:403 +0x88
2020-07-01T10:25:46.932+0200 [DEBUG] plugin.terraform-provider-phpipam: github.com/hashicorp/terraform-plugin-sdk/helper/schema.(*Provider).ReadDataApply(0xc0000b0900, 0xc00022fad0, 0xc00055a460, 0xc00055a460, 0x0, 0x0)
2020-07-01T10:25:46.932+0200 [DEBUG] plugin.terraform-provider-phpipam:         /home/rado/go/pkg/mod/github.com/hashicorp/terraform-plugin-sdk@v1.12.0/helper/schema/provider.go:451 +0x8f
2020-07-01T10:25:46.932+0200 [DEBUG] plugin.terraform-provider-phpipam: github.com/hashicorp/terraform-plugin-sdk/internal/helper/plugin.(*GRPCProviderServer).ReadDataSource(0xc0000a8500, 0x11f3f20, 0xc000105f20, 0xc000140500, 0xc0000a8500, 0xc000105f20, 0xc0005e7b78)
2020-07-01T10:25:46.932+0200 [DEBUG] plugin.terraform-provider-phpipam:         /home/rado/go/pkg/mod/github.com/hashicorp/terraform-plugin-sdk@v1.12.0/internal/helper/plugin/grpc_provider.go:1036 +0x45d
2020-07-01T10:25:46.932+0200 [DEBUG] plugin.terraform-provider-phpipam: github.com/hashicorp/terraform-plugin-sdk/internal/tfplugin5._Provider_ReadDataSource_Handler(0xf68760, 0xc0000a8500, 0x11f3f20, 0xc000105f20, 0xc000112960, 0x0, 0x11f3f20, 0xc000105f20, 0xc000220090, 0x82)
2020-07-01T10:25:46.932+0200 [DEBUG] plugin.terraform-provider-phpipam:         /home/rado/go/pkg/mod/github.com/hashicorp/terraform-plugin-sdk@v1.12.0/internal/tfplugin5/tfplugin5.pb.go:3341 +0x217
2020-07-01T10:25:46.932+0200 [DEBUG] plugin.terraform-provider-phpipam: google.golang.org/grpc.(*Server).processUnaryRPC(0xc00008ac00, 0x11fdd60, 0xc00008b500, 0xc00021e000, 0xc000104930, 0x18d70b0, 0x0, 0x0, 0x0)
2020-07-01T10:25:46.932+0200 [DEBUG] plugin.terraform-provider-phpipam:         /home/rado/go/pkg/mod/google.golang.org/grpc@v1.27.1/server.go:1024 +0x501
2020-07-01T10:25:46.932+0200 [DEBUG] plugin.terraform-provider-phpipam: google.golang.org/grpc.(*Server).handleStream(0xc00008ac00, 0x11fdd60, 0xc00008b500, 0xc00021e000, 0x0)
2020-07-01T10:25:46.932+0200 [DEBUG] plugin.terraform-provider-phpipam:         /home/rado/go/pkg/mod/google.golang.org/grpc@v1.27.1/server.go:1313 +0xd3d
2020-07-01T10:25:46.932+0200 [DEBUG] plugin.terraform-provider-phpipam: google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc000506180, 0xc00008ac00, 0x11fdd60, 0xc00008b500, 0xc00021e000)
2020-07-01T10:25:46.932+0200 [DEBUG] plugin.terraform-provider-phpipam:         /home/rado/go/pkg/mod/google.golang.org/grpc@v1.27.1/server.go:722 +0xa1
2020-07-01T10:25:46.932+0200 [DEBUG] plugin.terraform-provider-phpipam: created by google.golang.org/grpc.(*Server).serveStreams.func1
2020-07-01T10:25:46.932+0200 [DEBUG] plugin.terraform-provider-phpipam:         /home/rado/go/pkg/mod/google.golang.org/grpc@v1.27.1/server.go:720 +0xa1
2020-07-01T10:25:46.933+0200 [DEBUG] plugin: plugin process exited: path=/home/dish01/.terraform.d/plugins/linux_amd64/terraform-provider-phpipam pid=83274 error="exit status 2"
2020/07/01 10:25:46 [ERROR] <root>: eval: *terraform.EvalReadData, err: rpc error: code = Unavailable desc = transport is closing
2020/07/01 10:25:46 [ERROR] <root>: eval: *terraform.EvalSequence, err: rpc error: code = Unavailable desc = transport is closing
2020/07/01 10:25:46 [TRACE] [walkRefresh] Exiting eval tree: data.phpipam_addresses.address
2020/07/01 10:25:46 [TRACE] vertex "data.phpipam_addresses.address": visit complete
2020/07/01 10:25:46 [TRACE] vertex "data.phpipam_addresses.address": dynamic subgraph encountered errors
2020/07/01 10:25:46 [TRACE] vertex "data.phpipam_addresses.address": visit complete
2020/07/01 10:25:46 [TRACE] dag/walk: upstream of "output.search_result" errored, so skipping
2020/07/01 10:25:46 [TRACE] dag/walk: upstream of "provider.phpipam (close)" errored, so skipping
2020/07/01 10:25:46 [TRACE] dag/walk: upstream of "root" errored, so skipping
2020/07/01 10:25:46 [TRACE] statemgr.Filesystem: removing lock metadata file .terraform.tfstate.lock.info
2020/07/01 10:25:46 [TRACE] statemgr.Filesystem: unlocking terraform.tfstate using fcntl flock
2020-07-01T10:25:46.936+0200 [DEBUG] plugin: plugin exited

pavel-z1 · 2020-07-12T14:35:13Z

Issue is replicated on my side.
Reopen this issue.

basic39 · 2020-09-04T08:21:42Z

Hello @pavel-z1,

can we expect any solution to this problem in the foreseeable future?
At least the "searching by vlan id"-feature, if it is easier to make.

pavel-z1 · 2020-09-17T07:32:09Z

Hi @basic39 , I will try to look this week on this.

pavel-z1 · 2020-09-18T08:42:30Z

I've updated phpipam-sdk-go - pavel-z1/phpipam-sdk-go@c193c4a

Issue fixed.

My example:

data "phpipam_addresses" "address_search" {
  subnet_id = 3

  custom_field_filter = {
    custom_Vlan = "vlantest"
  }
}

data "phpipam_address" "addresses" {
  count      = length(data.phpipam_addresses.address_search.address_ids)
  address_id = element(data.phpipam_addresses.address_search.address_ids, count.index)
}

output "ip_addresses" {
  value = [data.phpipam_address.addresses.*.ip_address]
}

Result:

# terraform apply -var-file=credentials.tfvars
data.phpipam_addresses.address_search: Refreshing state... [id=5]
data.phpipam_address.addresses[0]: Refreshing state... [id=1]
data.phpipam_address.addresses[1]: Refreshing state... [id=4]

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

ip_addresses = [
  [
    "10.10.1.3",
    "10.10.1.6",
  ],
]

Please note, that option Nest custom fields inside API application configuration should be disable in the PHPIPAM.

@diamconoa , @basic39 try to rebuild phpipam provider and check now.

basic39 · 2020-09-22T16:24:29Z

@diamconoa , @basic39 try to rebuild phpipam provider and check now.

Hello @pavel-z1 ,

Terraform still crashes. My goal is to identify the subnet by VLAN Number.

Here my code:

data "phpipam_section" "section" {
  name = var.IPAM_SECTION
}

data "phpipam_subnet" "subnet" {
  section_id = data.phpipam_section.section.id
  custom_field_filter = {
    VLAN = "1234"
  }
}

data "phpipam_first_free_address" "next_address" {
  subnet_id = data.phpipam_subnet.subnet.subnet_id
}

resource "phpipam_address" "newip" {
  subnet_id   = data.phpipam_subnet.subnet.subnet_id
  ip_address  = data.phpipam_first_free_address.next_address.ip_address
  hostname    = var.SERVERNAME
}

How I made the rebuild:
First I replaced the file "phpipam/client/client.go" with the file from your repo, then:

[root@STF01 git]# go mod init terraform-provider-phpipam
go mod init: go.mod already exists
[root@STF01 git]# go mod vendor
go: downloading google.golang.org/appengine v1.6.1
[root@STF01 git]# make build
go get -u github.com/mitchellh/gox
go: github.com/hashicorp/go-version upgrade => v1.2.1
gox -osarch="linux/amd64 windows/amd64 darwin/amd64" \
-output="pkg/{{.OS}}_{{.Arch}}/terraform-provider-phpipam" .
Number of parallel builds: 3
-->    darwin/amd64: github.com/lord-kyron/terraform-provider-phpipam
-->     linux/amd64: github.com/lord-kyron/terraform-provider-phpipam
-->   windows/amd64: github.com/lord-kyron/terraform-provider-phpipam
[root@STF01 git]# go build -o terraform-provider-phpipam
[root@STF01 git]# cp terraform-provider-phpipam /root/.terraform.d/plugins/terraform-provider-phpipam

Here the output:
phpipam-tf-crash_2020-09-22.txt

Please note, that option Nest custom fields inside API application configuration should be disable in the PHPIPAM.

My colleagues, that are responsible for PHPIPAM couldnt find this option. Can you please clarify how to check if this option is enabled?

diamconoa · 2020-09-23T11:24:46Z

Hello @pavel-z1,

I've tried to rebuild the provider and so far it looks promising. It didn't crash on 'terraform plan' of a test job. Haven't tried the real application yet but plan to try it out soon.

@basic39 perhaps you need to install a new version of phpipam-sdk-go module before building the provider. I had to update the version of it to 'latest' in terraform-provider-phpipam/go.mod, it pointed to the old version otherwise.
Also I happen to know about the location of the "Nest custom fields" option in phpipam. Took me a while to find it. It is an option of a API management key that you can create under Administration/API.

diamconoa · 2020-10-02T10:27:54Z

Hello everyone,

I've now tested the new build more thoroughly and it works as expected. I suppose this repository should be updated to use the latest version of phpipam-sdk-go and then we can close this issue.

basic39 · 2020-10-02T12:52:16Z

Hello everyone,

I checked, that "Nest custom fields" is disabled. Thank you @diamconoa.

But terraform still crashes.

provider "phpipam" {
  app_id   = xx
  endpoint = xx
  password = xx
  username = xx
  insecure = true
}

data "phpipam_addresses" "address_search" {
  subnet_id = 341

  custom_field_filter = {
    custom_Vlan = "1234"
  }
}

data "phpipam_address" "addresses" {
  count      = length(data.phpipam_addresses.address_search.address_ids)
  address_id = element(data.phpipam_addresses.address_search.address_ids, count.index)
}

output "ip_addresses" {
  value = [data.phpipam_address.addresses.*.ip_address]
}

Did I something wrong?
I replaced the "phpipam-sdk-go"-folder with the file from @pavel-z1 repo and then executed:

go install
go build
cp terraform-provider-phpipam ~/.terraform.d/plugins/

This code:

data "phpipam_section" "section" {
  name = "Management"
}

data "phpipam_subnet" "subnet" {
  section_id = data.phpipam_section.section.id
  custom_field_filter = {  VLAN = "1234"  }
}


output "ip_addresses" {
  value = [data.phpipam_subnet.subnet.subnet_id]
}

produces the same error.

phpipam-tf-crash_2020-10-02.txt

Terraform v0.12.26

pavel-z1 · 2020-10-16T08:43:30Z

Hi @basic39 ,
I can't reproduce you error with latest php-sdk-go commit.

Perhaps @diamconoa is right about for your case. Version of github.com/pavel-z1/phpipam-sdk-go module should be updated, then you will get the latest and actual code during build process.

I've updated github.com/pavel-z1/phpipam-sdk-go module version.
Also, created the new version of terraform-provider-phpipam (v1.1.1). Source code link - https://github.com/lord-kyron/terraform-provider-phpipam/tree/v1.1.1

@basic39 try to build phpipam provider with this source code https://github.com/lord-kyron/terraform-provider-phpipam/tree/v1.1.1
If issue will be resolved - new version of phpipam will be released.

pavel-z1 · 2020-10-19T06:39:40Z

I've pushed changes to the master branch.
Closing this issue as solved.

@basic39 , If your error persists, let us know.

lord-kyron closed this as completed Jun 22, 2020

pavel-z1 mentioned this issue Jun 30, 2020

Terraform crashes when using custom_field_filter #19

Closed

pavel-z1 reopened this Jul 12, 2020

pavel-z1 self-assigned this Jul 12, 2020

pavel-z1 closed this as completed Oct 19, 2020

basic39 mentioned this issue Oct 21, 2020

Add "l2_domain_id" to the Argument Reference of "phpipam_vlan" Data Source #27

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

terraform crashes when trying to use "custom_field_filter" #17

terraform crashes when trying to use "custom_field_filter" #17

diamconoa commented Jun 17, 2020

lord-kyron commented Jun 22, 2020

diamconoa commented Jun 24, 2020 •

edited

Loading

pavel-z1 commented Jun 30, 2020

diamconoa commented Jul 1, 2020

pavel-z1 commented Jul 12, 2020

basic39 commented Sep 4, 2020

pavel-z1 commented Sep 17, 2020

pavel-z1 commented Sep 18, 2020

basic39 commented Sep 22, 2020

diamconoa commented Sep 23, 2020

diamconoa commented Oct 2, 2020

basic39 commented Oct 2, 2020

pavel-z1 commented Oct 16, 2020

pavel-z1 commented Oct 19, 2020

terraform crashes when trying to use "custom_field_filter" #17

terraform crashes when trying to use "custom_field_filter" #17

Comments

diamconoa commented Jun 17, 2020

Terraform Version

Terraform Configuration Files

Crash Output

Expected Behavior

Actual Behavior

Steps to Reproduce

lord-kyron commented Jun 22, 2020

diamconoa commented Jun 24, 2020 • edited Loading

pavel-z1 commented Jun 30, 2020

diamconoa commented Jul 1, 2020

pavel-z1 commented Jul 12, 2020

basic39 commented Sep 4, 2020

pavel-z1 commented Sep 17, 2020

pavel-z1 commented Sep 18, 2020

basic39 commented Sep 22, 2020

diamconoa commented Sep 23, 2020

diamconoa commented Oct 2, 2020

basic39 commented Oct 2, 2020

pavel-z1 commented Oct 16, 2020

pavel-z1 commented Oct 19, 2020

diamconoa commented Jun 24, 2020 •

edited

Loading