Feature Request - Allow list/array in query and result

[hashibot]

This issue was originally opened by @vikas027 as hashicorp/terraform#12249. It was migrated here as part of the provider split. The original body of the issue is below.

---

Terraform Version

0.8.7

Affected Resource(s)

external data source

Terraform Configuration Files

data "external" "create_policy" {
  program = ["bash", "${path.module}/create_policy.sh"]

  query = {
    list_of_images = "${var.list_of_images}"
  }
}

Expected Behavior

List list_of_images should have been passed to the script.

Actual Behavior

Error refreshing state: 1 error(s) occurred:

* query: 1 error(s) decoding:

* '[list_of_images]' expected type 'string', got unconvertible type '[]interface {}'

---

Terraform Version

0.8.7

Affected Resource(s)

• external

Terraform Configuration Files

data "external" "create_policy" {
  program = ["bash", "${path.module}/create_policy.sh"]
}

create_policy.sh

#!/bin/bash
set -e
# This works
jq -n --arg foobaz "$FOOBAZ" '{"foobaz":$foobaz}'
# This throws an error
jq -n '{"Version":"2008-10-17","Statement":[{"Sid":"repo_policy","Effect":"Allow","Principal":{"AWS":["arn:aws:iam::${account_id}:root","arn:aws:iam::${account_id}:role/ecr_restricted_admin"]},"Action":"*"}]}'

Expected Behavior

Terraform shouldn't thrown an error when the the bash script is producing a valid json

$ bash modules/ecr/create_policy.sh  | jq .
{
  "Version": "2008-10-17",
  "Statement": [
    {
      "Sid": "repo_policy",
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "arn:aws:iam::${account_id}:root",
          "arn:aws:iam::${account_id}:role/ecr_restricted_admin"
        ]
      },
      "Action": "*"
    }
  ]
}

Actual Behavior

* data.external.create_policy: command "bash" produced invalid JSON: json: cannot unmarshal array into Go value of type string

Steps to Reproduce

Create the resource and the script files and run terraform plan

[tomdavidson]

@apparentlymart Is this a viable universal work around?:

 query = {
    params = "base64endcode(jsonencode(var.list_of_images))"
  }

In the bash script use tf to decode:

Q=$(echo 'base64decode($PARAMS)' | terraform console) 

# do sripty stuff on Q

echo 'base64decode(jsonencode($OUTPUT))' | terraform console

[bgshacklett]

This workaround relies on bash being installed, meaning that it's not easily portable between macOS/Linux and Windows. In practice, I'm aware that Windows can run Bash, but depending on it for cross-platform compatibility is less than ideal.

In my personal use case, I've got a data structure which I'd like to pass directly to jq to keep the dependencies very lightweight. Most developers are likely to have jq on their systems so I'm much less bothered by requiring it as a dependency.

[casey-robertson]

Would really like this too. I want to build GCP custom roles by calling the gcloud CLI to grab permissions from built-in roles.... but for the life of me cannot get them back in a format TF is ok with. If it would just accept an array.

[karolinepauls]

Please rename the issue to include the result attribute. The issue request mentions both result and query in its contents. It would be a shame to get only one of the 2 cases mentioned fixed.

[bioshazard]

After being frustrated by the very limited JSON parsing capacity of the external provider, I found that jsondecode on a local file containing the same output works fine in a locals definition:

locals {
  # https://github.com/hashicorp/terraform/issues/12249
  # Ideally queried directly, but must be parsed locally.
  local_data = jsondecode(file("${path.module}/lib/test.json"))
  my_clusters = local.local_data.clusters

If the external provider simply parsed the stdout with the built-in jsondecode it would have the outcome I need.