Multi-prefix on JSON configuration file gets overwritten

[calvn]

Multi-prefix via configuration file does not seem to work, as keys are unique in JSON.

The following HCL:

consul = "127.0.0.1:8500"
token = "abcd1234"
retry = "10s"
max_stale = "10m"

auth {
  enabled = true
  username = "test"
  password = "test"
}

ssl {
  enabled = true
  verify = false
}

syslog {
  enabled = true
  facility = "LOCAL5"
}

prefix {
  source = "global@nyc1"
  destination = "local/nyc1"
}

prefix {
  source = "global@nyc2"
  destination = "local/nyc2"
}

Converts to this JSON:

{
    "auth": {
        "enabled": true,
        "password": "test",
        "username": "test"
    },
    "consul": "127.0.0.1:8500",
    "max_stale": "10m",
    "prefix": {
        "destination": "local/nyc2",
        "source": "global@nyc2"
    },
    "retry": "10s",
    "ssl": {
        "enabled": true,
        "verify": false
    },
    "syslog": {
        "enabled": true,
        "facility": "LOCAL5"
    },
    "token": "abcd1234"
}

In the example above, prefix configuration for global@nyc1 gets removed/overwritten since a duplicate key exists. A suggestion is to have prefix also accept an array of objects (i.e. hashes) if more than one prefix is specified. The same should apply for destination, so that it can be a string or an array of strings.

{
    "auth": {
        "enabled": true,
        "password": "test",
        "username": "test"
    },
    "consul": "127.0.0.1:8500",
    "max_stale": "10m",
    "prefix": [
        {
            "destination": "local/nyc1",
            "source": "global@nyc1"
        },
        {
            "destination": [
                "local/nyc2",
                "temp/nyc2"
            ],
            "source": "global@nyc2"
        }
    ],
    "retry": "10s",
    "ssl": {
        "enabled": true,
        "verify": false
    },
    "syslog": {
        "enabled": true,
        "facility": "LOCAL5"
    },
    "token": "abcd1234"
}

[calvn]

@sethvargo I was wondering if you have looked into this. I can start working on this and send you a PR if you are busy

[sethvargo]

@cleung2010 thank you for opening an issue.

The JSON you supplied is not a direct translation from the HCL. Top-level keys in HCL are actually keys to arrays of objects in JSON, i.e.:

consul = "127.0.0.1:8500"
token = "abcd1234"
retry = "10s"
max_stale = "10m"

auth {
  enabled = true
  username = "test"
  password = "test"
}

becomes:

{
  "auth": [
    {
      "enabled": true,
      "password": "test",
      "username": "test"
    }
  ]
}

Are you experiencing issues when you specify the prefixes as an array of objects, because that should definitely work?

[calvn]

So I am not using HCL at all as my config file. I want to use multi-prefix via JSON. The readme says I can do it like so in HCL:

prefix {
  source = "global@nyc1"
  destination = "local/nyc1"
}

prefix {
  source = "global@nyc2"
  destination = "local/nyc2"
}

But how can I do this in JSON since keys need to be unique (thus two prefix wouldn't work)?

[sethvargo]

@cleung2010 the same as my example for auth (and the example you provided yourself above):

{
  "prefix": [
    {
      "source": "...",
      "destination": "..."
    }
  ]
}

[calvn]

Oh nice, and I assume the same would apply for destination. I am going to go ahead and close the issue, thanks for clearing this up! It wouldn't hurt to provide a JSON example on the readme for people less familiar with explicit translations done by HCL :)

[sethvargo]

@cleung2010 no, destination must be a string. It's the same as the command line. If you want multiple destinations, you need to specify the prefix multiple times.

[calvn]

So

{
"prefix": [
        {
            "destination": "local/nyc1",
            "source": "global@nyc1"
        },
        {
            "destination": "local/nyc2",
            "source": "global@nyc2"
        },
        {
            "destination": "temp/nyc2",
            "source": "global@nyc2"
        }
    ]
}

Instead of

{
"prefix": [
        {
            "destination": "local/nyc1",
            "source": "global@nyc1"
        },
        {
            "destination": [
                "local/nyc2",
                "temp/nyc2"
            ],
            "source": "global@nyc2"
        }
    ]
}

[sethvargo]

Yes

[calvn]

Cool, thanks!