Option to use YAML instead of TOML for configuration

[gaui]

Do you want to request a feature or report a bug?

feature

Description

YAML is becoming the industry standard configuration format. Docker uses it for Compose/Stack, Kubernetes, GitLab CI, Travis CI, Drone CI, Google App Engine, etc. Wouldn't it be ok to support both configuration formats, for those who prefer YAML instead of TOML.

Let's take the following example, a configuration in both TOML and YAML from here. In my opinion YAML is more readable, better structured (with indent as context) and with less noise (less repetitions of parent elements).

TOML

[entryPoints]
  [entryPoints.http]
    address = ":80"
    whitelistSourceRange = ["10.42.0.0/16", "152.89.1.33/32", "afed:be44::/16"]
    compress = true

    [entryPoints.http.tls]
      minVersion = "VersionTLS12"
      cipherSuites = [
        "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
        "TLS_RSA_WITH_AES_256_GCM_SHA384"
       ]
      [[entryPoints.http.tls.certificates]]
        certFile = "path/to/my.cert"
        keyFile = "path/to/my.key"
      [[entryPoints.http.tls.certificates]]
        certFile = "path/to/other.cert"
        keyFile = "path/to/other.key"
      # ...
      [entryPoints.http.tls.clientCA]
        files = ["path/to/ca1.crt", "path/to/ca2.crt"]
        optional = false

    [entryPoints.http.redirect]
      entryPoint = "https"
      regex = "^http://localhost/(.*)"
      replacement = "http://mydomain/$1"

    [entryPoints.http.auth]
      headerField = "X-WebAuth-User"
      [entryPoints.http.auth.basic]
        users = [
          "test:$apr1$H6uskkkW$IgXLP6ewTrSuBkTrqE8wj/",
          "test2:$apr1$d9hr9HBB$4HxwgUir3HP4EsggP/QNo0",
        ]
        usersFile = "/path/to/.htpasswd"
      [entryPoints.http.auth.digest]
        users = [
          "test:traefik:a2688e031edb4be6a3797f3882655c05",
          "test2:traefik:518845800f9e2bfb1f1f740ec24f074e",
        ]
        usersFile = "/path/to/.htdigest"
      [entryPoints.http.auth.forward]
        address = "https://authserver.com/auth"
        trustForwardHeader = true
        [entryPoints.http.auth.forward.tls]
          ca =  [ "path/to/local.crt"]
          caOptional = true
          cert = "path/to/foo.cert"
          key = "path/to/foo.key"
          insecureSkipVerify = true

    [entryPoints.http.proxyProtocol]
      insecure = true
      trustedIPs = ["10.10.10.1", "10.10.10.2"]

    [entryPoints.http.forwardedHeaders]
      trustedIPs = ["10.10.10.1", "10.10.10.2"]

YAML

entryPoints:
  http:
    address: ':80'
    whitelistSourceRange:
      - 10.42.0.0/16
      - 152.89.1.33/32
      - 'afed:be44::/16'
    compress: true
    tls:
      minVersion: VersionTLS12
      cipherSuites:
        - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
        - TLS_RSA_WITH_AES_256_GCM_SHA384
      certificates:
        - certFile: path/to/my.cert
          keyFile: path/to/my.key
        - certFile: path/to/other.cert
          keyFile: path/to/other.key
      clientCA:
        files:
          - path/to/ca1.crt
          - path/to/ca2.crt
        optional: false
    redirect:
      entryPoint: https
      regex: '^http://localhost/(.*)'
      replacement: 'http://mydomain/$1'
    auth:
      headerField: X-WebAuth-User
      basic:
        users:
          - 'test:$apr1$H6uskkkW$IgXLP6ewTrSuBkTrqE8wj/'
          - 'test2:$apr1$d9hr9HBB$4HxwgUir3HP4EsggP/QNo0'
        usersFile: /path/to/.htpasswd
      digest:
        users:
          - 'test:traefik:a2688e031edb4be6a3797f3882655c05'
          - 'test2:traefik:518845800f9e2bfb1f1f740ec24f074e'
        usersFile: /path/to/.htdigest
      forward:
        address: 'https://authserver.com/auth'
        trustForwardHeader: true
        tls:
          ca:
            - path/to/local.crt
          caOptional: true
          cert: path/to/foo.cert
          key: path/to/foo.key
          insecureSkipVerify: true
    proxyProtocol:
      insecure: true
      trustedIPs:
        - 10.10.10.1
        - 10.10.10.2
    forwardedHeaders:
      trustedIPs:
        - 10.10.10.1
        - 10.10.10.2

Output of traefik version: (What version of Traefik are you using?)

Version:      v1.5.3
Codename:     cancoillotte
Go version:   go1.9.4
Built:        2018-02-27_02:47:04PM
OS/Arch:      linux/amd64

[kinghuang]

Switching to YAML would also help reduce repetition when creating multiple frontends to handle differing routes (#3026). There's no equivalent to YAML anchors in TOML, which leads to a lot of repeated configuration.

[timoreimann]

I see two general issues with YAML:

1. For anything but simple (read: not deeply nested) configurations, it's very hard to maintain as a human due to its heavy reliance on indentation. Everyone who has worked with Kubernetes manifests knows this pain first hand. TOML traded in indentation for ordering needs, which to me makes it easier to manage.
2. The format is pretty complex. TOML, on the other hand, only knows a handful of types.

In a nutshell: YAML is nicer to read, but TOML is easier to write.

From a technical perspective, I guess I'm okay with considering support if it's cheap to maintain. (That is, we can automatically and easily transform between the formats.)

From a end-user support perspective, I am somewhat concerned that YAML will open up a whole new dimension of headaches ("why is my configuration not working?" -- "you are missing an indent in line 43").

[kinghuang]

While TOML doesn't require indentation, I find a lot of TOML is indented anyways, because it can be hard to comprehend without it. All of Traefik's documentation shows indented TOML, for example.

Personally, I find TOML harder to write than YAML. YAML is definitely easier to read.

[gaui]

This is debatable, but the industry seems to have adopted it - so why shouldn't it be supported in Træfik?

@timoreimann Actually, I am a dev and I personally find it easier to read and write. I use an IDE with great YAML support. But I get your concern for the indentation problems on user-end but I think you should validate the config before using it - whether it's YAML or TOML.

[timoreimann]

@kinghuang

While TOML doesn't require indentation, I find a lot of TOML is indented anyways, because it can be hard to comprehend without it. All of Traefik's documentation shows indented TOML, for example.

Sure, we do that to improve readability, which indentation definitely helps with. Unlike YAML, however, I find that TOML makes it fairly easy to indent correctly as the explicitly required table specifiers provide the necessary scaffolding (at the price of added verbosity, no question). Furthermore, the cost of misaligning is just a minor cosmetic issue in TOML, whereas in YAML such a mistake can turn into an wrong configuration.

@gaui

Actually, I am a dev and I personally find it easier to read and write. I use an IDE with great YAML support. But I get your concern for the indentation problems on user-end but I think you should validate the config before using it - whether it's YAML or TOML.

I also use a plugin to assist with writing YAML. In fact, it's the only way for me to effectively write complex YAML files. I consider that to be a sign that the syntax and format is too complicated to be handled with simple editors. With TOML, I don't feel that pain.

Validating configurations in-code is a non-trivial problem. Strictly speaking, you'd have to treat every unknown configuration option as an error and report it to the user. Traefik doesn't do this at the moment -- instead, it tries hard to bootstrap into a somehow working mode. Whether that's desirable or not is debatable. But even if we had some kind of strict mode, I believe a broken YAML configuration is still harder to debug and get right, again based on my experience with working on Kubernetes manifests.

That said, I do understand the value for supporting a well-known format like YAML: a larger group is familiar with it, lots of tools exist, etc. TOML certainly isn't free of issues either -- the fact that any non-table option must go before the first table option has been a continuous source of end-user support questions ever since. So I'm not super opposed to the idea, I just think that it comes with a price we shouldn't underestimate when we make a decision.

[Johannestegner]

Maybe abstract the settings loader in a way that we could use either (or other types) through adapters? YAML, Toml, json and XML are all quite easy to convert to the same data output (not sure about toml, due to not being that used to it yet), and it's often more of a question of taste.
This would also make it easier for the devs to just implement the loader for toml and let us create our own loaders to plug in to the engine! ;)

[gaui]

One awesome pro with YAML: It has so called anchors which makes your config DRY.

[ldez]

Have the ability to repeat something can be useful for the dynamic configuration but it's not really useful for the static configuration.

With this PR #2991 (v1.6) you will be able to use Go Template syntax (functions, loops, variables, ...) in the TOML configuration (for the dynamic configuration).

[trondhindenes]

So much this. I don't understand how TOML has become as popular as it has. I'd love to see Traefik support both toml and yaml, these "dsl language arguments" never go anywhere constructive, there's always pros and cons.
For us as an Ansible/Kubernetes shop, we live in yaml. We have linters so we can test yaml configuration in CI for instance. But most importantly, we simply like and prefer yaml over toml.

[timoreimann]

@containous/traefik thoughts?

At least by looking at the number of thumbs up on the issue description, there seems to be a desire to support YAML.

[mrmachine]

I was excited to find out about traefik as a replacement for the deprecated dockercloud-haproxy, but the config examples in TOML immediately put me off big time. I personally find YAML is infinitely easier to read and write, even without any special linters. Although I'm a Python developer, so I'm used to indentation being important (and I think that's a good thing). And yes, everything is YAML these days. There's little to be gained by going against the grain on this one.

[slayer]

Obviously TOML is harder to read and understand. [] and [[ ]] adds pits and mounds for eyes.

[DanielJoyce]

go templates are overkill for something that yaml anchors fix trivially. Its an even more complicated solution than the yaml one. It's also not trivial to merge/compose some configuration in it. Those templates are even uglier and harder to use than yaml.

Traefik config is repetitive and ugly. Toml is intended for simple configurations like old school windows ini files, not emulating tree structures with dot seperated names. Also [[]] is ugly to look at and comprehend.

[DanielJoyce]

Put your confid in yaml, then convert with remarshal, a python util...

[ndeloof]

relying on https://github.com/spf13/viper would allow to support both toml and yaml, + json, hcl, and even java properties (!)

[thewilli]

Any update on this? I hoped to find some additional labels / milestones associated with this issue when YAML support was mentioned in the v2 blog post:

[bbinet]

Can we close this issue now that #4935 is merged?

[ldez]

In fact Traefik don't really support YAML: current (alpha7) we only added YAML support for the static configuration, the dynamic configuration still use TOML only.

This is why we have not communicated on this subject, nor added documentation, nor closed the issue.

[traefiker]

Closed by #5024.