Add HTTP Provider

* feat: add HTTP provider implementation

* refactor: add SetDefaults and struct tag for the new file parser

* feat: add TLS configuration property

* refactor: rework HTTP provider implementation

* feat: provide config only once if fetched config is unchanged

* style: lint

* ui: add HTTP provider icon

* tests: simplify and fix integration test

* docs: add reference config for file

* docs: move http reference config for file

Co-authored-by: Daniel Tomcej <daniel.tomcej@gmail.com>

docs/content/providers/http.md

@@ -0,0 +1,189 @@
+# Traefik & HTTP
+
+Provide your [dynamic configuration](./overview.md) via an HTTP(s) endpoint and let Traefik do the rest!
+
+## Routing Configuration
+
+The HTTP provider uses the same configuration as the [File Provider](./file.md) in YAML or JSON format.
+
+## Provider Configuration
+
+### `endpoint`
+
+_Required_
+
+Defines the HTTP(s) endpoint to poll.
+
+```toml tab="File (TOML)"
+[providers.http]
+  endpoint = "http://127.0.0.1:9000/api"
+```
+
+```yaml tab="File (YAML)"
+providers:
+  http:
+    endpoint:
+      - "http://127.0.0.1:9000/api"
+```
+
+```bash tab="CLI"
+--providers.http.endpoint=http://127.0.0.1:9000/api
+```
+
+### `pollInterval`
+
+_Optional, Default="5s"_
+
+Defines the polling interval.
+
+```toml tab="File (TOML)"
+[providers.http]
+  pollInterval = "5s"
+```
+
+```yaml tab="File (YAML)"
+providers:
+  http:
+    pollInterval: "5s"
+```
+
+```bash tab="CLI"
+--providers.http.pollInterval=5s
+```
+
+### `pollTimeout`
+
+_Optional, Default="5s"_
+
+Defines the polling timeout when connecting to the configured endpoint.
+
+```toml tab="File (TOML)"
+[providers.http]
+  pollTimeout = "5s"
+```
+
+```yaml tab="File (YAML)"
+providers:
+  http:
+    pollTimeout: "5s"
+```
+
+```bash tab="CLI"
+--providers.http.pollTimeout=5s
+```
+
+### `tls`
+
+_Optional_
+
+#### `tls.ca`
+
+Certificate Authority used for the secured connection to the configured Endpoint.
+
+```toml tab="File (TOML)"
+[providers.http.tls]
+  ca = "path/to/ca.crt"
+```
+
+```yaml tab="File (YAML)"
+providers:
+  http:
+    tls:
+      ca: path/to/ca.crt
+```
+
+```bash tab="CLI"
+--providers.http.tls.ca=path/to/ca.crt
+```
+
+#### `tls.caOptional`
+
+Policy followed for the secured connection with TLS Client Authentication to the configured Endpoint.
+Requires `tls.ca` to be defined.
+
+- `true`: VerifyClientCertIfGiven
+- `false`: RequireAndVerifyClientCert
+- if `tls.ca` is undefined NoClientCert
+
+```toml tab="File (TOML)"
+[providers.http.tls]
+  caOptional = true
+```
+
+```yaml tab="File (YAML)"
+providers:
+  http:
+    tls:
+      caOptional: true
+```
+
+```bash tab="CLI"
+--providers.http.tls.caOptional=true
+```
+
+#### `tls.cert`
+
+Public certificate used for the secured connection to the configured Endpoint.
+
+```toml tab="File (TOML)"
+[providers.http.tls]
+  cert = "path/to/foo.cert"
+  key = "path/to/foo.key"
+```
+
+```yaml tab="File (YAML)"
+providers:
+  http:
+    tls:
+      cert: path/to/foo.cert
+      key: path/to/foo.key
+```
+
+```bash tab="CLI"
+--providers.http.tls.cert=path/to/foo.cert
+--providers.http.tls.key=path/to/foo.key
+```
+
+#### `tls.key`
+
+Private certificate used for the secured connection to the configured Endpoint.
+
+```toml tab="File (TOML)"
+[providers.http.tls]
+  cert = "path/to/foo.cert"
+  key = "path/to/foo.key"
+```
+
+```yaml tab="File (YAML)"
+providers:
+  http:
+    tls:
+      cert: path/to/foo.cert
+      key: path/to/foo.key
+```
+
+```bash tab="CLI"
+--providers.http.tls.cert=path/to/foo.cert
+--providers.http.tls.key=path/to/foo.key
+```
+
+#### `tls.insecureSkipVerify`
+
+If `insecureSkipVerify` is `true`, TLS connection to the configured Endpoint accepts any certificate presented by the 
+server and any host name in that certificate.
+
+```toml tab="File (TOML)"
+[providers.http.tls]
+  insecureSkipVerify = true
+```
+
+```yaml tab="File (YAML)"
+providers:
+  http:
+    tls:
+      insecureSkipVerify: true
+```
+
+```bash tab="CLI"
+--providers.http.tls.insecureSkipVerify=true
+```

docs/content/providers/overview.md

@@ -35,9 +35,10 @@ Below is the list of the currently supported providers in Traefik.
 | [Rancher](./rancher.md)               | Orchestrator | Label                      |
 | [File](./file.md)                     | Manual       | TOML/YAML format           |
 | [Consul](./consul.md)                 | KV           | KV                         |
-| [etcd](./etcd.md)                     | KV           | KV                         |
+| [Etcd](./etcd.md)                     | KV           | KV                         |
 | [Redis](./redis.md)                   | KV           | KV                         |
 | [ZooKeeper](./zookeeper.md)           | KV           | KV                         |
+| [HTTP](./http.md)                     | Manual       | JSON format                |
 
 !!! info "More Providers"
 

docs/content/reference/static-configuration/cli-ref.md

@@ -486,6 +486,33 @@ Load dynamic configuration from a file.
 `--providers.file.watch`:  
 Watch provider. (Default: ```true```)
 
+`--providers.http`:  
+Enable HTTP backend with default settings. (Default: ```false```)
+
+`--providers.http.endpoint`:  
+Load configuration from this endpoint.
+
+`--providers.http.pollinterval`:  
+Polling interval for endpoint. (Default: ```5```)
+
+`--providers.http.polltimeout`:  
+Polling timeout for endpoint. (Default: ```5```)
+
+`--providers.http.tls.ca`:  
+TLS CA
+
+`--providers.http.tls.caoptional`:  
+TLS CA.Optional (Default: ```false```)
+
+`--providers.http.tls.cert`:  
+TLS cert
+
+`--providers.http.tls.insecureskipverify`:  
+TLS insecure skip verify (Default: ```false```)
+
+`--providers.http.tls.key`:  
+TLS key
+
 `--providers.kubernetescrd`:  
 Enable Kubernetes backend with default settings. (Default: ```false```)
 

docs/content/reference/static-configuration/env-ref.md

@@ -486,6 +486,33 @@ Load dynamic configuration from a file.
 `TRAEFIK_PROVIDERS_FILE_WATCH`:  
 Watch provider. (Default: ```true```)
 
+`TRAEFIK_PROVIDERS_HTTP`:  
+Enable HTTP backend with default settings. (Default: ```false```)
+
+`TRAEFIK_PROVIDERS_HTTP_ENDPOINT`:  
+Load configuration from this endpoint.
+
+`TRAEFIK_PROVIDERS_HTTP_POLLINTERVAL`:  
+Polling interval for endpoint. (Default: ```5```)
+
+`TRAEFIK_PROVIDERS_HTTP_POLLTIMEOUT`:  
+Polling timeout for endpoint. (Default: ```5```)
+
+`TRAEFIK_PROVIDERS_HTTP_TLS_CA`:  
+TLS CA
+
+`TRAEFIK_PROVIDERS_HTTP_TLS_CAOPTIONAL`:  
+TLS CA.Optional (Default: ```false```)
+
+`TRAEFIK_PROVIDERS_HTTP_TLS_CERT`:  
+TLS cert
+
+`TRAEFIK_PROVIDERS_HTTP_TLS_INSECURESKIPVERIFY`:  
+TLS insecure skip verify (Default: ```false```)
+
+`TRAEFIK_PROVIDERS_HTTP_TLS_KEY`:  
+TLS key
+
 `TRAEFIK_PROVIDERS_KUBERNETESCRD`:  
 Enable Kubernetes backend with default settings. (Default: ```false```)
 

docs/content/reference/static-configuration/file.toml

@@ -205,6 +205,16 @@
       cert = "foobar"
       key = "foobar"
       insecureSkipVerify = true
+  [providers.http]
+    endpoint = "foobar"
+    pollInterval = 42
+    pollTimeout = 42
+    [providers.http.tls]
+      ca = "foobar"
+      caOptional = true
+      cert = "foobar"
+      key = "foobar"
+      insecureSkipVerify = true
 
 [api]
   insecure = true

docs/content/reference/static-configuration/file.yaml

@@ -225,6 +225,16 @@ providers:
       cert: foobar
       key: foobar
       insecureSkipVerify: true
+  http:
+    endpoint: foobar
+    pollInterval: 42
+    pollTimeout: 42
+    tls:
+      ca: foobar
+      caOptional: true
+      cert: foobar
+      key: foobar
+      insecureSkipVerify: true
 api:
   insecure: true
   dashboard: true

docs/mkdocs.yml

@@ -85,6 +85,7 @@ nav:
       - 'Etcd': 'providers/etcd.md'
       - 'ZooKeeper': 'providers/zookeeper.md'
       - 'Redis': 'providers/redis.md'
+      - 'HTTP': 'providers/http.md'
   - 'Routing & Load Balancing':
       - 'Overview': 'routing/overview.md'
       - 'EntryPoints': 'routing/entrypoints.md'

integration/fixtures/http/simple.toml

@@ -0,0 +1,20 @@
+[global]
+  checkNewVersion = false
+  sendAnonymousUsage = false
+
+[log]
+  level = "DEBUG"
+
+[entryPoints]
+  [entryPoints.web]
+    address = ":8000"
+  [entryPoints.traefik]
+    address = ":9090"
+
+[api]
+  insecure = true
+
+[providers]
+  [providers.http]
+    endpoint = "http://127.0.0.1:9000"
+    pollInterval = "100ms"