_ in alertmanager.url gives 'ERRO[0000] invalid Alertmanager URL'

[rmeleromira]

What did you do?
Try to start prometheus with a - in alertmanager.url hostname.

What did you expect to see?
Prometheus startup

What did you see instead? Under which circumstances?
Prometheus fails to start with ERRO[0000] invalid Alertmanager URL

Environment

• System information:

root@aio1-prometheus-container-1ce4332a:~# uname -srm
Linux 4.4.0-77-generic x86_64

• Prometheus version:

root@aio1-prometheus-container-1ce4332a:~# prometheus -version
prometheus, version 1.6.2 (branch: master, revision: b38e977fd8cc2a0d13f47e7f0e17b82d1a908a9a)
  build user:       root@c99d9d650cf4
  build date:       20170511-12:59:13
  go version:       go1.8.1

• Prometheus configuration file:

insert configuration here

• Alertmanager configuration file:

---
global:
  scrape_interval:     15s
  evaluation_interval: 15s
rule_files:
  - /etc/prometheus/alarm.rules
scrape_configs:
  - job_name: 'prometheus'
    scrape_interval: 5s
    static_configs:
      - targets: ['aio1_prometheus_container-1ce4332a:9090']
      - targets: ['aio1_prometheus_container-80fbb462:9090']
      - targets: ['aio1_prometheus_container-c162b818:9090']
  - job_name: 'node'
    static_configs:
      - targets: ['aio1:9100']
      - targets: ['aio1_cinder_scheduler_container-dd28c03d:9100']
      - targets: ['aio1_swift_proxy_container-93e91eb6:9100']
      - targets: ['aio1_neutron_server_container-ca547b0f:9100']
      - targets: ['aio1_neutron_agents_container-a7bc08a8:9100']
      - targets: ['aio1_nova_api_os_compute_container-50c11b60:9100']
      - targets: ['aio1_galera_container-d62f36fb:9100']
      - targets: ['aio1_memcached_container-00d6d829:9100']
      - targets: ['aio1_rabbit_mq_container-1fc0866b:9100']
      - targets: ['aio1_designate_container-16f22f3b:9100']
      - targets: ['aio1_nova_console_container-4f609da5:9100']
      - targets: ['aio1_cinder_api_container-67a9eee0:9100']
      - targets: ['aio1_blackbox_container-1b3e1133:9100']
      - targets: ['aio1_blackbox_container-ef36ca7d:9100']
      - targets: ['aio1_blackbox_container-93936ea5:9100']
      - targets: ['aio1_nova_conductor_container-19089b7d:9100']
      - targets: ['aio1_horizon_container-1b6daf91:9100']
      - targets: ['aio1_keystone_container-0be3c486:9100']
      - targets: ['aio1_grafana_container-00075ed0:9100']
      - targets: ['aio1_grafana_container-8ec960f9:9100']
      - targets: ['aio1_grafana_container-fe2142ca:9100']
      - targets: ['aio1_glance_container-b0220d11:9100']
      - targets: ['aio1_nova_api_metadata_container-2bb03f2c:9100']
      - targets: ['aio1_prometheus_container-c162b818:9100']
      - targets: ['aio1_prometheus_container-1ce4332a:9100']
      - targets: ['aio1_prometheus_container-80fbb462:9100']
      - targets: ['aio1_nova_scheduler_container-b51f7087:9100']
      - targets: ['aio1_rsyslog_container-973c250b:9100']
      - targets: ['aio1_nova_api_placement_container-1c03ea59:9100']
      - targets: ['aio1_heat_apis_container-7d40935f:9100']
      - targets: ['aio1_heat_engine_container-ee0e94fc:9100']
      - targets: ['aio1_alertmanager_container-10c18cc4:9100']
      - targets: ['aio1_alertmanager_container-63376c45:9100']
      - targets: ['aio1_alertmanager_container-639c8482:9100']
  - job_name: 'haproxy'
    static_configs:
      - targets: ['aio1:9101']
  - job_name: 'mysql'
    static_configs:
      - targets: ['aio1_galera_container-d62f36fb:9104']
  - job_name: 'alertmanager'
    static_configs:
      - targets: ['aio1_alertmanager_container-63376c45:9093']
      - targets: ['aio1_alertmanager_container-639c8482:9093']
      - targets: ['aio1_alertmanager_container-10c18cc4:9093']

• Logs:

root@aio1-prometheus-container-1ce4332a:~# /usr/bin/prometheus -config.file=/etc/prometheus/prometheus.yml -storage.local.path=/var/lib/prometheus/data -alertmanager.url http://aio1_alertmanager_container-63376c45:9093,http://aio1_alertmanager_container-639c8482:9093,http://aio1_alertmanager_container-10c18cc4:9093 -web.console.templates=/etc/prometheus/consoles -web.console.libraries=/etc/prometheus/console_libraries
ERRO[0000] invalid Alertmanager URL: http://aio1_alertmanager_container-63376c45:9093  source=main.go:75

After changing _ in alertmanager.url to -

root@aio1-prometheus-container-1ce4332a:~# /usr/bin/prometheus -config.file=/etc/prometheus/prometheus.yml -storage.local.path=/var/lib/prometheus/data -alertmanager.url http://aio1-alertmanager-container-63376c45:9093,http://aio1-alertmanager-container-639c8482:9093,http://aio1-alertmanager-container-10c18cc4:9093 -web.console.templates=/etc/prometheus/consoles -web.console.libraries=/etc/prometheus/console_libraries
INFO[0000] Starting prometheus (version=1.6.2, branch=master, revision=b38e977fd8cc2a0d13f47e7f0e17b82d1a908a9a)  source=main.go:88
INFO[0000] Build context (go=go1.8.1, user=root@c99d9d650cf4, date=20170511-12:59:13)  source=main.go:89
INFO[0000] Loading configuration file /etc/prometheus/prometheus.yml  source=main.go:251
INFO[0000] Loading series map and head chunks...         source=storage.go:421
INFO[0001] 40498 series loaded.                          source=storage.go:432
INFO[0001] Starting target manager...                    source=targetmanager.go:61
INFO[0001] Listening on :9090                            source=web.go:259

RFC 952 says that:

1. A "name" (Net, Host, Gateway, or Domain name) is a text string up
   to 24 characters drawn from the alphabet (A-Z), digits (0-9), minus
   sign (-), and period (.)

But the rule apparently isn't strictly followed by DNS servers and names in an /etc/hosts file can contain underscores.

Just pointing out that there's a little inconsistency in hostname handling, and we possibly need a better error for special characters? Something like:

Invalid characters, only A-Z,0-9 and . allowed

[brian-brazil]

The library we use for checking urls probably needs updating again.

[gouthamve]

We use govalidator to do it here: https://github.com/prometheus/prometheus/blob/master/cmd/prometheus/config.go#L354-L369

I verified in the latest master of govalidator that the URL specified above: aio1_alertmanager_container-63376c45:9093 is returning false.

Opened asaskevich/govalidator#212 upstream but not sure it will be fixed there soon. One thing we could do is just pull out the IsURL logic out into prometheus itself and add support for _ as IsURL is the only thing we are using.

[grobie]

I'm against adding support for underscores, it's clearly forbidden by the RFC 952. That there is no validation for entries in /etc/hosts does not change that.

[grobie]

We relaxed the validation criteria after all.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.