http2Client.notifyError

[Luncher]

What did you do?

GETTING STARTED

What did you expect to see?

configuring prometheus to monitor itself

What did you see instead? Under which circumstances?

2017/11/21 11:33:04 transport: http2Client.notifyError got notified thatthe client transport was broken EOF.
2017/11/21 11:33:04 transport: http2Client.notifyError got notified thatthe client transport was broken EOF.
2017/11/21 11:33:04 transport: http2Client.notifyError got notified thatthe client transport was broken EOF.
2017/11/21 11:33:05 transport: http2Client.notifyError got notified thatthe client transport was broken EOF.
2017/11/21 11:33:05 transport: http2Client.notifyError got notified thatthe client transport was broken EOF.

Environment

Mac OsX

• System information:

Darwin 16.7.0 x86_64

• Prometheus version:

  prometheus, version 2.0.0 (branch: HEAD, revision: 0a74f98)
  build user: root@615b82cb36b6
  build date: 20171108-07:15:39
  go version: go1.9.2

• Alertmanager version:

  insert output of alertmanager --version here (if relevant to the issue)

• Prometheus configuration file:

# my global config
global:
  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

# Alertmanager configuration
alerting:
  alertmanagers:
  - static_configs:
    - targets:
      # - alertmanager:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  # - "first_rules.yml"
  # - "second_rules.yml"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'prometheus'

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
      - targets: ['localhost:9090']

• Alertmanager configuration file:

insert configuration here (if relevant to the issue)

• Logs:

level=info ts=2017-11-21T03:33:04.480720551Z caller=main.go:215 msg="Starting Prometheus" version="(version=2.0.0, branch=HEAD, revision=0a74f9862
8a0463dddc90528220c94de5032d1a0)"level=info ts=2017-11-21T03:33:04.480796657Z caller=main.go:216 build_con
text="(go=go1.9.2, user=root@615b82cb36b6, date=20171108-07:15:39)"level=info ts=2017-11-21T03:33:04.480827863Z caller=main.go:217 host_deta
ils=(darwin)level=info ts=2017-11-21T03:33:04.482914403Z caller=main.go:314 msg="Star
ting TSDB"level=info ts=2017-11-21T03:33:04.482961351Z caller=web.go:380 component=
web msg="Start listening for connections" address=0.0.0.0:9090level=info ts=2017-11-21T03:33:04.482954596Z caller=targetmanager.go:71 c
omponent="target manager" msg="Starting target manager..."
level=info ts=2017-11-21T03:33:04.494202679Z caller=main.go:326 msg="TSDB started"
level=info ts=2017-11-21T03:33:04.494246169Z caller=main.go:394 msg="Loading configuration file" filename=prometheus.yml
level=info ts=2017-11-21T03:33:04.496043133Z caller=main.go:371 msg="Server is ready to receive requests."
2017/11/21 11:33:04 transport: http2Client.notifyError got notified thatthe client transport was broken EOF.
2017/11/21 11:33:04 transport: http2Client.notifyError got notified thatthe client transport was broken EOF.
2017/11/21 11:33:04 transport: http2Client.notifyError got notified thatthe client transport was broken EOF.
2017/11/21 11:33:05 transport: http2Client.notifyError got notified thatthe client transport was broken EOF.
2017/11/21 11:33:05 transport: http2Client.notifyError got notified thatthe client transport was broken EOF.

[brian-brazil]

Do you have any weird network setup? Such as say a firewall in the way of Prometheus talking to itself?

[Luncher]

I can link the network，I'm starting with docker now ==

[fate-lovely]

@Luncher check http_proxy and https_proxy environment variables, I met the same problem and it turned out to be these variables.

[brian-brazil]

Sounds like a network issue on the user's end.

[Luncher]

@fate-lovely thanks, you are right!

[nani2say]

@Luncher Could you please let me know what was the problem with http_proxy setting?
I have them in place but I still get the same error.
Thanks!

[acheaito]

@nani2say The problem, for me, was the presence of the http_proxy environment variable. To get around the issue, I set no_proxy=0.0.0.0,127.0.0.1,localhost

[nani2say]

@acheaito - Thanks for letting me know. Did you replace http_proxy with no_proxy ?
I am asking because I don't want to loose the benefit of http_proxy in my environment. may be a stupid question but I would appreciate your response.
Thanks!

[acheaito]

@nani2say - no_proxy works together with http_proxy. It's a mechanism for specifying hosts that the system should not use the proxy for. Connections to all other hosts still goes through the proxy.

[nani2say]

@acheaito that makes sense. Thank you!

[nikhilvgs]

no_proxy didnt work for me. Worked for me by switching to 127.0.0.1 as the web listen address. Suspecting something to do with ipv6 as well - the error is pretty arbitrary though.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.