Where to get exit logs for prometheus

[YanjieGao]

What did you do?
Once i meet prometheus crash for long running then hope to stress test prometheus to get the root cause of it.

First do stress test on prometheus and hope to get the exit reason when scrape very large scape logs.
i tried log level error, info, debug, warn.
I deployed use docker & config scrape 240000 samples per seconds.
sudo docker run -d -m 3g -p 9090:9090 -v /home/core/yanjie/config:/etc/prometheus prom/prometheus:v2.1.0 --config.file=/etc/prometheus/prometheus.yml --log.level=error

What did you expect to see?
Hope to see the exit logs
What did you see instead? Under which circumstances?
no stderr or error level logs tose
Environment
version:v2.1

• System information: ubuntu

  insert output of uname -srm here

• Prometheus version: 2.1

  insert output of prometheus --version here

• Alertmanager version:

  insert output of alertmanager --version here (if relevant to the issue)

• Prometheus configuration file:

insert configuration here

scrape_configs:
- job_name: "prometheus"
scrape_interval: "15s"
static_configs:
- targets:
- ip:9090
- job_name: 'node_exporter'
scrape_interval: "1000ms"
scrape_timeout: "1s"
kubernetes_sd_configs:
- api_server: 'http://ip:8080/'
role: node
relabel_configs:
- source_labels: [address]
regex: '(.):(.)'
replacement: '${1}:9100'
target_label: address
- job_name: 'cadvisor'
scrape_interval: "1000ms"
scrape_timeout: "1s"
kubernetes_sd_configs:
- api_server: 'http://ip:8080/'
role: node
relabel_configs:
- source_labels: [address]
regex: '(.*):10250'
replacement: '${1}:8089'
target_label: address

• Alertmanager configuration file:

insert configuration here (if relevant to the issue)

• Logs:

insert Prometheus and Alertmanager logs relevant to the issue here

only get logs as follows and no other logs:
--log.level=error
No logs
--log.level=debug
no exit reason only

--log.level=info

[gouthamve]

It's highly likely that the prometheus was OOM killed. I see this as a usage question.
It makes more sense to ask questions like this on the prometheus-users mailing list rather than in a GitHub issue. On the mailing list, more people are available to potentially respond to your question, and the whole community can benefit from the answers provided.

If you think this is a bug, please feel free to re-open.

[YanjieGao]

Thanks submit a question at mail list https://groups.google.com/forum/#!topic/prometheus-users/vqhSTXPC8b8

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.