Prometheus 2.0 fails to start up after couple of restarts #3191

JorritSalverda · 2017-09-19T09:02:15Z

What did you do?

We run 2 Prometheus instances on Google Container Engine using preemptibles. This means the instances are relocated at least every 24 hours (the max lifetime of a preemptible).

With version 1.7.1 this caused issues because the graceful shutdown sometimes took too long and didn't fully finish. After that startup took longer than the initialDelaySeconds of 1200s causing Kubernetes to restart Prometheus over and over and making Prometheus unavailable. Deleting the instances and their data was the only simple way to get things up and running again.

With the much faster storage system in Prometheus 2 I had high hopes that this would no longer happen, but we seem to experience another cause of failure. After some restarts Prometheus does start and shows the following 4 log lines, but it fails to respond to the liveness check using the /status endpoint.

time="2017-09-19T07:31:22Z" level=info msg="Starting prometheus (version=2.0.0-beta.3, branch=HEAD, revision=066783b3991dd64729325fc4f880dfffb484a2c2)" source="main.go:210"
time="2017-09-19T07:31:22Z" level=info msg="Build context (go=go1.8.3, user=root@0cbc320660dc, date=20170912-10:17:45)" source="main.go:211"
time="2017-09-19T07:31:22Z" level=info msg="Host details (Linux 4.4.52+ #1 SMP Thu Jul 13 11:47:20 PDT 2017 x86_64 prometheus-1 (none))" source="main.go:212"
time="2017-09-19T07:31:22Z" level=info msg="Starting tsdb" source="main.go:224"

What did you expect to see?

Prometheus to cope well with restarts and being able to gracefully shutdown within the max 30 seconds a preemptible shutdown allows.

What did you see instead? Under which circumstances?

The restarts / relocations work fine most of the time, but every now and then lead to failure. This usually seems to happen for both instances at pretty much the same time (or at least same day).

Environment

System information:

Linux 4.4.64+ x86_64

Prometheus version:

prometheus, version 2.0.0-beta.3 (branch: HEAD, revision: 066783b3991dd64729325fc4f880dfffb484a2c2)
  build user:       root@0cbc320660dc
  build date:       20170912-10:17:45
  go version:       go1.8.3

Prometheus configuration file:

global:
  scrape_interval: 10s
  scrape_timeout: 10s
  evaluation_interval: 10s
alerting:
  alertmanagers:
  - static_configs:
    - targets:
      - alertmanager-0.alertmanager-headless:9093
      - alertmanager-1.alertmanager-headless:9093
      - alertmanager-2.alertmanager-headless:9093
    scheme: http
    timeout: 10s
rule_files:
- /prometheus-rules/alert.yaml
- /prometheus-rules/aggregation.yaml
scrape_configs:
- job_name: kubernetes-apiservers
  scrape_interval: 10s
  scrape_timeout: 10s
  metrics_path: /metrics
  scheme: https
  kubernetes_sd_configs:
  - api_server: null
    role: endpoints
    namespaces:
      names: []
  bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
  tls_config:
    ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
    insecure_skip_verify: false
  relabel_configs:
  - source_labels: [__meta_kubernetes_namespace, __meta_kubernetes_service_name, __meta_kubernetes_endpoint_port_name]
    separator: ;
    regex: default;kubernetes;https
    replacement: $1
    action: keep
- job_name: kubernetes-nodes
  scrape_interval: 10s
  scrape_timeout: 10s
  metrics_path: /metrics
  scheme: https
  kubernetes_sd_configs:
  - api_server: null
    role: node
    namespaces:
      names: []
  bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
  tls_config:
    ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
    insecure_skip_verify: false
  relabel_configs:
  - separator: ;
    regex: __meta_kubernetes_node_label_(.+)
    replacement: $1
    action: labelmap
  - separator: ;
    regex: (.*)
    target_label: __address__
    replacement: kubernetes.default.svc:443
    action: replace
  - source_labels: [__meta_kubernetes_node_name]
    separator: ;
    regex: (.+)
    target_label: __metrics_path__
    replacement: /api/v1/nodes/${1}/proxy/metrics
    action: replace
- job_name: kubernetes-cadvisor
  scrape_interval: 10s
  scrape_timeout: 10s
  metrics_path: /metrics
  scheme: https
  kubernetes_sd_configs:
  - api_server: null
    role: node
    namespaces:
      names: []
  bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
  tls_config:
    ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
    insecure_skip_verify: false
  relabel_configs:
  - separator: ;
    regex: __meta_kubernetes_node_label_(.+)
    replacement: $1
    action: labelmap
  - separator: ;
    regex: (.*)
    target_label: __address__
    replacement: kubernetes.default.svc:443
    action: replace
  - source_labels: [__meta_kubernetes_node_name]
    separator: ;
    regex: (.+)
    target_label: __metrics_path__
    replacement: /api/v1/nodes/${1}:4194/proxy/metrics
    action: replace
- job_name: kubernetes-service-endpoints
  scrape_interval: 10s
  scrape_timeout: 10s
  metrics_path: /metrics
  scheme: http
  kubernetes_sd_configs:
  - api_server: null
    role: endpoints
    namespaces:
      names: []
  relabel_configs:
  - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scrape]
    separator: ;
    regex: "true"
    replacement: $1
    action: keep
  - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scheme]
    separator: ;
    regex: (https?)
    target_label: __scheme__
    replacement: $1
    action: replace
  - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_path]
    separator: ;
    regex: (.+)
    target_label: __metrics_path__
    replacement: $1
    action: replace
  - source_labels: [__address__, __meta_kubernetes_service_annotation_prometheus_io_port]
    separator: ;
    regex: ([^:;]+);(\d+)
    target_label: __address__
    replacement: ${1}:${2}
    action: replace
  - source_labels: [__address__, __meta_kubernetes_service_annotation_prometheus_io_port]
    separator: ;
    regex: ([^:;]+):(\d+);(\d+)
    target_label: __address__
    replacement: ${1}:${3}
    action: replace
  - separator: ;
    regex: __meta_kubernetes_service_label_(.+)
    replacement: $1
    action: labelmap
  - source_labels: [__meta_kubernetes_namespace]
    separator: ;
    regex: (.*)
    target_label: kubernetes_namespace
    replacement: $1
    action: replace
  - source_labels: [__meta_kubernetes_service_name]
    separator: ;
    regex: (.*)
    target_label: kubernetes_name
    replacement: $1
    action: replace
- job_name: kubernetes-services
  params:
    module:
    - http_2xx
  scrape_interval: 10s
  scrape_timeout: 10s
  metrics_path: /probe
  scheme: http
  kubernetes_sd_configs:
  - api_server: null
    role: service
    namespaces:
      names: []
  relabel_configs:
  - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_probe]
    separator: ;
    regex: "true"
    replacement: $1
    action: keep
  - source_labels: [__address__]
    separator: ;
    regex: (.*?):(:80|:443)
    replacement: $1
    action: keep
  - source_labels: [__address__]
    separator: ;
    regex: (.*?):80
    target_label: __param_target
    replacement: http://${1}
    action: replace
  - source_labels: [__address__]
    separator: ;
    regex: (.*?):443
    target_label: __param_target
    replacement: https://${1}
    action: replace
  - source_labels: [__param_target, __meta_kubernetes_service_annotation_prometheus_io_probe_path]
    separator: ;
    regex: (.*?);(.*?)
    target_label: __param_target
    replacement: ${1}${2}
    action: replace
  - separator: ;
    regex: (.*)
    target_label: __address__
    replacement: blackbox-exporter
    action: replace
  - source_labels: [__param_target]
    separator: ;
    regex: (.*)
    target_label: instance
    replacement: $1
    action: replace
  - separator: ;
    regex: __meta_kubernetes_service_label_(.+)
    replacement: $1
    action: labelmap
  - source_labels: [__meta_kubernetes_namespace]
    separator: ;
    regex: (.*)
    target_label: kubernetes_namespace
    replacement: $1
    action: replace
  - source_labels: [__meta_kubernetes_service_name]
    separator: ;
    regex: (.*)
    target_label: kubernetes_name
    replacement: $1
    action: replace
- job_name: kubernetes-pods
  scrape_interval: 10s
  scrape_timeout: 10s
  metrics_path: /metrics
  scheme: http
  kubernetes_sd_configs:
  - api_server: null
    role: pod
    namespaces:
      names: []
  relabel_configs:
  - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
    separator: ;
    regex: "true"
    replacement: $1
    action: keep
  - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
    separator: ;
    regex: (.+)
    target_label: __metrics_path__
    replacement: $1
    action: replace
  - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
    separator: ;
    regex: ([^:;]+);(\d+)
    target_label: __address__
    replacement: ${1}:${2}
    action: replace
  - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
    separator: ;
    regex: ([^:;]+):(\d+);(\d+)
    target_label: __address__
    replacement: ${1}:${3}
    action: replace
  - separator: ;
    regex: __meta_kubernetes_pod_label_(.+)
    replacement: $1
    action: labelmap
  - source_labels: [__meta_kubernetes_namespace]
    separator: ;
    regex: (.*)
    target_label: kubernetes_namespace
    replacement: $1
    action: replace
  - source_labels: [__meta_kubernetes_pod_name]
    separator: ;
    regex: (.*)
    target_label: kubernetes_pod_name
    replacement: $1
    action: replace
- job_name: kubernetes-nginx-sidecar
  scrape_interval: 10s
  scrape_timeout: 10s
  metrics_path: /metrics
  scheme: http
  kubernetes_sd_configs:
  - api_server: null
    role: pod
    namespaces:
      names: []
  relabel_configs:
  - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape_nginx_sidecar]
    separator: ;
    regex: "true"
    replacement: $1
    action: keep
  - source_labels: [__address__]
    separator: ;
    regex: (.*):(\d+)
    target_label: __address__
    replacement: ${1}:9101
    action: replace
  - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_scrape_nginx_sidecar_port]
    separator: ;
    regex: ([^:;]+):(\d+);(\d+)
    target_label: __address__
    replacement: ${1}:${3}
    action: replace
  - separator: ;
    regex: __meta_kubernetes_pod_label_(.+)
    replacement: $1
    action: labelmap
  - source_labels: [__meta_kubernetes_namespace]
    separator: ;
    regex: (.*)
    target_label: kubernetes_namespace
    replacement: $1
    action: replace
  - source_labels: [__meta_kubernetes_pod_name]
    separator: ;
    regex: (.*)
    target_label: kubernetes_pod_name
    replacement: $1
    action: replace
- job_name: gce-vms
  scrape_interval: 10s
  scrape_timeout: 10s
  metrics_path: /metrics
  scheme: http
  gce_sd_configs:
  - project: travix-production
    zone: europe-west1-c
    refresh_interval: 1m
    port: 9101
    tag_separator: ','
  relabel_configs:
  - source_labels: [__meta_gce_tags]
    separator: ;
    regex: .*,prometheus,.*
    replacement: $1
    action: keep
  - source_labels: [__meta_gce_network]
    separator: ;
    regex: .*/production
    replacement: $1
    action: keep
  - source_labels: [__meta_gce_instance_name]
    separator: ;
    regex: (.*)
    target_label: instance
    replacement: $1
    action: replace
  - source_labels: [__meta_gce_tags]
    separator: ;
    regex: .*,app-([^,]+),.*
    target_label: app
    replacement: ${1}
    action: replace
  - source_labels: [__meta_gce_tags]
    separator: ;
    regex: .*,team-([^,]+),.*
    target_label: team
    replacement: ${1}
    action: replace
  - source_labels: [__meta_gce_tags]
    separator: ;
    regex: .*,version-([^,]+),.*
    target_label: version
    replacement: ${1}
    action: replace
- job_name: gce-vms-win-metrics
  scrape_interval: 10s
  scrape_timeout: 10s
  metrics_path: /metrics
  scheme: http
  gce_sd_configs:
  - project: travix-production
    zone: europe-west1-c
    refresh_interval: 1m
    port: 9182
    tag_separator: ','
  relabel_configs:
  - source_labels: [__meta_gce_tags]
    separator: ;
    regex: .*,prometheus,.*
    replacement: $1
    action: keep
  - source_labels: [__meta_gce_network]
    separator: ;
    regex: .*/production
    replacement: $1
    action: keep
  - source_labels: [__meta_gce_instance_name]
    separator: ;
    regex: (.*)
    target_label: instance
    replacement: $1
    action: replace
  - source_labels: [__meta_gce_tags]
    separator: ;
    regex: .*,app-([^,]+),.*
    target_label: app
    replacement: ${1}
    action: replace
  - source_labels: [__meta_gce_tags]
    separator: ;
    regex: .*,team-([^,]+),.*
    target_label: team
    replacement: ${1}
    action: replace
  - source_labels: [__meta_gce_tags]
    separator: ;
    regex: .*,version-([^,]+),.*
    target_label: version
    replacement: ${1}
    action: replace

Logs:

Last logs before it starts to fail

Hundreds of log lines similar to

time="2017-09-19T03:36:08Z" level=warning msg="append failed" err="no token found" source="scrape.go:670" target="{__address__="10.122.158.29:5000", __metrics_path__="/metrics", __scheme__="http", app="flightapi", instance="10.122.158.29:5000", job="kubernetes-pods", kubernetes_namespace="production", kubernetes_pod_name="flightapi-4094240061-9gpgb", pod_template_hash="4094240061", team="loki", tier="platform", track="stable", version="1.1.772"}

Logs when restarting after failure

time="2017-09-19T07:31:22Z" level=info msg="Starting prometheus (version=2.0.0-beta.3, branch=HEAD, revision=066783b3991dd64729325fc4f880dfffb484a2c2)" source="main.go:210"
time="2017-09-19T07:31:22Z" level=info msg="Build context (go=go1.8.3, user=root@0cbc320660dc, date=20170912-10:17:45)" source="main.go:211"
time="2017-09-19T07:31:22Z" level=info msg="Host details (Linux 4.4.52+ #1 SMP Thu Jul 13 11:47:20 PDT 2017 x86_64 prometheus-1 (none))" source="main.go:212"
time="2017-09-19T07:31:22Z" level=info msg="Starting tsdb" source="main.go:224"

The text was updated successfully, but these errors were encountered:

fabxc · 2017-09-19T11:58:01Z

Thanks for reporting. You are likely hitting the startup deadlock reported previously in #3185. It got fixed in prometheus-junkyard/tsdb#146.

We fixed and a couple more things over the last few days. I'm currently testing things with this container image: quay.io/fabxc/prometheus:v2.0.0-beta.4-debug-2955. You can try it out yourself in the mean time.
The changes will get released once my testing confirms it's all still working :)

Those changes also include the web server being reachable while we are restoring the database on restart.

JorritSalverda · 2017-09-19T15:12:40Z

@fabxc I've rolled out your version across all of our non-production clusters to see if the issues goes and things run well. So far things seem okay, apart from having to wipe out all data. I hope the data format stabilizes quickly :)

JorritSalverda · 2017-09-20T06:46:09Z

So far so good with the debug version, while the beta 3 instances were hanging again.

AndreaGiardini · 2017-09-20T09:10:51Z

We experienced the same problem. Deploying the debug version made it work again.

JorritSalverda · 2017-09-20T12:43:50Z

I do still run into the following issue(s) with this debug version:

fabxc · 2017-09-20T13:00:04Z

Can you share the exact logs you are seeing? Also, did you run this against a storage written by a beta.4?
We were not writing the WAL correctly so there's a chance this bug persisted in the WAL and carried through.

fabxc · 2017-09-20T13:02:22Z

Nevermind, also just hit that one now. I'll investigate.

JorritSalverda · 2017-09-20T13:09:07Z

It was indeed fresh data since upgrading to your debug version. Hope you find the cause and fix.

fabxc · 2017-09-20T13:17:50Z

Thinking about it – this may actually be intended behavior and we are just forgetting to skip the logging. The log line was added just recently: prometheus-junkyard/tsdb@162a48e

JorritSalverda · 2017-09-20T13:18:53Z

Eventually the instance outputting these logs started, it just took quite long. About 5 minutes.

fabxc · 2017-09-20T16:18:49Z

Thanks @JorritSalverda – this was a legitimate semantical race happening. Fix in prometheus-junkyard/tsdb#150

JorritSalverda · 2017-09-21T07:42:29Z

I hope you have a test version soon, our production instances have gone into CrashLoopBackoff state. Probably because recovery takes too long. Happy to test your fix.

JorritSalverda · 2017-09-21T19:21:30Z

Upgrading to prom/prometheus:v2.0.0-beta.5 now. I'll report tomorrow if all is still well.

JorritSalverda · 2017-09-22T07:21:16Z

@fabxc so far all of our 24 Prometheus instances are running fine. Feel free to close the issue. I'll reopen or create a new one if anything new pops up. Thanks for your super fast fixes!

fabxc · 2017-09-22T07:22:15Z

Awesome, thank you so much!

cauwulixuan · 2018-01-20T03:02:06Z

Hi, I came the same issue here.
I ran prometheus2.0.0 on kubernetes v1.8.5

kubectl get all -n monitoring
NAME                          DESIRED   CURRENT   AGE
statefulsets/prometheus-k8s   0         2         16h

NAME                  READY     STATUS             RESTARTS   AGE
po/prometheus-k8s-0   0/1       CrashLoopBackOff   81         16h
po/prometheus-k8s-1   0/1       CrashLoopBackOff   22         16h

The pod status turned CrashLoopBackOff and restarts over and over again, but still failed.
logs here:

[root@k8s-1 prometheus]# kubectl logs prometheus-k8s-0 -n monitoring
level=info ts=2018-01-20T02:54:05.96260589Z caller=main.go:215 msg="Starting Prometheus" version="(version=2.0.0, branch=HEAD, revision=0a74f98628a0463dddc90528220c94de5032d1a0)"
level=info ts=2018-01-20T02:54:05.96280392Z caller=main.go:216 build_context="(go=go1.9.2, user=root@615b82cb36b6, date=20171108-07:11:59)"
level=info ts=2018-01-20T02:54:05.962830996Z caller=main.go:217 host_details="(Linux 3.10.0-229.el7.x86_64 #1 SMP Fri Mar 6 11:36:42 UTC 2015 x86_64 prometheus-k8s-0 (none))"
level=info ts=2018-01-20T02:54:05.967568348Z caller=web.go:380 component=web msg="Start listening for connections" address=0.0.0.0:9090
level=info ts=2018-01-20T02:54:05.969231264Z caller=main.go:314 msg="Starting TSDB"
level=info ts=2018-01-20T02:54:05.969498932Z caller=targetmanager.go:71 component="target manager" msg="Starting target manager..."
level=error ts=2018-01-20T02:54:33.993295801Z caller=main.go:323 msg="Opening storage failed" err="invalid block sequence: block time ranges overlap (1516348800000, 1516356000000)"

[root@k8s-1 prometheus]# kubectl describe po/prometheus-k8s-0 -n monitoring
Name:           prometheus-k8s-0
Namespace:      monitoring
Node:           k8s-3/172.16.1.8
Start Time:     Fri, 19 Jan 2018 17:59:38 +0800
Labels:         app=prometheus
                controller-revision-hash=prometheus-k8s-7d86dfbd86
                prometheus=k8s
Annotations:    kubernetes.io/created-by={"kind":"SerializedReference","apiVersion":"v1","reference":{"kind":"StatefulSet","namespace":"monitoring","name":"prometheus-k8s","uid":"7593d8ac-fcff-11e7-9333-fa163e48f857"...
Status:         Running
IP:             10.244.2.54
Created By:     StatefulSet/prometheus-k8s
Controlled By:  StatefulSet/prometheus-k8s
Containers:
  prometheus:
    Container ID:  docker://f4707fc35d02683ed457596ca7433c46c4eb27d40257353bc30aa2d352873a26
    Image:         quay.io/prometheus/prometheus:v2.0.0
    Image ID:      docker-pullable://quay.io/prometheus/prometheus@sha256:53afe934a8d497bb703dbbf7db273681a56677775c462833da8d85015471f7a3
    Port:          9090/TCP
    Args:
      --config.file=/etc/prometheus/config/prometheus.yaml
      --storage.tsdb.path=/cephfs/prometheus/data
      --storage.tsdb.retention=180d
      --web.route-prefix=/
      --web.enable-lifecycle
      --web.enable-admin-api
    State:          Terminated
      Reason:       Error
      Exit Code:    1
      Started:      Sat, 20 Jan 2018 10:59:47 +0800
      Finished:     Sat, 20 Jan 2018 11:00:16 +0800
    Last State:     Terminated
      Reason:       Error
      Exit Code:    1
      Started:      Sat, 20 Jan 2018 10:54:05 +0800
      Finished:     Sat, 20 Jan 2018 10:54:34 +0800
    Ready:          False
    Restart Count:  82
    Limits:
      cpu:     500m
      memory:  500Mi
    Requests:
      cpu:        100m
      memory:     200Mi
    Liveness:     http-get http://:web/status delay=30s timeout=3s period=5s #success=1 #failure=10
    Readiness:    http-get http://:web/status delay=0s timeout=3s period=5s #success=1 #failure=6
    Environment:  <none>
    Mounts:
      /cephfs/prometheus/data from data (rw)
      /etc/prometheus/config from config (rw)
      /etc/prometheus/rules from rules (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from prometheus-k8s-token-x8xzh (ro)
Conditions:
  Type           Status
  Initialized    True 
  Ready          False 
  PodScheduled   True 
Volumes:
  config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      prometheus-k8s-config
    Optional:  false
  rules:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      prometheus-k8s-rules
    Optional:  false
  data:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  cephfs-pvc
    ReadOnly:   false
  prometheus-k8s-token-x8xzh:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  prometheus-k8s-token-x8xzh
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     node.alpha.kubernetes.io/notReady:NoExecute for 300s
                 node.alpha.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason      Age                  From            Message
  ----     ------      ----                 ----            -------
  Warning  Unhealthy   50m (x429 over 17h)  kubelet, k8s-3  Readiness probe failed: HTTP probe failed with statuscode: 503
  Warning  FailedSync  5m (x1718 over 6h)   kubelet, k8s-3  Error syncing pod
  Normal   Pulled      33s (x83 over 17h)   kubelet, k8s-3  Container image "quay.io/prometheus/prometheus:v2.0.0" already present on machine

According to the logs here, I still cannt figure out what is going on here, any suggestions? Many Thanks.

Kirchen99 · 2018-04-03T15:16:35Z

Prometheus 2.2.1 fails to start up with similar issue:

level=info ts=2018-04-03T10:56:01.819807459Z caller=main.go:220 msg="Starting Prometheus" version="(version=2.2.1, branch=HEAD, revision=bc6058c81272a8d938c05e75607371284236aadc)"
level=info ts=2018-04-03T10:56:01.819870594Z caller=main.go:221 build_context="(go=go1.10, user=root@149e5b3f0829, date=20180314-14:15:45)"
level=info ts=2018-04-03T10:56:01.819889331Z caller=main.go:222 host_details="(Linux 4.9.75-rancher #1 SMP Sat Jan 6 00:16:10 UTC 2018 x86_64 prometheus-0 (none))"
level=info ts=2018-04-03T10:56:01.819906466Z caller=main.go:223 fd_limits="(soft=1000000, hard=1000000)"
level=info ts=2018-04-03T10:56:01.822554312Z caller=main.go:504 msg="Starting TSDB ..."
level=info ts=2018-04-03T10:56:01.823018508Z caller=web.go:382 component=web msg="Start listening for connections" address=0.0.0.0:9090

adilnaimi · 2018-04-04T06:05:00Z

same issue

prometheus --version
prometheus, version 2.2.1 (branch: HEAD, revision: bc6058c81272a8d938c05e75607371284236aadc)
  build user:       root@149e5b3f0829
  build date:       20180314-14:15:45
  go version:       go1.10

systemctl status -l prometheus
● prometheus.service - Prometheus
   Loaded: loaded (/etc/systemd/system/prometheus.service; enabled; vendor preset: disabled)
   Active: active (running) since Wed 2018-04-04 09:02:46 +03; 402ms ago
 Main PID: 1109 (prometheus)
   Memory: 74.9M
   CGroup: /system.slice/prometheus.service
           └─1109 /app/prometheus/bin/prometheus --config.file=/app/prometheus/config/prometheus.yml --storage.tsdb.path=/app/prometheus/db --storage.tsdb.retention=30d --web.listen-address=
0.0.0.0:9090 --web.external-url=

Apr 04 09:02:46 prometheus-instance-01 systemd[1]: Started Prometheus.
Apr 04 09:02:46 prometheus-instance-01 systemd[1]: Starting Prometheus...
Apr 04 09:02:46 prometheus-instance-01 prometheus[1109]: level=info ts=2018-04-04T06:02:46.150382858Z caller=main.go:220 msg="Starting Prometheus" version="(version=2.2.1, branch=HEAD,
 revision=bc6058c81272a8d938c05e75607371284236aadc)"
Apr 04 09:02:46 prometheus-instance-01 prometheus[1109]: level=info ts=2018-04-04T06:02:46.150470673Z caller=main.go:221 build_context="(go=go1.10, user=root@149e5b3f0829, date=2018031
4-14:15:45)"
Apr 04 09:02:46 prometheus-instance-01 prometheus[1109]: level=info ts=2018-04-04T06:02:46.150500434Z caller=main.go:222 host_details="(Linux 3.10.0-693.5.2.el7.x86_64 #1 SMP Fri Oct 1
3 10:46:25 EDT 2017 x86_64 prometheus-instance-01 (none))"
Apr 04 09:02:46 prometheus-instance-01 prometheus[1109]: level=info ts=2018-04-04T06:02:46.150523108Z caller=main.go:223 fd_limits="(soft=1024, hard=4096)"
Apr 04 09:02:46 prometheus-instance-01 prometheus[1109]: level=info ts=2018-04-04T06:02:46.155917136Z caller=main.go:504 msg="Starting TSDB ..."
Apr 04 09:02:46 prometheus-instance-01 prometheus[1109]: level=info ts=2018-04-04T06:02:46.155939911Z caller=web.go:382 component=web msg="Start listening for connections" address=0.0.
0.0:9090

gouthamve · 2018-04-04T06:19:34Z

Hi @adilnaimi @Kirchen99 depending on the size of the data, it might take a minute or two to start. If it never starts, please open a new issue describing the issue in detail.

adilnaimi · 2018-04-04T10:56:41Z

Maybe this can help someone else, please check the configuration files.

for my case, it was my configuration file prometheus.yml has a wrong yml entry, I fixed and Prometheus works fine

gouthamve · 2018-04-04T10:58:24Z

@adilnaimi That should error out rather than hang prometheus. Could you share if the process is exiting or hanging? If hanging, which entry caused this?

adilnaimi · 2018-04-04T11:03:34Z

@gouthamve it was hanging

I had this entry in log file

level=error ts=2018-04-04T10:36:32.741554373Z caller=main.go:582 err="Error loading config couldn't load configuration (--config.file=/app/prometheus/config/prometheus.yml): parsing YAML file /app/prometheus/config/prometheus.yml: yaml: line 110: could not find expected ':'"

yanc0 · 2018-05-30T13:13:51Z

Our Prometheus had a lot of restarts too. We added 500MB to the resourceRequest and the resourceLimit. Values were previously 1024MB each. There is no more restarts now but I think it can be nice to log this behaviour on stdout or stderr telling that Prometheus have to Kill itself because of the lack of memory.

version=2.2.1, branch=HEAD, revision=bc6058c81272a8d938c05e75607371284236aadc

krasi-georgiev · 2018-05-30T13:30:29Z

AFAIC Prometheus is killed by the OS so can't show any logs. Otherwise any other porblems are reported by a log or go routine trace.

yanc0 · 2018-05-30T13:47:04Z

I looked to Kubernetes events and the is no trace of OOMKilling. I suspect a bad catch of the return code of malloc or something like that. I really don't know...

krasi-georgiev · 2018-05-30T18:45:15Z

so far I have not seen any report of silent restarts so I would advise to keep digging the k8s logs and if you are 100% this is a bug please open a new issue and how to reproduce.

kaspernissen · 2018-06-20T06:04:47Z

I think I'm experiencing the same issue. Last week I updated our k8s cluster from 1.8.10 to 1.9.6. After this upgrade, my Prometheus installation ends up in a CrashLoopBackOff with the following logs:

level=info ts=2018-06-20T05:53:42.854160365Z caller=main.go:222 msg="Starting Prometheus" version="(version=2.3.1, branch=HEAD, revision=188ca45bd85ce843071e768d855722a9d9dabe03)"
level=info ts=2018-06-20T05:53:42.854222935Z caller=main.go:223 build_context="(go=go1.10.3, user=root@82ef94f1b8f7, date=20180619-15:56:22)"
level=info ts=2018-06-20T05:53:42.854329715Z caller=main.go:224 host_details="(Linux 4.4.121-k8s #1 SMP Sun Mar 11 19:39:47 UTC 2018 x86_64 prom-699b5675b-l5l47 (none))"
level=info ts=2018-06-20T05:53:42.854403569Z caller=main.go:225 fd_limits="(soft=1048576, hard=1048576)"
level=info ts=2018-06-20T05:53:42.85558605Z caller=main.go:514 msg="Starting TSDB ..."
level=info ts=2018-06-20T05:53:42.855772367Z caller=web.go:415 component=web msg="Start listening for connections" address=0.0.0.0:9090
level=info ts=2018-06-20T05:53:42.859350794Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1526428800000 maxt=1526623200000 ulid=01CDS7MTAA8E2ZEAVB4CM502HH
level=info ts=2018-06-20T05:53:42.859999296Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1526623200000 maxt=1526817600000 ulid=01CDZ11E69G5R99X7WT7E51673
level=info ts=2018-06-20T05:53:42.860629301Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1526817600000 maxt=1527012000000 ulid=01CE4TE1VSSN02W1BDCYBENPC3
level=info ts=2018-06-20T05:53:42.861290138Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1527012000000 maxt=1527206400000 ulid=01CEAKTN9YTK0QRMNWAWYANYJD
level=info ts=2018-06-20T05:53:42.861924241Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1527206400000 maxt=1527400800000 ulid=01CEGD784X4RK9ZMHYRQ4400JX
level=info ts=2018-06-20T05:53:42.862575151Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1527400800000 maxt=1527595200000 ulid=01CEP6KWPARJA9H62A7Z16ZF6W
level=info ts=2018-06-20T05:53:42.863139188Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1527595200000 maxt=1527789600000 ulid=01CEW00HV93TQB822J27SC8XND
level=info ts=2018-06-20T05:53:42.863694467Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1527789600000 maxt=1527984000000 ulid=01CF1SD68EVZB6YB1BC80X15NC
level=info ts=2018-06-20T05:53:42.864268684Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1527984000000 maxt=1528178400000 ulid=01CF7JSS2S8K43HNRPTZGTHM7E
level=info ts=2018-06-20T05:53:42.86481637Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1528178400000 maxt=1528372800000 ulid=01CFDC6BWJCHK00Q6965RJRXZR
level=info ts=2018-06-20T05:53:42.865394658Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1528372800000 maxt=1528567200000 ulid=01CFK5JZVYZ99YSK4HCF17R4F4
level=info ts=2018-06-20T05:53:42.865913061Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1528567200000 maxt=1528761600000 ulid=01CFRYZKSXV0VPS77BWR4K2M64
level=info ts=2018-06-20T05:53:42.86649269Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1528761600000 maxt=1528956000000 ulid=01CFYRC97BBABYEJYG20T0M69E
level=info ts=2018-06-20T05:53:42.867012119Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1528956000000 maxt=1529150400000 ulid=01CG4HRSFWJN382QKXEHBE4QXK
level=info ts=2018-06-20T05:53:42.86762438Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1529215200000 maxt=1529222400000 ulid=01CG6FHXP6R936HG4M14XEXKST
level=info ts=2018-06-20T05:53:42.868136652Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1529150400000 maxt=1529215200000 ulid=01CG6FJNQ9BVPSY10C3MYHGNF6
level=info ts=2018-06-20T05:54:27.973833633Z caller=main.go:524 msg="TSDB started"
level=info ts=2018-06-20T05:54:27.973939054Z caller=main.go:603 msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml
level=info ts=2018-06-20T05:54:27.977011891Z caller=kubernetes.go:188 component="discovery manager scrape" discovery=k8s msg="Using pod service account via in-cluster config"
level=info ts=2018-06-20T05:54:27.978278605Z caller=kubernetes.go:188 component="discovery manager scrape" discovery=k8s msg="Using pod service account via in-cluster config"
level=info ts=2018-06-20T05:54:27.979594832Z caller=kubernetes.go:188 component="discovery manager scrape" discovery=k8s msg="Using pod service account via in-cluster config"
level=info ts=2018-06-20T05:54:27.980891135Z caller=kubernetes.go:188 component="discovery manager scrape" discovery=k8s msg="Using pod service account via in-cluster config"
level=info ts=2018-06-20T05:54:27.98218082Z caller=kubernetes.go:188 component="discovery manager scrape" discovery=k8s msg="Using pod service account via in-cluster config"
level=info ts=2018-06-20T05:54:27.989575538Z caller=main.go:500 msg="Server is ready to receive web requests."

Watching the status of kubectl get pods displays the status OOMKilled.

I was previously on Prometheus version 2.2.1 and further tried upgrading to 2.3.1 without any luck. The above logs are from 2.3.1.

kaspernissen · 2018-06-20T06:14:00Z

A small follow-up. Since this was happening in one of our test environments, deleting and recreating the pvc was a possible solution, which works. However, not a solution for production environments.

brian-brazil · 2018-06-20T07:57:56Z

@kaspernissen Please don't me too on unrelated (and closed) issues. If you believe you have found a bug, file a new issue.

kaspernissen · 2018-06-20T11:05:17Z

@brian-brazil Sure, I can create a new issue. However, this issue seemed to be the most relevant place because of the recent activity despite being closed.

Neru007 · 2018-06-21T13:09:35Z

I'm also facing similar kind of issue after upgrading to 2.3. It's taking eternity to start.

level=info ts=2018-06-21T13:03:16.209817079Z caller=main.go:223 build_context="(go=go1.10.2, user=root@d539e167976a, date=20180607-08:46:54)" level=info ts=2018-06-21T13:03:16.209840142Z caller=main.go:224 host_details="(Linux 3.10.0-514.el7.x86_64 #1 SMP Wed Oct 19 11:24:13 EDT 2016 x86_64 JC-PD-RH-PRM-Prometheus-Node2 (none))" level=info ts=2018-06-21T13:03:16.209857984Z caller=main.go:225 fd_limits="(soft=1024, hard=4096)" level=info ts=2018-06-21T13:03:16.210799855Z caller=main.go:514 msg="Starting TSDB ..." level=info ts=2018-06-21T13:03:16.210837089Z caller=web.go:426 component=web msg="Start listening for connections" address=0.0.0.0:9200 level=info ts=2018-06-21T13:03:16.211733376Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1528243200000 maxt=1528308000000 ulid=01CFB7WEXBC0V6TTQJ4N9STKF2 level=info ts=2018-06-21T13:03:16.211845796Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1528308000000 maxt=1528372800000 ulid=01CFD5M3VFS5AJZHKVP0FEMENH level=info ts=2018-06-21T13:03:16.211932872Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1528372800000 maxt=1528437600000 ulid=01CFF3D9T4VTMKNK5EC0787QH3 level=info ts=2018-06-21T13:03:16.212000917Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1528437600000 maxt=1528502400000 ulid=01CFH0WCNM9XWB36Y7W184VGAQ level=info ts=2018-06-21T13:03:16.212052193Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1528502400000 maxt=1528567200000 ulid=01CFJZ0MME8CTVESNBPVX5TS52 level=info ts=2018-06-21T13:03:16.212120134Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1528567200000 maxt=1528632000000 ulid=01CFMWV8GHABGBG8TJ6BNME1BN level=info ts=2018-06-21T13:03:16.21217986Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1528632000000 maxt=1528696800000 ulid=01CFPTK8SVBPRFRRRY73TPHP3H level=info ts=2018-06-21T13:03:16.212240528Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1528696800000 maxt=1528761600000 ulid=01CFRR51AQW1WFHSQ6T8N725JK level=info ts=2018-06-21T13:03:16.212298281Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1528761600000 maxt=1528826400000 ulid=01CFTPASNJQ3JVB1AWEG6XXA8X level=info ts=2018-06-21T13:03:16.212348635Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1528826400000 maxt=1528891200000 ulid=01CFWM4BDRQEVDCPZHDA4V3CGY level=info ts=2018-06-21T13:03:16.21240339Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1528891200000 maxt=1528956000000 ulid=01CFYHWZE6R6B020MSTFYF1CPP level=info ts=2018-06-21T13:03:16.212452853Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1528956000000 maxt=1529020800000 ulid=01CG0FB368B8WJMWMPWQZC2W6D level=info ts=2018-06-21T13:03:16.212507316Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1529020800000 maxt=1529085600000 ulid=01CG2DSTRWFZ5Z5PNX036RYHA4 level=info ts=2018-06-21T13:03:16.212566705Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1529085600000 maxt=1529150400000 ulid=01CG4BE4NYWGWZRDQWVT339FZN level=info ts=2018-06-21T13:03:16.212617838Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1529150400000 maxt=1529215200000 ulid=01CG69G5D5NXWVGFGPP5J6TCJP level=info ts=2018-06-21T13:03:16.212680356Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1529215200000 maxt=1529280000000 ulid=01CG86PZ9HGWCB21E6QD3HBGTX level=info ts=2018-06-21T13:03:16.212735617Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1529280000000 maxt=1529344800000 ulid=01CGABPP6D9HWP3AREP89BEZ47 level=info ts=2018-06-21T13:03:16.212785483Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1529344800000 maxt=1529409600000 ulid=01CGC9KZKCX388MKBQYEK2EJSB level=info ts=2018-06-21T13:03:16.212833596Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1529409600000 maxt=1529474400000 ulid=01CGE7EW6P58W9C1ZCQ9TWPMH5 level=info ts=2018-06-21T13:03:16.212920033Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1529474400000 maxt=1529539200000 ulid=01CGG50EFDB5N57ENCP101R7Q5 level=info ts=2018-06-21T13:03:16.212988293Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1529560800000 maxt=1529568000000 ulid=01CGGRY8DW2GSXPM10H3ZKJ1FF level=info ts=2018-06-21T13:03:16.213034254Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1529539200000 maxt=1529560800000 ulid=01CGGSH796KNG1TKV0AMQP7JGD level=info ts=2018-06-21T13:03:16.213077236Z caller=repair.go:39 component=tsdb msg="found healthy block" mint=1529568000000 maxt=1529575200000 ulid=01CGGZSZNYDWPBWQ0S1ZB9D2EP

RetroDreams · 2018-06-28T17:57:56Z

@brian-brazil that doesn't seem like a very constructive response to someone providing evidence of what is a related issue. I am also experiencing the same behavior on 2.3.1 and this is the only issue that seems to be relevant in my searches. Each time a new pod is brought up in kubernetes my UI just returns "Service Unavailable". This is with using a persistent volume also.

brian-brazil · 2018-06-28T18:18:10Z

@sdeehring We want to fix bugs that users report, however it is very difficult to do so when an issue ends up with in this case 5 distinct behaviours being discussed, which is unfortunately common. Accordingly when an unrelated issue is spotted we ask the reporter to file a new bug so that both it an the original issue can get appropriate attention. A deadlock issue that was fixed in 2.0.0beta4 is unlikely to be related to a OOM issue in 2.3.1.

Each time a new pod is brought up in kubernetes my UI just returns "Service Unavailable".

This sounds like a 6th distinct behaviour. If you think this is a bug please file a new one so it can be looked into.

tvvignesh · 2018-07-12T05:36:24Z

Hi. Facing the same issue with the latest version of Prometheus. It was working well for 2 days but the next day, I got this error and even using the tsdb tool does not help. Am I doing something wrong?

`[ramcoazure@HCMDEMAZIND01 data]$ ls
01CHF863S201TBMD667KNN3TE3 01CJ57V0D8SZRCPT3YECRWF4BA 01CJ5CCK52FZNA1JYSNZ89W9D7 01CJ5H1B03HZRE18HHN7TRW7XP 01CJ5NH77VACYZV4E54DMZGD6A 01CJ5T0719MR6K00SR3422RZQN
01CHGZ3YMXBMM09NDKTJA2P2BH 01CJ57WVRE8CQRXFVE3HPFGX25 01CJ5CEJDTM8REY02X40T4BGSV 01CJ5H36KAH6N28HEJQJQEABDH 01CJ5NK4XN165Z8PEC80EPBJBT 01CJ5T2205DQJQQSWBTA9R4HA7
01CHJWXF056J7YPX5ZP2SSYGW4 01CJ57YR6WXNHJY0SD4X9ZBT6X 01CJ5CGF5E3EKMWJ25XQ7BZ5DY 01CJ5H52B7NNK5QV1PX0J1N7V4 01CJ5NN02XSQWJ0ABKERCA3TAA 01CJ5T3X07N6V6M958HGBHD4AV
01CHMTQ21WA532AY7P68XZ02F8 01CJ580M77MS3BC4FZBHNKFQ8T 01CJ5CJBR540JJGPG7RW8KDJJF 01CJ5H713PPS77S8JXVWT6WWPQ 01CJ5NPV2PST0QS1VRK2MZX3XH 01CJ5T5QYFXKJ2X4SXVM3DB1KR
01CHPRGN1E7W6VYQV032B95ZAA 01CJ582JPGBJYK9RRKQDSZ5553 01CJ5CM7HF7BW5F34S9JB292VK 01CJ5H8ZHAF912PQPGFRG7AS6Y 01CJ5NRSZMVXWJ4S1EQV67P3ZP 01CJ5T7JX7M8H6M6D2VWAREVXJ
01CHRPA0DHK5GTW7GV65GKTQC0 01CJ584FM8TPV3JSYDC8N4QH2T 01CJ5CP4FWR6C7XJ162Z85JABG 01CJ5HAVXZTMK9XEF15Y7K9WV0 01CJ5NTMVK0E67D9NMTPJY6NY4 01CJ5T9E1FFV9JCFAEW81KCQKB
01CHTM3JCMQZQ5KJ66G8H5H4GE 01CJ586E43A1WY89D7DSYJB8C7 01CJ5CR244KCP4T4VK5DHE1W5G 01CJ5HCRGG4PYVJRKRJ56AFQF0 01CJ5NWFXV2EXSRPW679SKFDN0 01CJ5TBA1W66XYJYVQ0RYJFPRH
01CHWHX2Y0CXWFWYD7C41MH7YB 01CJ588A9HENY3D3BPKDRC14QE 01CJ5CT04S3F185CEXABE5MYA2 01CJ5HEQRR8XGXPZZ8HD4YWXE0 01CJ5NYF7YH0XDDZBC419WW3M3 01CJ5TD5ZF8MCQNWJ5PGFMK7M6
01CHYFPPFQASSQ6N649EN75803 01CJ58A7XB8VS778F0PD45NABZ 01CJ5CVYSC41MB50CDP6ZSS6E1 01CJ5HGQ11NA03BCEPYW46DF34 01CJ5P0E2HWVM8GNMT7EFRX2H3 01CJ5TF0Y8TAT9B0T37ERMC5NZ
01CJ0DG8HAPPYAFDPY4KCH6ATM 01CJ58C7WY07MDNXSQ4EJF2RP3 01CJ5CXZNY7RFFNXEJM82S44K8 01CJ5HJKQHBD4XQHQEJ6HAHYEB 01CJ5P296BBVN8BG1H6P36AJCE 01CJ5TGVW0MCJD45MCPGWMBYB8
01CJ2B9QHK9Z4909NNNB2S9SDR 01CJ58E6KRCK2VP3V2KYPPEARM 01CJ5CZYWCRM0F8DQP9AT3TWMJ 01CJ5HMFM3PZJC2M4HD7TAVBV2 01CJ5P448ZWR8Z98GHRTDD7XFE 01CJ5TJTDRB579KS6ZZM8JFRTJ
01CJ493678EFRDSFZNPQDHXNAP 01CJ58G6KRY94RRJST7TP2WBFS 01CJ5D269KP079D0CRVKQ9RCJS 01CJ5HPD69QTRQW6MJRS5SXDZR 01CJ5P631960H6PTYNWQKS20HM 01CJ5TMNE7X929HE32R7WXXR2G
01CJ4936XK9AE031X8133MXDVT 01CJ58J1ZQETE7WBW5BSKV7XQH 01CJ5D45V9NAR0NJKRFE49346A 01CJ5HR9GPZ6D2JJQTJP5RV739 01CJ5P7Y1BANGJXDKK9PJAA7DD 01CJ5TPGGF3KPV26B7JA85VF4W
01CJ4FYXFG0V5VM0G2FQXT6G3A 01CJ58KXBG6AVTWD7E202403BN 01CJ5D63B0GSBB9Y0PG8XSN446 01CJ5HT91PA53VHNZQ322HGJ3N 01CJ5P9WV0TVEH2TA57DH2K0XF 01CJ5TREMWEHZDHETXND7EEYRE
01CJ4XPD0BPWZKPJNW08JPC0DF 01CJ58NTFCSRSRG0ZA62XYS9SN 01CJ5D80ZZ83AQHHT677VM7CDZ 01CJ5HW5R3WVFWYAHQ5GTC4ZJZ 01CJ5PBWDGPFKAPNWS36W2SRN1 01CJ5TT9KTR6ZA046FX5APWA89
01CJ4XPDNXD5AWTB44DZ0QTGQG 01CJ58QPTR6DMZ511E2D975ZZP 01CJ5D9X01PQ2SHV4J72GR51V5 01CJ5HY2VR6YC8N1AWWCNV75YQ 01CJ5PDRKTRTFK7X10PZESDWNQ 01CJ5TW4QBNDMDEMXYV7K529GP
01CJ54J496SFZ7QDHSSZC03G32 01CJ58SMQ3515RYFAV04CGAMNQ 01CJ5DBVJC2DPKNWGJSD3MYJTS 01CJ5HZZ4AZJFJBNEJFA3WCAJ7 01CJ5PFKK9N6MZGBW2H1MYHEHR 01CJ5TXZMEPDY4EX6R8GV80P4Y
01CJ54J5V66TX4YSMV3RDMK917 01CJ58VGKDZZMXWZYH7MX2QZH3 01CJ5DDV4N0600G1F6BDXK41PY 01CJ5J1W43ZR6SY4VEJRD9GHF7 01CJ5PHEJWVFM1YQ0XSZ2F4Z8C 01CJ5TZY1CY7Z3AAH2NM4QDPG7
01CJ54JBTCDV83661ATDJFN034 01CJ58XBX6Z1MFPKR6700G9ZYX 01CJ5DFVPAB4JBXHENVC9HE6P1 01CJ5J3QWR5Y3XDK54B0FDT0R9 01CJ5PKBV7GF53SFVQ26JR00EE 01CJ5V1S0EK00A08WG3W9K6BEQ
01CJ54JGRPJ55YMZ0WPJA5WTXH 01CJ58Z8HH42M791KVHQ4XJ1KF 01CJ5DK6HNCH7J3BV866DRSR9N 01CJ5J5RWKJ797302SFFZ534MD 01CJ5PN6SF1XE2MQDA1QZSBYG1 01CJ5V3PZVQC5VZTKMCK5KTED4
01CJ54JVRXSY42VH8EGNKWYPDY 01CJ591476RHHKA8JKNPDFREQ3 01CJ5DN36VB72VPH2AC188G2JZ 01CJ5J7NSZPGTZ2RKMWMQX47FP 01CJ5PQ1R16CTMS1QXMQ59JPGV 01CJ5V5J1P9SNBSKG45MXG7HDC
01CJ54KBNRWJ7912V1YR9TDXT1 01CJ59346E341E613QPX90FH64 01CJ5DQ3QSV6CG2DHKJ9S66AJ0 01CJ5J9HP7DKTJD5SYBFW3KQHE 01CJ5PRZ1DJFZ01XNWM9SQZGGE 01CJ5V7CYTHJW1HBJHJAA2KC2G
01CJ54MF1A7DR8QJ9963KDE65E 01CJ59500WKW82ZEN5CP6BPPXH 01CJ5DS05Z2PCKCTEZ8DDQCT62 01CJ5JBE0RT66F9QRXMB1H6V89 01CJ5PTTMCACVEZ86M79SN4BDG 01CJ5V9B9543GCZN2CT7K713V8
01CJ54PA2SB46CAT9CG4HDS74T 01CJ596XWEPX1QY6KV6P9C9B2N 01CJ5DTX6D50SFJFAKTA8KAVQ0 01CJ5JDA0N9RK9K2J50NM2S217 01CJ5PWNMZNZFZG3DCCVWQ0APW 01CJ5VB9EJABPCFQSB7HYSPXAV
01CJ54R573TKF597WTWMYAQWP7 01CJ598SAY8QNEZA8FZ3CA2ZWP 01CJ5DXB5QQGGYX7XC40ZXJGCY 01CJ5JF6SMG7VDSXCQMHFYV0P4 01CJ5PYGMHRCCHJ37PMRWDVTPS 01CJ5VD4F3M713BJKM81JTKZGC
01CJ54T0V70HZ68QQM3NXXAS2X 01CJ59AMQKSBBM9CG1HPCNM042 01CJ5DZ7DS9AM890WXBMQ1KVQM 01CJ5JH39TX5M22V6SYBCBP2KG 01CJ5Q0C46HDHWXB7W2JVJT1QZ 01CJ5VEZZJE01KMPG3Y54Q3Q06
01CJ54W07ZXFEC2KGEJ97C490G 01CJ59CHJ2FG7734383Y64ZXXZ 01CJ5E15CXKH2QYF6PR5N9FNBA 01CJ5JK1W5XN67VXS5FFVKH6RG 01CJ5Q277AK5SM6HJD70MVVPPQ 01CJ5VGV8NDPDZ8FVRH43QRKPE
01CJ54Y0BA2FKGWXN4EFT8SHSV 01CJ59EE9HCAVKQSZSSPJR095X 01CJ5E31TA67W6968J6F4AM6FX 01CJ5JMYW2SB5P597M06J8T1HJ 01CJ5Q425R66Z7CWF486BYS11K 01CJ5VJPB6XA44AR5RPPQ4651G
01CJ54ZVX31VSS5Y91820DFVPM 01CJ59GDA1HQY6WN9CSKY1K2QE 01CJ5E6C43Y456FS95PQ6VVQ68 01CJ5JPXC3S07Y3YXFM5YM0YH2 01CJ5Q5X5ZQ3QNZBZ050XKPPPS 01CJ5VMJVNK1M0XKMGH8KNA6WS
01CJ551V3TC0NPBVD4DCC9B7RK 01CJ59JBVW66J3WYX79TQ0F1N5 01CJ5E888HECNC32ERF466DRT5 01CJ5JRS371SM8CKQJAC0HNKH4 01CJ5Q7TDWSH5FX7KEA5HPXSF8 01CJ5VPFXYWS8B8FZH6BY9EANT
01CJ553QR9543PBZK7PXKTQ9SE 01CJ59M7A2WZMGK7KMGY4VYM9P 01CJ5EA3XNH6A2ZTCKK1Z0D5R9 01CJ5JTNQ0MRJ1BGYRYS1PBNEC 01CJ5Q9S9HNHMV7QRF3YQJBK9R 01CJ5VRAY5ZAD2WJZ7SRM22X2V
01CJ555JRCNMBVSEQV67FVYJRH 01CJ59P2ZQE330RF08V2AT4KSN 01CJ5EC2S3H7TFHZKSSF1Q1SSR 01CJ5JWHHDZQWDZHRM58EFVVMS 01CJ5QBNQ2537BWE8Z15HGTRMZ 01CJ5VT64X07ZH6CEK1D589AMH
01CJ557H26AXFVJD14GEE4EW61 01CJ59QZRV5YBQJV99T2XCEJSJ 01CJ5EE0BR6DRH3B9RJ2C0P3TA 01CJ5JYDC53Z9B9ZAJKK1KH0W7 01CJ5QDGQEB2X9BA9RH3RTR26A 01CJ5VW14AS2D38S1X0JBXDB3E
01CJ559C32XHQ6VQCSQGMQ47K7 01CJ59SVBD0N6C3BW32W2YYJ58 01CJ5EFW0CTCWA31406J19MD09 01CJ5K0BG5F02FBK7S08W4FRTZ 01CJ5QFBNTZY7PPPWDZSSBKD4D 01CJ5VXW5QQ5S4HEBH5MDEV6BA
01CJ55BB52GYD23A7P3VF2Z3BP 01CJ59VPZ28VYDQNF7BEWZXP4S 01CJ5EHQH43ZBWF0MX0RVCSNXD 01CJ5K29XCN00Q94A348MRAWYF 01CJ5QH6NZC65DN0F7W1B60RW6 01CJ5VZQ8H0EE3MN54ZR5D27MP
01CJ55D62K50AS5GHGYKB7980B 01CJ59XKX6S1DHF2X6N108F026 01CJ5EKMG22JHG7TGG3X59J4T9 01CJ5K45WP64RWCAB37H7P8C7N 01CJ5QK205N91E11FQTESZSMFA 01CJ5W1JGQBP361SBQGRDTBVP3
01CJ55F11B9YN0ZHJ6VTMGDFX2 01CJ59ZRJHAJ1QNGXWZVC7KJQ6 01CJ5ENMCZT7MNNVJWF4ZWYBZ7 01CJ5K6232RWJQ9QYS1DW3DMV4 01CJ5QN0KWZ4ZSYCQYRQEQDTFZ 01CJ5W3GFC8VJ032NY3KSR266V
01CJ55GW6TQ63JKJB5K66P5ZA6 01CJ5A1N5M6WHZB9VVNF4KKCFQ 01CJ5EQGDRJAS9BZ8ZD48C3SPX 01CJ5K808CGTNS03TTQ4T79PVH 01CJ5QPVHPSYK0P4GKVFKJP6FM 01CJ5W5E71F2AADM41VFXF5Y35
01CJ55JRYGV6FSSWZY0D468ZTA 01CJ5A3J8JAPWMCEVMDQH3JKPW 01CJ5ESDX8VPYA1RQMPEQK21QQ 01CJ5K9W1Z3NCN4YHDCGTDXBXP 01CJ5QRQQTY8A72R3F1MV3QSM2 01CJ5W79KNAGMYBM5KG3J0AMJF
01CJ55MM20CT1KC8FKKEMR2N3E 01CJ5A5FJD2VAKEYM9BBE4K3F6 01CJ5EV9Z2FCV30T7F59TD5CNG 01CJ5KBVFH4H4JS1675CXSGSFA 01CJ5QTMDATDK3FV27K0B0QHGV 01CJ5W94SNSMSRF1J1616CMMCN
01CJ55PJW6YY7TCM2P5T33VS3G 01CJ5A7EK3M0ZF3ESY7E3J5VAM 01CJ5EX5F411QDQHV3KMHWA2H4 01CJ5KDS12032C3PQ8R2PSW8XV 01CJ5QWFH4SCFJ62AARKZBD132 01CJ5WB2KZX0WTKFY0NTMVRDQF
01CJ55RE6VBA2J0SGSW6HYQC1G 01CJ5A9A4J2GGXS83MFWXBN0FX 01CJ5EZ1BRYHHWD5DYXRD08Z81 01CJ5KFNM9BC9JWJNBVJ0EEJ28 01CJ5QYD2YK9YEX9PSZJ4VFP7M 01CJ5WCXKPF6BXR3RWCC6DATCB
01CJ55T9XXV5DAYRRX25G6NTAP 01CJ5AB8J09GCDKT5A872D6M57 01CJ5F0XD27P368QJB5JVRXCMZ 01CJ5KHHTRDDQBBTDV76EYHDGW 01CJ5R0937ZJDMGQF96ZZ39V6N 01CJ5WEVFCQ280P2SK4S0BVQSC
01CJ55W532VFKDYGQQD4SJEQG7 01CJ5AD7QEQCMC0QA51VNC32QF 01CJ5F2TRSF0PVSB2KPVVASKY8 01CJ5KKDNEHR2Q7B7NDXPER55S 01CJ5R240J56F49NTFZZ80QXF8 01CJ5WGQ1PV4WHKNM3HV5TTEMZ
01CJ55Y419P5FSV4MNSH1JAGBG 01CJ5AF3AVTV4MW1F2JKRCBJ48 01CJ5F4QH1MAT9G2VA6KRBHDDG 01CJ5KN8JR8XKM77Y6G8V6HWV3 01CJ5R3YZM687T6TPCGMKD405A 01CJ5WJJ7R5SYXCENSCD1FMESK
01CJ55ZZ56PSDKE2SQ6GZ66JGT 01CJ5AH0FMAMB054QN866R21DK 01CJ5F6K93EBBXZ664BD1TE46A 01CJ5KQ858T5ZF6EZN61M61NDV 01CJ5R5T2QQXFSNC9E5R502RN0 01CJ5WMDBRJTKPR6ZVNJS45H5T
01CJ561T7X3TA7KYTHE5VXHH41 01CJ5AJZA94W8H2AB20QX8R3F2 01CJ5F8ES475PK0M16DSPSNFDD 01CJ5KS3C8N6D4QTXAR0CWSRKE 01CJ5R7PVVDG3023S725XGR08A 01CJ5WP8B614MNT7ZCKTQTWZ5P
01CJ563QJ60HKTS7TG6C9989MW 01CJ5AMWB4S1CW2PH52RQFDDJP 01CJ5FAB6ZTN00CTERGYTXMBPZ 01CJ5KTYAEAM08BXHPGYKJ5R71 01CJ5R9NJV9WFJWVS15DXEGPZ9 01CJ5WR6970N2R48361RC1E9M4
01CJ565NFQF77EK0P7NR9MTPR1 01CJ5APQT4XABEF7GT7A1ER8MD 01CJ5FC70K2SXBWZZZGBDJQQXS 01CJ5KWSQ3W8SR62SWJXQS61GK 01CJ5RBGS5J9ENEQBPSJSGX0NV 01CJ5WT1HJJ0GJAB0F3ES2R4BR
01CJ567HM7R13A9WVKTSW1ZHDG 01CJ5ARN4YG00S0Z8XDBR2K0QQ 01CJ5FE2JNSEWFGZYF94MWJ8HK 01CJ5KYNHB9RV2NGNZ876ZHGA9 01CJ5RDFQHAM186AYNJXEDMJAE 01CJ5WW0AX8PM7XFVKMSWJG4SR
01CJ569CYMTW0N9Z83HSP3NB25 01CJ5ATN1BB0J10ZPM4GQE0CEH 01CJ5FG16HE72DHK64BN92RY86 01CJ5M0K3JHBK92MVRCE4FZ5SQ 01CJ5RFASVTJSNFBBKNS4VNE05 01CJ5WXVSSPD0K998Y3DZBVAR1
01CJ56BBPTMJKB4V2AYF8C8AST 01CJ5AWGP790Q6RXCQT1KNW08Y 01CJ5FJ06N00J5KT7RJB6EZSBW 01CJ5M2E00SPH1K5WA57NCWKSE 01CJ5RH5QGEQWPDKQYNB94DE9W 01CJ5WZPZ4S4PH435T1EFEM2AJ
01CJ56DARWDYXEJ70Y4AF35A9C 01CJ5AYC9RWWPZDF87JXH874BE 01CJ5FKW0CK2M86RYW7ZYK4TSQ 01CJ5M48Y39PTKX246HHNEPSSN 01CJ5RK34JQD4V35ZSBFP321KQ 01CJ5X1HZQRFWQEB2B7MZ3X2MB
01CJ56FANKEZFWDGTM60F9A0XG 01CJ5B0AP0J5G1CD49HBVFT2KS 01CJ5FNQXKFQ1JM5HC4NYGK9BM 01CJ5M656B4SWK0DC0Z9EQGKF8 01CJ5RMYZ5WR9FJ18165ED7BPF 01CJ5X3CZ1E8TSAA21020M5JVD
01CJ56H8V6Z6RXPM8AZPWTNWSM 01CJ5B26AGDA9JV638XMV9EV53 01CJ5FQKEYZXN6Z05K522AKHWN 01CJ5M80A08MWEJPSWA7VKP29J 01CJ5RPTW2RE2HEPCP0EYZT9DG 01CJ5X5B5NZTGMZPS7YCQKMK1X
01CJ56K3WN6DP69CZEP36YJKF7 01CJ5B44SXPNDEHZGGKVYJ7NVF 01CJ5FSJC097N3CYVHWG8HTD9H 01CJ5M9YNMRM7DYEQSTXYHATTE 01CJ5RRQQV09JKE2PZ9PQ7HQHE 01CJ5X76HHQ4NN2NQRGGJNPVKA
01CJ56MZZ638P31GJX3NS9SW4N 01CJ5B61F4PER905ZA1DYCSNZZ 01CJ5FVF3CMHVF61N2R68VAZDV 01CJ5MBV1EGFFXNQ5VH1YZSME7 01CJ5RTJS6BAFK9K4SVKMVXXP2 01CJ5X93CPC8Q5VV3G8ZZP0E6B
01CJ56PXGRJMW86HKFARVK68VP 01CJ5B7XDN3BVZ9R1PAHG56ZRK 01CJ5FXBPR4VPCNHQBM2YVJBGS 01CJ5MDP0ANFVJ08R5W4K36T32 01CJ5RWEQH8ETTG1Y6YB28XHKG 01CJ5XAZQNR9N7JCHEW70KCPV9
01CJ56RSF2GASZ1AV89F6BY042 01CJ5B9T0NP2JYCTGHF2CVSMG2 01CJ5FZAHZN74CZ15CFZ6SVZB0 01CJ5MFH16E9ANREJYXDK53FWD 01CJ5RY9RT9J66A8N9MMXVVZZN 01CJ5XCYQ12MCNNNP5PZV4BPMV
01CJ56TNF20XTTH47YB7XBM4EJ 01CJ5BBR6PEZ7SRVCB3KFD936H 01CJ5G18KVA0VD0D2WHQGWS1QJ 01CJ5MHCD3SGPR7MNPKRDEDFE9 01CJ5S04R3XQ2A82TXX1B55ET9 01CJ5XESPDSFC11B377KJQHEP6
01CJ56WKNYP5KTE6WW4N3WFQG2 01CJ5BDKW448AN4ZM09H421TWD 01CJ5G3479W1FW0WR4AHQ2H8Y9 01CJ5MKBHX4KRVJ8Q6C28S4D3X 01CJ5S24M853AQDFQ4EXWMCJA0 01CJ5XGMPJF29EP4JS72B453B8
01CJ56YENRNJQRBX3EFV3WYZHC 01CJ5BFFYPKBPPS67Y8CE8YGED 01CJ5G508YAPWTS0P6ZN3WE6D3 01CJ5MN6HN9C534KF2XJP1RWKC 01CJ5S3ZKWKBSEWMSJZS0462WH 01CJ5XJFY353M0CW4YCXJAS3EK
01CJ570A8YMECEC4ZDZD2SDFNE 01CJ5BHCCT9AN0FR7CPH5XERDM 01CJ5G6VXEB32WQE1RVJW4MHX3 01CJ5MQ4BQ8AC2ZSHNGSRG0GZC 01CJ5S5V3GXN5KR8Y5KGWHXG30 01CJ5XMB43A4G1K64M754HFHCM
01CJ5728CMSM828BZBJCJCTJYD 01CJ5BKCTM0M7P3TKHE4E0K6PW 01CJ5G8RM31V2CDQ6818PTJ503 01CJ5MRZEW6D47XGFAKQW8KQK2 01CJ5S7S4S5E4XG9TPMD81H4JZ 01CJ5XP634WP7G1QEG1QWJ6DFS
01CJ5743ZBZCWB8CK243XW3TTA 01CJ5BNC6FC4VNSHRPB5H5ZHQM 01CJ5GAME0T059G2T23TNM87TH 01CJ5MTWN7WVGFBPFESEAVRVQ2 01CJ5S9PVT6WEC5SPCNV6MFBB4 01CJ5XR3G44FC9C7NXN4AB9Y7Q
01CJ576296PEMZMB75NHEYGG2F 01CJ5BQ7Z4MJP2D5Q0X1V2NZMF 01CJ5GCH37FCT0AG4Y8VAZCRW5 01CJ5MWQT21Y6PHPQWVMJNEHYM 01CJ5SBHV7F10S4H63MNZFE4MK 01CJ5XT14X52EEAZD15QR7YPBT
01CJ5781NM1X4M1Q6HWPW6WG7B 01CJ5BS5256S0MJREP2YG91WKV 01CJ5GECVPPMYJ1K4DFQ2NZXZ2 01CJ5MYJVK48TA98QTHXZVTN52 01CJ5SDE3JXNH9DPTR8T7NBYJ1 01CJ5XVXB90DXHED0DY0Y4RKZC
01CJ579X6XSP73KAMRV2DYRVFF 01CJ5BV6PZNKRCV7TKSSVD0DEP 01CJ5GG8G5N3W73JC8VN1ZVD5C 01CJ5N0FFXANSBCFF15891EV43 01CJ5SFCSXKW5KD2JV901ZFP1X 01CJ5XXV435NRC1RN9PV6V2K80
01CJ57BS2BWBY51H8YQXY9NM3B 01CJ5BX5J1T75MQ2K7SCKRJH5F 01CJ5GJ66YD82134NEVWS57FVW 01CJ5N2ANZGFBWJR9WP0G8NVMF 01CJ5SH81BHXN6QHVFXNS0PPMX 01CJ5XZQ3V0KD7P1KTVZQXMTQC
01CJ57DNKWP6R257T2WF2BW4JD 01CJ5BZ31ZFZM3VWQ7EWBRNHH5 01CJ5GM4KYE1XF7EGZ239A1QJV 01CJ5N46NB0EXM6A9HDTSYHQA8 01CJ5SK31GGXS0JXT2MGD94VYY 01CJ5Y1J6EJMEQTVFGKZTF997H
01CJ57FN31PJPFCB28E0DJZFZD 01CJ5C13MZSS5CKNS17YNB90EP 01CJ5GP11WRR1B2WHJ2C5H394N 01CJ5N628GNQTTSF2W38VXKSCF 01CJ5SMY2X8E2Y0K7DWHZDAXE7 01CJ5Y3D51HKM8XXA6AVAE6ZDF
01CJ57HH2BK30GSEK29803SQGH 01CJ5C2ZVC06A47TSC9WVA4AGG 01CJ5GQWQPN26F5ZF0AJ3922V6 01CJ5N7X72015A8FSQMFBQRN3A 01CJ5SPW30DWW7FRPTDK7818DA lock
01CJ57KCFXHB5HM36S85X3YAZP 01CJ5C4W5ZD863V5MAS4G78JW9 01CJ5GSRN9J4Y6NVG82112HKCR 01CJ5N9RB8WX5NEMDVFVVHZNNQ 01CJ5SRQ715G4EYWATG37E9SC4 wal
01CJ57NA6B8V9K865FDNJEZJJQ 01CJ5C6STQPWJXT895FDA6MH0V 01CJ5GVNJM9H3TVRQ4K58R1ACM 01CJ5NBK9BMWH5YRS0EK9AV92D 01CJ5STNMFHJSKV16FQDMDC6KZ
01CJ57Q5KYXS3SDZ115KDSW23A 01CJ5C8PT24XNY32PDZ9EVRZ4D 01CJ5GXHGJSGX8FFV2MHRJBBZ8 01CJ5NDEY4E1APQ3RZ3KWQ2BAD 01CJ5SWGNMMP46JK2TYMCX4VY1
01CJ57S52CGE2S89JK1N0DRKNR 01CJ5CAM39QA3T7BD0VYSNHQDZ 01CJ5GZFBHXQ3YG3CZENGPR2E7 01CJ5NFC929EE2FV02QCF4D5BH 01CJ5SYBY7Z43SF3Y0Z30E2YPG
[ramcoazure@HCMDEMAZIND01 data]$ cd ..
[ramcoazure@HCMDEMAZIND01 prometheus-2.3.1.linux-amd64]$ ./tsdb scan -h ./data/
[data/01CHF863S201TBMD667KNN3TE3 data/01CHGZ3YMXBMM09NDKTJA2P2BH data/01CHJWXF056J7YPX5ZP2SSYGW4 data/01CHMTQ21WA532AY7P68XZ02F8 data/01CHPRGN1E7W6VYQV032B95ZAA data/01CHRPA0DHK5GTW7GV65GKTQC0 data/01CHTM3JCMQZQ5KJ66G8H5H4GE data/01CHWHX2Y0CXWFWYD7C41MH7YB data/01CHYFPPFQASSQ6N649EN75803 data/01CJ0DG8HAPPYAFDPY4KCH6ATM data/01CJ2B9QHK9Z4909NNNB2S9SDR data/01CJ493678EFRDSFZNPQDHXNAP data/01CJ4936XK9AE031X8133MXDVT data/01CJ4FYXFG0V5VM0G2FQXT6G3A data/01CJ4XPD0BPWZKPJNW08JPC0DF data/01CJ4XPDNXD5AWTB44DZ0QTGQG data/01CJ54J496SFZ7QDHSSZC03G32 data/01CJ54J5V66TX4YSMV3RDMK917 data/01CJ54JBTCDV83661ATDJFN034 data/01CJ54JGRPJ55YMZ0WPJA5WTXH data/01CJ54JVRXSY42VH8EGNKWYPDY data/01CJ54KBNRWJ7912V1YR9TDXT1 data/01CJ54MF1A7DR8QJ9963KDE65E data/01CJ54PA2SB46CAT9CG4HDS74T data/01CJ54R573TKF597WTWMYAQWP7 data/01CJ54T0V70HZ68QQM3NXXAS2X data/01CJ54W07ZXFEC2KGEJ97C490G data/01CJ54Y0BA2FKGWXN4EFT8SHSV data/01CJ54ZVX31VSS5Y91820DFVPM data/01CJ551V3TC0NPBVD4DCC9B7RK data/01CJ553QR9543PBZK7PXKTQ9SE data/01CJ555JRCNMBVSEQV67FVYJRH data/01CJ557H26AXFVJD14GEE4EW61 data/01CJ559C32XHQ6VQCSQGMQ47K7 data/01CJ55BB52GYD23A7P3VF2Z3BP data/01CJ55D62K50AS5GHGYKB7980B data/01CJ55F11B9YN0ZHJ6VTMGDFX2 data/01CJ55GW6TQ63JKJB5K66P5ZA6 data/01CJ55JRYGV6FSSWZY0D468ZTA data/01CJ55MM20CT1KC8FKKEMR2N3E data/01CJ55PJW6YY7TCM2P5T33VS3G data/01CJ55RE6VBA2J0SGSW6HYQC1G data/01CJ55T9XXV5DAYRRX25G6NTAP data/01CJ55W532VFKDYGQQD4SJEQG7 data/01CJ55Y419P5FSV4MNSH1JAGBG data/01CJ55ZZ56PSDKE2SQ6GZ66JGT data/01CJ561T7X3TA7KYTHE5VXHH41 data/01CJ563QJ60HKTS7TG6C9989MW data/01CJ565NFQF77EK0P7NR9MTPR1 data/01CJ567HM7R13A9WVKTSW1ZHDG data/01CJ569CYMTW0N9Z83HSP3NB25 data/01CJ56BBPTMJKB4V2AYF8C8AST data/01CJ56DARWDYXEJ70Y4AF35A9C data/01CJ56FANKEZFWDGTM60F9A0XG data/01CJ56H8V6Z6RXPM8AZPWTNWSM data/01CJ56K3WN6DP69CZEP36YJKF7 data/01CJ56MZZ638P31GJX3NS9SW4N data/01CJ56PXGRJMW86HKFARVK68VP data/01CJ56RSF2GASZ1AV89F6BY042 data/01CJ56TNF20XTTH47YB7XBM4EJ data/01CJ56WKNYP5KTE6WW4N3WFQG2 data/01CJ56YENRNJQRBX3EFV3WYZHC data/01CJ570A8YMECEC4ZDZD2SDFNE data/01CJ5728CMSM828BZBJCJCTJYD data/01CJ5743ZBZCWB8CK243XW3TTA data/01CJ576296PEMZMB75NHEYGG2F data/01CJ5781NM1X4M1Q6HWPW6WG7B data/01CJ579X6XSP73KAMRV2DYRVFF data/01CJ57BS2BWBY51H8YQXY9NM3B data/01CJ57DNKWP6R257T2WF2BW4JD data/01CJ57FN31PJPFCB28E0DJZFZD data/01CJ57HH2BK30GSEK29803SQGH data/01CJ57KCFXHB5HM36S85X3YAZP data/01CJ57NA6B8V9K865FDNJEZJJQ data/01CJ57Q5KYXS3SDZ115KDSW23A data/01CJ57S52CGE2S89JK1N0DRKNR data/01CJ57V0D8SZRCPT3YECRWF4BA data/01CJ57WVRE8CQRXFVE3HPFGX25 data/01CJ57YR6WXNHJY0SD4X9ZBT6X data/01CJ580M77MS3BC4FZBHNKFQ8T data/01CJ582JPGBJYK9RRKQDSZ5553 data/01CJ584FM8TPV3JSYDC8N4QH2T data/01CJ586E43A1WY89D7DSYJB8C7 data/01CJ588A9HENY3D3BPKDRC14QE data/01CJ58A7XB8VS778F0PD45NABZ data/01CJ58C7WY07MDNXSQ4EJF2RP3 data/01CJ58E6KRCK2VP3V2KYPPEARM data/01CJ58G6KRY94RRJST7TP2WBFS data/01CJ58J1ZQETE7WBW5BSKV7XQH data/01CJ58KXBG6AVTWD7E202403BN data/01CJ58NTFCSRSRG0ZA62XYS9SN data/01CJ58QPTR6DMZ511E2D975ZZP data/01CJ58SMQ3515RYFAV04CGAMNQ data/01CJ58VGKDZZMXWZYH7MX2QZH3 data/01CJ58XBX6Z1MFPKR6700G9ZYX data/01CJ58Z8HH42M791KVHQ4XJ1KF data/01CJ591476RHHKA8JKNPDFREQ3 data/01CJ59346E341E613QPX90FH64 data/01CJ59500WKW82ZEN5CP6BPPXH data/01CJ596XWEPX1QY6KV6P9C9B2N data/01CJ598SAY8QNEZA8FZ3CA2ZWP data/01CJ59AMQKSBBM9CG1HPCNM042 data/01CJ59CHJ2FG7734383Y64ZXXZ data/01CJ59EE9HCAVKQSZSSPJR095X data/01CJ59GDA1HQY6WN9CSKY1K2QE data/01CJ59JBVW66J3WYX79TQ0F1N5 data/01CJ59M7A2WZMGK7KMGY4VYM9P data/01CJ59P2ZQE330RF08V2AT4KSN data/01CJ59QZRV5YBQJV99T2XCEJSJ data/01CJ59SVBD0N6C3BW32W2YYJ58 data/01CJ59VPZ28VYDQNF7BEWZXP4S data/01CJ59XKX6S1DHF2X6N108F026 data/01CJ59ZRJHAJ1QNGXWZVC7KJQ6 data/01CJ5A1N5M6WHZB9VVNF4KKCFQ data/01CJ5A3J8JAPWMCEVMDQH3JKPW data/01CJ5A5FJD2VAKEYM9BBE4K3F6 data/01CJ5A7EK3M0ZF3ESY7E3J5VAM data/01CJ5A9A4J2GGXS83MFWXBN0FX data/01CJ5AB8J09GCDKT5A872D6M57 data/01CJ5AD7QEQCMC0QA51VNC32QF data/01CJ5AF3AVTV4MW1F2JKRCBJ48 data/01CJ5AH0FMAMB054QN866R21DK data/01CJ5AJZA94W8H2AB20QX8R3F2 data/01CJ5AMWB4S1CW2PH52RQFDDJP data/01CJ5APQT4XABEF7GT7A1ER8MD data/01CJ5ARN4YG00S0Z8XDBR2K0QQ data/01CJ5ATN1BB0J10ZPM4GQE0CEH data/01CJ5AWGP790Q6RXCQT1KNW08Y data/01CJ5AYC9RWWPZDF87JXH874BE data/01CJ5B0AP0J5G1CD49HBVFT2KS data/01CJ5B26AGDA9JV638XMV9EV53 data/01CJ5B44SXPNDEHZGGKVYJ7NVF data/01CJ5B61F4PER905ZA1DYCSNZZ data/01CJ5B7XDN3BVZ9R1PAHG56ZRK data/01CJ5B9T0NP2JYCTGHF2CVSMG2 data/01CJ5BBR6PEZ7SRVCB3KFD936H data/01CJ5BDKW448AN4ZM09H421TWD data/01CJ5BFFYPKBPPS67Y8CE8YGED data/01CJ5BHCCT9AN0FR7CPH5XERDM data/01CJ5BKCTM0M7P3TKHE4E0K6PW data/01CJ5BNC6FC4VNSHRPB5H5ZHQM data/01CJ5BQ7Z4MJP2D5Q0X1V2NZMF data/01CJ5BS5256S0MJREP2YG91WKV data/01CJ5BV6PZNKRCV7TKSSVD0DEP data/01CJ5BX5J1T75MQ2K7SCKRJH5F data/01CJ5BZ31ZFZM3VWQ7EWBRNHH5 data/01CJ5C13MZSS5CKNS17YNB90EP data/01CJ5C2ZVC06A47TSC9WVA4AGG data/01CJ5C4W5ZD863V5MAS4G78JW9 data/01CJ5C6STQPWJXT895FDA6MH0V data/01CJ5C8PT24XNY32PDZ9EVRZ4D data/01CJ5CAM39QA3T7BD0VYSNHQDZ data/01CJ5CCK52FZNA1JYSNZ89W9D7 data/01CJ5CEJDTM8REY02X40T4BGSV data/01CJ5CGF5E3EKMWJ25XQ7BZ5DY data/01CJ5CJBR540JJGPG7RW8KDJJF data/01CJ5CM7HF7BW5F34S9JB292VK data/01CJ5CP4FWR6C7XJ162Z85JABG data/01CJ5CR244KCP4T4VK5DHE1W5G data/01CJ5CT04S3F185CEXABE5MYA2 data/01CJ5CVYSC41MB50CDP6ZSS6E1 data/01CJ5CXZNY7RFFNXEJM82S44K8 data/01CJ5CZYWCRM0F8DQP9AT3TWMJ data/01CJ5D269KP079D0CRVKQ9RCJS data/01CJ5D45V9NAR0NJKRFE49346A data/01CJ5D63B0GSBB9Y0PG8XSN446 data/01CJ5D80ZZ83AQHHT677VM7CDZ data/01CJ5D9X01PQ2SHV4J72GR51V5 data/01CJ5DBVJC2DPKNWGJSD3MYJTS data/01CJ5DDV4N0600G1F6BDXK41PY data/01CJ5DFVPAB4JBXHENVC9HE6P1 data/01CJ5DK6HNCH7J3BV866DRSR9N data/01CJ5DN36VB72VPH2AC188G2JZ data/01CJ5DQ3QSV6CG2DHKJ9S66AJ0 data/01CJ5DS05Z2PCKCTEZ8DDQCT62 data/01CJ5DTX6D50SFJFAKTA8KAVQ0 data/01CJ5DXB5QQGGYX7XC40ZXJGCY data/01CJ5DZ7DS9AM890WXBMQ1KVQM data/01CJ5E15CXKH2QYF6PR5N9FNBA data/01CJ5E31TA67W6968J6F4AM6FX data/01CJ5E6C43Y456FS95PQ6VVQ68 data/01CJ5E888HECNC32ERF466DRT5 data/01CJ5EA3XNH6A2ZTCKK1Z0D5R9 data/01CJ5EC2S3H7TFHZKSSF1Q1SSR data/01CJ5EE0BR6DRH3B9RJ2C0P3TA data/01CJ5EFW0CTCWA31406J19MD09 data/01CJ5EHQH43ZBWF0MX0RVCSNXD data/01CJ5EKMG22JHG7TGG3X59J4T9 data/01CJ5ENMCZT7MNNVJWF4ZWYBZ7 data/01CJ5EQGDRJAS9BZ8ZD48C3SPX data/01CJ5ESDX8VPYA1RQMPEQK21QQ data/01CJ5EV9Z2FCV30T7F59TD5CNG data/01CJ5EX5F411QDQHV3KMHWA2H4 data/01CJ5EZ1BRYHHWD5DYXRD08Z81 data/01CJ5F0XD27P368QJB5JVRXCMZ data/01CJ5F2TRSF0PVSB2KPVVASKY8 data/01CJ5F4QH1MAT9G2VA6KRBHDDG data/01CJ5F6K93EBBXZ664BD1TE46A data/01CJ5F8ES475PK0M16DSPSNFDD data/01CJ5FAB6ZTN00CTERGYTXMBPZ data/01CJ5FC70K2SXBWZZZGBDJQQXS data/01CJ5FE2JNSEWFGZYF94MWJ8HK data/01CJ5FG16HE72DHK64BN92RY86 data/01CJ5FJ06N00J5KT7RJB6EZSBW data/01CJ5FKW0CK2M86RYW7ZYK4TSQ data/01CJ5FNQXKFQ1JM5HC4NYGK9BM data/01CJ5FQKEYZXN6Z05K522AKHWN data/01CJ5FSJC097N3CYVHWG8HTD9H data/01CJ5FVF3CMHVF61N2R68VAZDV data/01CJ5FXBPR4VPCNHQBM2YVJBGS data/01CJ5FZAHZN74CZ15CFZ6SVZB0 data/01CJ5G18KVA0VD0D2WHQGWS1QJ data/01CJ5G3479W1FW0WR4AHQ2H8Y9 data/01CJ5G508YAPWTS0P6ZN3WE6D3 data/01CJ5G6VXEB32WQE1RVJW4MHX3 data/01CJ5G8RM31V2CDQ6818PTJ503 data/01CJ5GAME0T059G2T23TNM87TH data/01CJ5GCH37FCT0AG4Y8VAZCRW5 data/01CJ5GECVPPMYJ1K4DFQ2NZXZ2 data/01CJ5GG8G5N3W73JC8VN1ZVD5C data/01CJ5GJ66YD82134NEVWS57FVW data/01CJ5GM4KYE1XF7EGZ239A1QJV data/01CJ5GP11WRR1B2WHJ2C5H394N data/01CJ5GQWQPN26F5ZF0AJ3922V6 data/01CJ5GSRN9J4Y6NVG82112HKCR data/01CJ5GVNJM9H3TVRQ4K58R1ACM data/01CJ5GXHGJSGX8FFV2MHRJBBZ8 data/01CJ5GZFBHXQ3YG3CZENGPR2E7 data/01CJ5H1B03HZRE18HHN7TRW7XP data/01CJ5H36KAH6N28HEJQJQEABDH data/01CJ5H52B7NNK5QV1PX0J1N7V4 data/01CJ5H713PPS77S8JXVWT6WWPQ data/01CJ5H8ZHAF912PQPGFRG7AS6Y data/01CJ5HAVXZTMK9XEF15Y7K9WV0 data/01CJ5HCRGG4PYVJRKRJ56AFQF0 data/01CJ5HEQRR8XGXPZZ8HD4YWXE0 data/01CJ5HGQ11NA03BCEPYW46DF34 data/01CJ5HJKQHBD4XQHQEJ6HAHYEB data/01CJ5HMFM3PZJC2M4HD7TAVBV2 data/01CJ5HPD69QTRQW6MJRS5SXDZR data/01CJ5HR9GPZ6D2JJQTJP5RV739 data/01CJ5HT91PA53VHNZQ322HGJ3N data/01CJ5HW5R3WVFWYAHQ5GTC4ZJZ data/01CJ5HY2VR6YC8N1AWWCNV75YQ data/01CJ5HZZ4AZJFJBNEJFA3WCAJ7 data/01CJ5J1W43ZR6SY4VEJRD9GHF7 data/01CJ5J3QWR5Y3XDK54B0FDT0R9 data/01CJ5J5RWKJ797302SFFZ534MD data/01CJ5J7NSZPGTZ2RKMWMQX47FP data/01CJ5J9HP7DKTJD5SYBFW3KQHE data/01CJ5JBE0RT66F9QRXMB1H6V89 data/01CJ5JDA0N9RK9K2J50NM2S217 data/01CJ5JF6SMG7VDSXCQMHFYV0P4 data/01CJ5JH39TX5M22V6SYBCBP2KG data/01CJ5JK1W5XN67VXS5FFVKH6RG data/01CJ5JMYW2SB5P597M06J8T1HJ data/01CJ5JPXC3S07Y3YXFM5YM0YH2 data/01CJ5JRS371SM8CKQJAC0HNKH4 data/01CJ5JTNQ0MRJ1BGYRYS1PBNEC data/01CJ5JWHHDZQWDZHRM58EFVVMS data/01CJ5JYDC53Z9B9ZAJKK1KH0W7 data/01CJ5K0BG5F02FBK7S08W4FRTZ data/01CJ5K29XCN00Q94A348MRAWYF data/01CJ5K45WP64RWCAB37H7P8C7N data/01CJ5K6232RWJQ9QYS1DW3DMV4 data/01CJ5K808CGTNS03TTQ4T79PVH data/01CJ5K9W1Z3NCN4YHDCGTDXBXP data/01CJ5KBVFH4H4JS1675CXSGSFA data/01CJ5KDS12032C3PQ8R2PSW8XV data/01CJ5KFNM9BC9JWJNBVJ0EEJ28 data/01CJ5KHHTRDDQBBTDV76EYHDGW data/01CJ5KKDNEHR2Q7B7NDXPER55S data/01CJ5KN8JR8XKM77Y6G8V6HWV3 data/01CJ5KQ858T5ZF6EZN61M61NDV data/01CJ5KS3C8N6D4QTXAR0CWSRKE data/01CJ5KTYAEAM08BXHPGYKJ5R71 data/01CJ5KWSQ3W8SR62SWJXQS61GK data/01CJ5KYNHB9RV2NGNZ876ZHGA9 data/01CJ5M0K3JHBK92MVRCE4FZ5SQ data/01CJ5M2E00SPH1K5WA57NCWKSE data/01CJ5M48Y39PTKX246HHNEPSSN data/01CJ5M656B4SWK0DC0Z9EQGKF8 data/01CJ5M80A08MWEJPSWA7VKP29J data/01CJ5M9YNMRM7DYEQSTXYHATTE data/01CJ5MBV1EGFFXNQ5VH1YZSME7 data/01CJ5MDP0ANFVJ08R5W4K36T32 data/01CJ5MFH16E9ANREJYXDK53FWD data/01CJ5MHCD3SGPR7MNPKRDEDFE9 data/01CJ5MKBHX4KRVJ8Q6C28S4D3X data/01CJ5MN6HN9C534KF2XJP1RWKC data/01CJ5MQ4BQ8AC2ZSHNGSRG0GZC data/01CJ5MRZEW6D47XGFAKQW8KQK2 data/01CJ5MTWN7WVGFBPFESEAVRVQ2 data/01CJ5MWQT21Y6PHPQWVMJNEHYM data/01CJ5MYJVK48TA98QTHXZVTN52 data/01CJ5N0FFXANSBCFF15891EV43 data/01CJ5N2ANZGFBWJR9WP0G8NVMF data/01CJ5N46NB0EXM6A9HDTSYHQA8 data/01CJ5N628GNQTTSF2W38VXKSCF data/01CJ5N7X72015A8FSQMFBQRN3A data/01CJ5N9RB8WX5NEMDVFVVHZNNQ data/01CJ5NBK9BMWH5YRS0EK9AV92D data/01CJ5NDEY4E1APQ3RZ3KWQ2BAD data/01CJ5NFC929EE2FV02QCF4D5BH data/01CJ5NH77VACYZV4E54DMZGD6A data/01CJ5NK4XN165Z8PEC80EPBJBT data/01CJ5NN02XSQWJ0ABKERCA3TAA data/01CJ5NPV2PST0QS1VRK2MZX3XH data/01CJ5NRSZMVXWJ4S1EQV67P3ZP data/01CJ5NTMVK0E67D9NMTPJY6NY4 data/01CJ5NWFXV2EXSRPW679SKFDN0 data/01CJ5NYF7YH0XDDZBC419WW3M3 data/01CJ5P0E2HWVM8GNMT7EFRX2H3 data/01CJ5P296BBVN8BG1H6P36AJCE data/01CJ5P448ZWR8Z98GHRTDD7XFE data/01CJ5P631960H6PTYNWQKS20HM data/01CJ5P7Y1BANGJXDKK9PJAA7DD data/01CJ5P9WV0TVEH2TA57DH2K0XF data/01CJ5PBWDGPFKAPNWS36W2SRN1 data/01CJ5PDRKTRTFK7X10PZESDWNQ data/01CJ5PFKK9N6MZGBW2H1MYHEHR data/01CJ5PHEJWVFM1YQ0XSZ2F4Z8C data/01CJ5PKBV7GF53SFVQ26JR00EE data/01CJ5PN6SF1XE2MQDA1QZSBYG1 data/01CJ5PQ1R16CTMS1QXMQ59JPGV data/01CJ5PRZ1DJFZ01XNWM9SQZGGE data/01CJ5PTTMCACVEZ86M79SN4BDG data/01CJ5PWNMZNZFZG3DCCVWQ0APW data/01CJ5PYGMHRCCHJ37PMRWDVTPS data/01CJ5Q0C46HDHWXB7W2JVJT1QZ data/01CJ5Q277AK5SM6HJD70MVVPPQ data/01CJ5Q425R66Z7CWF486BYS11K data/01CJ5Q5X5ZQ3QNZBZ050XKPPPS data/01CJ5Q7TDWSH5FX7KEA5HPXSF8 data/01CJ5Q9S9HNHMV7QRF3YQJBK9R data/01CJ5QBNQ2537BWE8Z15HGTRMZ data/01CJ5QDGQEB2X9BA9RH3RTR26A data/01CJ5QFBNTZY7PPPWDZSSBKD4D data/01CJ5QH6NZC65DN0F7W1B60RW6 data/01CJ5QK205N91E11FQTESZSMFA data/01CJ5QN0KWZ4ZSYCQYRQEQDTFZ data/01CJ5QPVHPSYK0P4GKVFKJP6FM data/01CJ5QRQQTY8A72R3F1MV3QSM2 data/01CJ5QTMDATDK3FV27K0B0QHGV data/01CJ5QWFH4SCFJ62AARKZBD132 data/01CJ5QYD2YK9YEX9PSZJ4VFP7M data/01CJ5R0937ZJDMGQF96ZZ39V6N data/01CJ5R240J56F49NTFZZ80QXF8 data/01CJ5R3YZM687T6TPCGMKD405A data/01CJ5R5T2QQXFSNC9E5R502RN0 data/01CJ5R7PVVDG3023S725XGR08A data/01CJ5R9NJV9WFJWVS15DXEGPZ9 data/01CJ5RBGS5J9ENEQBPSJSGX0NV data/01CJ5RDFQHAM186AYNJXEDMJAE data/01CJ5RFASVTJSNFBBKNS4VNE05 data/01CJ5RH5QGEQWPDKQYNB94DE9W data/01CJ5RK34JQD4V35ZSBFP321KQ data/01CJ5RMYZ5WR9FJ18165ED7BPF data/01CJ5RPTW2RE2HEPCP0EYZT9DG data/01CJ5RRQQV09JKE2PZ9PQ7HQHE data/01CJ5RTJS6BAFK9K4SVKMVXXP2 data/01CJ5RWEQH8ETTG1Y6YB28XHKG data/01CJ5RY9RT9J66A8N9MMXVVZZN data/01CJ5S04R3XQ2A82TXX1B55ET9 data/01CJ5S24M853AQDFQ4EXWMCJA0 data/01CJ5S3ZKWKBSEWMSJZS0462WH data/01CJ5S5V3GXN5KR8Y5KGWHXG30 data/01CJ5S7S4S5E4XG9TPMD81H4JZ data/01CJ5S9PVT6WEC5SPCNV6MFBB4 data/01CJ5SBHV7F10S4H63MNZFE4MK data/01CJ5SDE3JXNH9DPTR8T7NBYJ1 data/01CJ5SFCSXKW5KD2JV901ZFP1X data/01CJ5SH81BHXN6QHVFXNS0PPMX data/01CJ5SK31GGXS0JXT2MGD94VYY data/01CJ5SMY2X8E2Y0K7DWHZDAXE7 data/01CJ5SPW30DWW7FRPTDK7818DA data/01CJ5SRQ715G4EYWATG37E9SC4 data/01CJ5STNMFHJSKV16FQDMDC6KZ data/01CJ5SWGNMMP46JK2TYMCX4VY1 data/01CJ5SYBY7Z43SF3Y0Z30E2YPG data/01CJ5T0719MR6K00SR3422RZQN data/01CJ5T2205DQJQQSWBTA9R4HA7 data/01CJ5T3X07N6V6M958HGBHD4AV data/01CJ5T5QYFXKJ2X4SXVM3DB1KR data/01CJ5T7JX7M8H6M6D2VWAREVXJ data/01CJ5T9E1FFV9JCFAEW81KCQKB data/01CJ5TBA1W66XYJYVQ0RYJFPRH data/01CJ5TD5ZF8MCQNWJ5PGFMK7M6 data/01CJ5TF0Y8TAT9B0T37ERMC5NZ data/01CJ5TGVW0MCJD45MCPGWMBYB8 data/01CJ5TJTDRB579KS6ZZM8JFRTJ data/01CJ5TMNE7X929HE32R7WXXR2G data/01CJ5TPGGF3KPV26B7JA85VF4W data/01CJ5TREMWEHZDHETXND7EEYRE data/01CJ5TT9KTR6ZA046FX5APWA89 data/01CJ5TW4QBNDMDEMXYV7K529GP data/01CJ5TXZMEPDY4EX6R8GV80P4Y data/01CJ5TZY1CY7Z3AAH2NM4QDPG7 data/01CJ5V1S0EK00A08WG3W9K6BEQ data/01CJ5V3PZVQC5VZTKMCK5KTED4 data/01CJ5V5J1P9SNBSKG45MXG7HDC data/01CJ5V7CYTHJW1HBJHJAA2KC2G data/01CJ5V9B9543GCZN2CT7K713V8 data/01CJ5VB9EJABPCFQSB7HYSPXAV data/01CJ5VD4F3M713BJKM81JTKZGC data/01CJ5VEZZJE01KMPG3Y54Q3Q06 data/01CJ5VGV8NDPDZ8FVRH43QRKPE data/01CJ5VJPB6XA44AR5RPPQ4651G data/01CJ5VMJVNK1M0XKMGH8KNA6WS data/01CJ5VPFXYWS8B8FZH6BY9EANT data/01CJ5VRAY5ZAD2WJZ7SRM22X2V data/01CJ5VT64X07ZH6CEK1D589AMH data/01CJ5VW14AS2D38S1X0JBXDB3E data/01CJ5VXW5QQ5S4HEBH5MDEV6BA data/01CJ5VZQ8H0EE3MN54ZR5D27MP data/01CJ5W1JGQBP361SBQGRDTBVP3 data/01CJ5W3GFC8VJ032NY3KSR266V data/01CJ5W5E71F2AADM41VFXF5Y35 data/01CJ5W79KNAGMYBM5KG3J0AMJF data/01CJ5W94SNSMSRF1J1616CMMCN data/01CJ5WB2KZX0WTKFY0NTMVRDQF data/01CJ5WCXKPF6BXR3RWCC6DATCB data/01CJ5WEVFCQ280P2SK4S0BVQSC data/01CJ5WGQ1PV4WHKNM3HV5TTEMZ data/01CJ5WJJ7R5SYXCENSCD1FMESK data/01CJ5WMDBRJTKPR6ZVNJS45H5T data/01CJ5WP8B614MNT7ZCKTQTWZ5P data/01CJ5WR6970N2R48361RC1E9M4 data/01CJ5WT1HJJ0GJAB0F3ES2R4BR data/01CJ5WW0AX8PM7XFVKMSWJG4SR data/01CJ5WXVSSPD0K998Y3DZBVAR1 data/01CJ5WZPZ4S4PH435T1EFEM2AJ data/01CJ5X1HZQRFWQEB2B7MZ3X2MB data/01CJ5X3CZ1E8TSAA21020M5JVD data/01CJ5X5B5NZTGMZPS7YCQKMK1X data/01CJ5X76HHQ4NN2NQRGGJNPVKA data/01CJ5X93CPC8Q5VV3G8ZZP0E6B data/01CJ5XAZQNR9N7JCHEW70KCPV9 data/01CJ5XCYQ12MCNNNP5PZV4BPMV data/01CJ5XESPDSFC11B377KJQHEP6 data/01CJ5XGMPJF29EP4JS72B453B8 data/01CJ5XJFY353M0CW4YCXJAS3EK data/01CJ5XMB43A4G1K64M754HFHCM data/01CJ5XP634WP7G1QEG1QWJ6DFS data/01CJ5XR3G44FC9C7NXN4AB9Y7Q data/01CJ5XT14X52EEAZD15QR7YPBT data/01CJ5XVXB90DXHED0DY0Y4RKZC data/01CJ5XXV435NRC1RN9PV6V2K80 data/01CJ5XZQ3V0KD7P1KTVZQXMTQC data/01CJ5Y1J6EJMEQTVFGKZTF997H data/01CJ5Y3D51HKM8XXA6AVAE6ZDF]
Overlaping blocks.
Deleting these will remove all data in the listed time range.
overlapping blocks : 18/07/11 06:00-08:00 18/07/11

Block 01CJ4XPDNXD5AWTB44DZ0QTGQG contains highest samples count and is ommited from the deletion list!

overlapping blocks : 18/07/11 08:00-10:00 18/07/11

Block 01CJ4FYXFG0V5VM0G2FQXT6G3A contains highest samples count and is ommited from the deletion list!

overlapping blocks : 18/07/11 14:00-16:00 18/07/11

Block 01CJ54J5V66TX4YSMV3RDMK917 contains highest samples count and is ommited from the deletion list!

BLOCK ULID MIN TIME MAX TIME 01CJ54J5V66TX4YSMV3RDMK917 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ54JBTCDV83661ATDJFN034 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ54JGRPJ55YMZ0WPJA5WTXH 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ54JVRXSY42VH8EGNKWYPDY 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ54KBNRWJ7912V1YR9TDXT1 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ54MF1A7DR8QJ9963KDE65E 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ54PA2SB46CAT9CG4HDS74T 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ54R573TKF597WTWMYAQWP7 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ54T0V70HZ68QQM3NXXAS2X 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ54W07ZXFEC2KGEJ97C490G 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ54Y0BA2FKGWXN4EFT8SHSV 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ54ZVX31VSS5Y91820DFVPM 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ551V3TC0NPBVD4DCC9B7RK 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ553QR9543PBZK7PXKTQ9SE 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ555JRCNMBVSEQV67FVYJRH 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ557H26AXFVJD14GEE4EW61 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ559C32XHQ6VQCSQGMQ47K7 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ55BB52GYD23A7P3VF2Z3BP 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ55D62K50AS5GHGYKB7980B 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ55F11B9YN0ZHJ6VTMGDFX2 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ55GW6TQ63JKJB5K66P5ZA6 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ55JRYGV6FSSWZY0D468ZTA 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ55MM20CT1KC8FKKEMR2N3E 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ55PJW6YY7TCM2P5T33VS3G 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ55RE6VBA2J0SGSW6HYQC1G 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ55T9XXV5DAYRRX25G6NTAP 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ55W532VFKDYGQQD4SJEQG7 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ55Y419P5FSV4MNSH1JAGBG 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ55ZZ56PSDKE2SQ6GZ66JGT 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ561T7X3TA7KYTHE5VXHH41 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ563QJ60HKTS7TG6C9989MW 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ565NFQF77EK0P7NR9MTPR1 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ567HM7R13A9WVKTSW1ZHDG 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ569CYMTW0N9Z83HSP3NB25 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ56BBPTMJKB4V2AYF8C8AST 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ56DARWDYXEJ70Y4AF35A9C 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ56FANKEZFWDGTM60F9A0XG 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ56H8V6Z6RXPM8AZPWTNWSM 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ56K3WN6DP69CZEP36YJKF7 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ54J496SFZ7QDHSSZC03G32 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ56PXGRJMW86HKFARVK68VP 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ56RSF2GASZ1AV89F6BY042 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ56TNF20XTTH47YB7XBM4EJ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ56WKNYP5KTE6WW4N3WFQG2 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ56YENRNJQRBX3EFV3WYZHC 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ570A8YMECEC4ZDZD2SDFNE 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5728CMSM828BZBJCJCTJYD 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5743ZBZCWB8CK243XW3TTA 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ576296PEMZMB75NHEYGG2F 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5781NM1X4M1Q6HWPW6WG7B 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ579X6XSP73KAMRV2DYRVFF 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ57BS2BWBY51H8YQXY9NM3B 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ57DNKWP6R257T2WF2BW4JD 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ57FN31PJPFCB28E0DJZFZD 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ57HH2BK30GSEK29803SQGH 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ57KCFXHB5HM36S85X3YAZP 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ57NA6B8V9K865FDNJEZJJQ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ57Q5KYXS3SDZ115KDSW23A 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ57S52CGE2S89JK1N0DRKNR 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ57V0D8SZRCPT3YECRWF4BA 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ57WVRE8CQRXFVE3HPFGX25 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ57YR6WXNHJY0SD4X9ZBT6X 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ580M77MS3BC4FZBHNKFQ8T 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ582JPGBJYK9RRKQDSZ5553 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ584FM8TPV3JSYDC8N4QH2T 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ586E43A1WY89D7DSYJB8C7 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ588A9HENY3D3BPKDRC14QE 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ58A7XB8VS778F0PD45NABZ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ58C7WY07MDNXSQ4EJF2RP3 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ58E6KRCK2VP3V2KYPPEARM 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ58G6KRY94RRJST7TP2WBFS 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ58J1ZQETE7WBW5BSKV7XQH 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ58KXBG6AVTWD7E202403BN 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ58NTFCSRSRG0ZA62XYS9SN 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ58QPTR6DMZ511E2D975ZZP 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ58SMQ3515RYFAV04CGAMNQ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ58VGKDZZMXWZYH7MX2QZH3 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ58XBX6Z1MFPKR6700G9ZYX 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ58Z8HH42M791KVHQ4XJ1KF 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ591476RHHKA8JKNPDFREQ3 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ59346E341E613QPX90FH64 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ59500WKW82ZEN5CP6BPPXH 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ596XWEPX1QY6KV6P9C9B2N 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ598SAY8QNEZA8FZ3CA2ZWP 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ59AMQKSBBM9CG1HPCNM042 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ59CHJ2FG7734383Y64ZXXZ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ59EE9HCAVKQSZSSPJR095X 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ59GDA1HQY6WN9CSKY1K2QE 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ59JBVW66J3WYX79TQ0F1N5 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ59M7A2WZMGK7KMGY4VYM9P 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ59P2ZQE330RF08V2AT4KSN 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ59QZRV5YBQJV99T2XCEJSJ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ59SVBD0N6C3BW32W2YYJ58 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ59VPZ28VYDQNF7BEWZXP4S 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ59XKX6S1DHF2X6N108F026 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ59ZRJHAJ1QNGXWZVC7KJQ6 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5A1N5M6WHZB9VVNF4KKCFQ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5A3J8JAPWMCEVMDQH3JKPW 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5A5FJD2VAKEYM9BBE4K3F6 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5A7EK3M0ZF3ESY7E3J5VAM 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5A9A4J2GGXS83MFWXBN0FX 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5AB8J09GCDKT5A872D6M57 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5AD7QEQCMC0QA51VNC32QF 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5AF3AVTV4MW1F2JKRCBJ48 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5AH0FMAMB054QN866R21DK 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5AJZA94W8H2AB20QX8R3F2 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5AMWB4S1CW2PH52RQFDDJP 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5APQT4XABEF7GT7A1ER8MD 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5ARN4YG00S0Z8XDBR2K0QQ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5ATN1BB0J10ZPM4GQE0CEH 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5AWGP790Q6RXCQT1KNW08Y 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5AYC9RWWPZDF87JXH874BE 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5B0AP0J5G1CD49HBVFT2KS 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5B26AGDA9JV638XMV9EV53 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5B44SXPNDEHZGGKVYJ7NVF 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5B61F4PER905ZA1DYCSNZZ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5B7XDN3BVZ9R1PAHG56ZRK 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5B9T0NP2JYCTGHF2CVSMG2 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5BBR6PEZ7SRVCB3KFD936H 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5BDKW448AN4ZM09H421TWD 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5BFFYPKBPPS67Y8CE8YGED 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5BHCCT9AN0FR7CPH5XERDM 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5BKCTM0M7P3TKHE4E0K6PW 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5BNC6FC4VNSHRPB5H5ZHQM 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5BQ7Z4MJP2D5Q0X1V2NZMF 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5BS5256S0MJREP2YG91WKV 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5BV6PZNKRCV7TKSSVD0DEP 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5BX5J1T75MQ2K7SCKRJH5F 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5BZ31ZFZM3VWQ7EWBRNHH5 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5C13MZSS5CKNS17YNB90EP 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5C2ZVC06A47TSC9WVA4AGG 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5C4W5ZD863V5MAS4G78JW9 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5C6STQPWJXT895FDA6MH0V 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5C8PT24XNY32PDZ9EVRZ4D 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5CAM39QA3T7BD0VYSNHQDZ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5CCK52FZNA1JYSNZ89W9D7 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5CEJDTM8REY02X40T4BGSV 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5CGF5E3EKMWJ25XQ7BZ5DY 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5CJBR540JJGPG7RW8KDJJF 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5CM7HF7BW5F34S9JB292VK 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5CP4FWR6C7XJ162Z85JABG 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5CR244KCP4T4VK5DHE1W5G 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5CT04S3F185CEXABE5MYA2 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5CVYSC41MB50CDP6ZSS6E1 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5CXZNY7RFFNXEJM82S44K8 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5CZYWCRM0F8DQP9AT3TWMJ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5D269KP079D0CRVKQ9RCJS 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5D45V9NAR0NJKRFE49346A 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5D63B0GSBB9Y0PG8XSN446 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5D80ZZ83AQHHT677VM7CDZ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5D9X01PQ2SHV4J72GR51V5 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5DBVJC2DPKNWGJSD3MYJTS 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5DDV4N0600G1F6BDXK41PY 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5DFVPAB4JBXHENVC9HE6P1 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5DK6HNCH7J3BV866DRSR9N 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5DN36VB72VPH2AC188G2JZ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5DQ3QSV6CG2DHKJ9S66AJ0 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5DS05Z2PCKCTEZ8DDQCT62 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5DTX6D50SFJFAKTA8KAVQ0 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5DXB5QQGGYX7XC40ZXJGCY 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5DZ7DS9AM890WXBMQ1KVQM 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5E15CXKH2QYF6PR5N9FNBA 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5E31TA67W6968J6F4AM6FX 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5E6C43Y456FS95PQ6VVQ68 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5E888HECNC32ERF466DRT5 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5EA3XNH6A2ZTCKK1Z0D5R9 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5EC2S3H7TFHZKSSF1Q1SSR 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5EE0BR6DRH3B9RJ2C0P3TA 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5EFW0CTCWA31406J19MD09 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5EHQH43ZBWF0MX0RVCSNXD 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5EKMG22JHG7TGG3X59J4T9 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5ENMCZT7MNNVJWF4ZWYBZ7 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5EQGDRJAS9BZ8ZD48C3SPX 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5ESDX8VPYA1RQMPEQK21QQ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5EV9Z2FCV30T7F59TD5CNG 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5EX5F411QDQHV3KMHWA2H4 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5EZ1BRYHHWD5DYXRD08Z81 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5F0XD27P368QJB5JVRXCMZ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5F2TRSF0PVSB2KPVVASKY8 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5F4QH1MAT9G2VA6KRBHDDG 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5F6K93EBBXZ664BD1TE46A 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5F8ES475PK0M16DSPSNFDD 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5FAB6ZTN00CTERGYTXMBPZ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5FC70K2SXBWZZZGBDJQQXS 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5FE2JNSEWFGZYF94MWJ8HK 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5FG16HE72DHK64BN92RY86 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5FJ06N00J5KT7RJB6EZSBW 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5FKW0CK2M86RYW7ZYK4TSQ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5FNQXKFQ1JM5HC4NYGK9BM 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5FQKEYZXN6Z05K522AKHWN 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5FSJC097N3CYVHWG8HTD9H 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5FVF3CMHVF61N2R68VAZDV 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5FXBPR4VPCNHQBM2YVJBGS 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5FZAHZN74CZ15CFZ6SVZB0 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5G18KVA0VD0D2WHQGWS1QJ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5G3479W1FW0WR4AHQ2H8Y9 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5G508YAPWTS0P6ZN3WE6D3 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5G6VXEB32WQE1RVJW4MHX3 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5G8RM31V2CDQ6818PTJ503 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5GAME0T059G2T23TNM87TH 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5GCH37FCT0AG4Y8VAZCRW5 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5GECVPPMYJ1K4DFQ2NZXZ2 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5GG8G5N3W73JC8VN1ZVD5C 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5GJ66YD82134NEVWS57FVW 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5GM4KYE1XF7EGZ239A1QJV 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5GP11WRR1B2WHJ2C5H394N 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5GQWQPN26F5ZF0AJ3922V6 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5GSRN9J4Y6NVG82112HKCR 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5GVNJM9H3TVRQ4K58R1ACM 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5GXHGJSGX8FFV2MHRJBBZ8 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5GZFBHXQ3YG3CZENGPR2E7 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5H1B03HZRE18HHN7TRW7XP 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5H36KAH6N28HEJQJQEABDH 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5H52B7NNK5QV1PX0J1N7V4 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5H713PPS77S8JXVWT6WWPQ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5H8ZHAF912PQPGFRG7AS6Y 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5HAVXZTMK9XEF15Y7K9WV0 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5HCRGG4PYVJRKRJ56AFQF0 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5HEQRR8XGXPZZ8HD4YWXE0 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5HGQ11NA03BCEPYW46DF34 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5HJKQHBD4XQHQEJ6HAHYEB 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5HMFM3PZJC2M4HD7TAVBV2 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5HPD69QTRQW6MJRS5SXDZR 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5HR9GPZ6D2JJQTJP5RV739 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5HT91PA53VHNZQ322HGJ3N 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5HW5R3WVFWYAHQ5GTC4ZJZ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5HY2VR6YC8N1AWWCNV75YQ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5HZZ4AZJFJBNEJFA3WCAJ7 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5J1W43ZR6SY4VEJRD9GHF7 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5J3QWR5Y3XDK54B0FDT0R9 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5J5RWKJ797302SFFZ534MD 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5J7NSZPGTZ2RKMWMQX47FP 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5J9HP7DKTJD5SYBFW3KQHE 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5JBE0RT66F9QRXMB1H6V89 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5JDA0N9RK9K2J50NM2S217 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5JF6SMG7VDSXCQMHFYV0P4 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5JH39TX5M22V6SYBCBP2KG 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5JK1W5XN67VXS5FFVKH6RG 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5JMYW2SB5P597M06J8T1HJ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5JPXC3S07Y3YXFM5YM0YH2 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5JRS371SM8CKQJAC0HNKH4 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5JTNQ0MRJ1BGYRYS1PBNEC 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5JWHHDZQWDZHRM58EFVVMS 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5JYDC53Z9B9ZAJKK1KH0W7 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5K0BG5F02FBK7S08W4FRTZ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5K29XCN00Q94A348MRAWYF 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5K45WP64RWCAB37H7P8C7N 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5K6232RWJQ9QYS1DW3DMV4 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5K808CGTNS03TTQ4T79PVH 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5K9W1Z3NCN4YHDCGTDXBXP 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5KBVFH4H4JS1675CXSGSFA 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5KDS12032C3PQ8R2PSW8XV 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5KFNM9BC9JWJNBVJ0EEJ28 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5KHHTRDDQBBTDV76EYHDGW 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5KKDNEHR2Q7B7NDXPER55S 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5KN8JR8XKM77Y6G8V6HWV3 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5KQ858T5ZF6EZN61M61NDV 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5KS3C8N6D4QTXAR0CWSRKE 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5KTYAEAM08BXHPGYKJ5R71 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5KWSQ3W8SR62SWJXQS61GK 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5KYNHB9RV2NGNZ876ZHGA9 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5M0K3JHBK92MVRCE4FZ5SQ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5M2E00SPH1K5WA57NCWKSE 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5M48Y39PTKX246HHNEPSSN 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5M656B4SWK0DC0Z9EQGKF8 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5M80A08MWEJPSWA7VKP29J 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5M9YNMRM7DYEQSTXYHATTE 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5MBV1EGFFXNQ5VH1YZSME7 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5MDP0ANFVJ08R5W4K36T32 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5MFH16E9ANREJYXDK53FWD 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5MHCD3SGPR7MNPKRDEDFE9 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5MKBHX4KRVJ8Q6C28S4D3X 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5MN6HN9C534KF2XJP1RWKC 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5MQ4BQ8AC2ZSHNGSRG0GZC 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5MRZEW6D47XGFAKQW8KQK2 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5MTWN7WVGFBPFESEAVRVQ2 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5MWQT21Y6PHPQWVMJNEHYM 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5MYJVK48TA98QTHXZVTN52 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5N0FFXANSBCFF15891EV43 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5N2ANZGFBWJR9WP0G8NVMF 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5N46NB0EXM6A9HDTSYHQA8 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5N628GNQTTSF2W38VXKSCF 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5N7X72015A8FSQMFBQRN3A 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5N9RB8WX5NEMDVFVVHZNNQ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5NBK9BMWH5YRS0EK9AV92D 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5NDEY4E1APQ3RZ3KWQ2BAD 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5NFC929EE2FV02QCF4D5BH 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5NH77VACYZV4E54DMZGD6A 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5NK4XN165Z8PEC80EPBJBT 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5NN02XSQWJ0ABKERCA3TAA 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5NPV2PST0QS1VRK2MZX3XH 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5NRSZMVXWJ4S1EQV67P3ZP 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5NTMVK0E67D9NMTPJY6NY4 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5NWFXV2EXSRPW679SKFDN0 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5NYF7YH0XDDZBC419WW3M3 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5P0E2HWVM8GNMT7EFRX2H3 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5P296BBVN8BG1H6P36AJCE 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5P448ZWR8Z98GHRTDD7XFE 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5P631960H6PTYNWQKS20HM 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5P7Y1BANGJXDKK9PJAA7DD 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5P9WV0TVEH2TA57DH2K0XF 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5PBWDGPFKAPNWS36W2SRN1 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5PDRKTRTFK7X10PZESDWNQ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5PFKK9N6MZGBW2H1MYHEHR 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5PHEJWVFM1YQ0XSZ2F4Z8C 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5PKBV7GF53SFVQ26JR00EE 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5PN6SF1XE2MQDA1QZSBYG1 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5PQ1R16CTMS1QXMQ59JPGV 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5PRZ1DJFZ01XNWM9SQZGGE 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5PTTMCACVEZ86M79SN4BDG 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5PWNMZNZFZG3DCCVWQ0APW 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5PYGMHRCCHJ37PMRWDVTPS 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5Q0C46HDHWXB7W2JVJT1QZ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5Q277AK5SM6HJD70MVVPPQ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5Q425R66Z7CWF486BYS11K 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5Q5X5ZQ3QNZBZ050XKPPPS 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5Q7TDWSH5FX7KEA5HPXSF8 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5Q9S9HNHMV7QRF3YQJBK9R 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5QBNQ2537BWE8Z15HGTRMZ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5QDGQEB2X9BA9RH3RTR26A 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5QFBNTZY7PPPWDZSSBKD4D 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5QH6NZC65DN0F7W1B60RW6 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5QK205N91E11FQTESZSMFA 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5QN0KWZ4ZSYCQYRQEQDTFZ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5QPVHPSYK0P4GKVFKJP6FM 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5QRQQTY8A72R3F1MV3QSM2 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5QTMDATDK3FV27K0B0QHGV 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5QWFH4SCFJ62AARKZBD132 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5QYD2YK9YEX9PSZJ4VFP7M 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5R0937ZJDMGQF96ZZ39V6N 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5R240J56F49NTFZZ80QXF8 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5R3YZM687T6TPCGMKD405A 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5R5T2QQXFSNC9E5R502RN0 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5R7PVVDG3023S725XGR08A 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5R9NJV9WFJWVS15DXEGPZ9 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5RBGS5J9ENEQBPSJSGX0NV 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5RDFQHAM186AYNJXEDMJAE 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5RFASVTJSNFBBKNS4VNE05 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5RH5QGEQWPDKQYNB94DE9W 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5RK34JQD4V35ZSBFP321KQ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5RMYZ5WR9FJ18165ED7BPF 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5RPTW2RE2HEPCP0EYZT9DG 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5RRQQV09JKE2PZ9PQ7HQHE 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5RTJS6BAFK9K4SVKMVXXP2 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5RWEQH8ETTG1Y6YB28XHKG 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5RY9RT9J66A8N9MMXVVZZN 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5S04R3XQ2A82TXX1B55ET9 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5S24M853AQDFQ4EXWMCJA0 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5S3ZKWKBSEWMSJZS0462WH 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5S5V3GXN5KR8Y5KGWHXG30 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5S7S4S5E4XG9TPMD81H4JZ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5S9PVT6WEC5SPCNV6MFBB4 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5SBHV7F10S4H63MNZFE4MK 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5SDE3JXNH9DPTR8T7NBYJ1 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5SFCSXKW5KD2JV901ZFP1X 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5SH81BHXN6QHVFXNS0PPMX 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5SK31GGXS0JXT2MGD94VYY 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5SMY2X8E2Y0K7DWHZDAXE7 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5SPW30DWW7FRPTDK7818DA 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5SRQ715G4EYWATG37E9SC4 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5STNMFHJSKV16FQDMDC6KZ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5SWGNMMP46JK2TYMCX4VY1 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5SYBY7Z43SF3Y0Z30E2YPG 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5T0719MR6K00SR3422RZQN 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5T2205DQJQQSWBTA9R4HA7 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5T3X07N6V6M958HGBHD4AV 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5T5QYFXKJ2X4SXVM3DB1KR 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5T7JX7M8H6M6D2VWAREVXJ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5T9E1FFV9JCFAEW81KCQKB 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5TBA1W66XYJYVQ0RYJFPRH 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5TD5ZF8MCQNWJ5PGFMK7M6 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5TF0Y8TAT9B0T37ERMC5NZ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5TGVW0MCJD45MCPGWMBYB8 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5TJTDRB579KS6ZZM8JFRTJ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5TMNE7X929HE32R7WXXR2G 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5TPGGF3KPV26B7JA85VF4W 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5TREMWEHZDHETXND7EEYRE 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5TT9KTR6ZA046FX5APWA89 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5TW4QBNDMDEMXYV7K529GP 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5TXZMEPDY4EX6R8GV80P4Y 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5TZY1CY7Z3AAH2NM4QDPG7 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5V1S0EK00A08WG3W9K6BEQ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5V3PZVQC5VZTKMCK5KTED4 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5V5J1P9SNBSKG45MXG7HDC 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5V7CYTHJW1HBJHJAA2KC2G 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5V9B9543GCZN2CT7K713V8 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5VB9EJABPCFQSB7HYSPXAV 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5VD4F3M713BJKM81JTKZGC 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5VEZZJE01KMPG3Y54Q3Q06 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5VGV8NDPDZ8FVRH43QRKPE 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5VJPB6XA44AR5RPPQ4651G 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5VMJVNK1M0XKMGH8KNA6WS 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5VPFXYWS8B8FZH6BY9EANT 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5VRAY5ZAD2WJZ7SRM22X2V 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5VT64X07ZH6CEK1D589AMH 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5VW14AS2D38S1X0JBXDB3E 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5VXW5QQ5S4HEBH5MDEV6BA 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5VZQ8H0EE3MN54ZR5D27MP 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5W1JGQBP361SBQGRDTBVP3 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5W3GFC8VJ032NY3KSR266V 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5W5E71F2AADM41VFXF5Y35 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5W79KNAGMYBM5KG3J0AMJF 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5W94SNSMSRF1J1616CMMCN 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5WB2KZX0WTKFY0NTMVRDQF 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5WCXKPF6BXR3RWCC6DATCB 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5WEVFCQ280P2SK4S0BVQSC 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5WGQ1PV4WHKNM3HV5TTEMZ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5WJJ7R5SYXCENSCD1FMESK 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5WMDBRJTKPR6ZVNJS45H5T 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5WP8B614MNT7ZCKTQTWZ5P 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5WR6970N2R48361RC1E9M4 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5WT1HJJ0GJAB0F3ES2R4BR 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5WW0AX8PM7XFVKMSWJG4SR 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5WXVSSPD0K998Y3DZBVAR1 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5WZPZ4S4PH435T1EFEM2AJ 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5X1HZQRFWQEB2B7MZ3X2MB 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5X3CZ1E8TSAA21020M5JVD 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5X5B5NZTGMZPS7YCQKMK1X 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5X76HHQ4NN2NQRGGJNPVKA 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5X93CPC8Q5VV3G8ZZP0E6B 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5XAZQNR9N7JCHEW70KCPV9 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5XCYQ12MCNNNP5PZV4BPMV 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5XESPDSFC11B377KJQHEP6 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5XGMPJF29EP4JS72B453B8 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5XJFY353M0CW4YCXJAS3EK 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5XMB43A4G1K64M754HFHCM 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5XP634WP7G1QEG1QWJ6DFS 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5XR3G44FC9C7NXN4AB9Y7Q 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5XT14X52EEAZD15QR7YPBT 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5XVXB90DXHED0DY0Y4RKZC 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5XXV435NRC1RN9PV6V2K80 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5XZQ3V0KD7P1KTVZQXMTQC 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5Y1J6EJMEQTVFGKZTF997H 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 01CJ5Y3D51HKM8XXA6AVAE6ZDF 2018-07-11 14:00:00 +0000 UTC 2018-07-11 16:00:00 +0000 UTC 2488520 DELETE (y/N)? y
Scan complete!
Hooray! The db is clean(or the scan tool is broken):😸
[ramcoazure@HCMDEMAZIND01 prometheus-2.3.1.linux-amd64]$ ./prometheus
level=info ts=2018-07-12T05:31:15.722898808Z caller=main.go:222 msg="Starting Prometheus" level=info ts=2018-07-12T05:31:15.722982011Z caller=main.go:223 build_context="(go=go1.10.3, level=info ts=2018-07-12T05:31:15.723005712Z caller=main.go:224 host_details="(Linux 3.10.0-123.2 level=info ts=2018-07-12T05:31:15.723025013Z caller=main.go:225 fd_limits="(soft=1024, hard=4096) level=info ts=2018-07-12T05:31:15.723805542Z caller=main.go:514 msg="Starting TSDB ..."
level=info ts=2018-07-12T05:31:15.724037951Z caller=repair.go:39 component=tsdb msg="found level=info ts=2018-07-12T05:31:15.724107453Z caller=repair.go:39 component=tsdb msg="found level=info ts=2018-07-12T05:31:15.724156255Z caller=repair.go:39 component=tsdb msg="found level=info ts=2018-07-12T05:31:15.724200957Z caller=repair.go:39 component=tsdb msg="found level=info ts=2018-07-12T05:31:15.724244459Z caller=repair.go:39 component=tsdb msg="found level=info ts=2018-07-12T05:31:15.72428696Z caller=repair.go:39 component=tsdb msg="found level=info ts=2018-07-12T05:31:15.724336962Z caller=repair.go:39 component=tsdb msg="found level=info ts=2018-07-12T05:31:15.724386864Z caller=repair.go:39 component=tsdb msg="found level=info ts=2018-07-12T05:31:15.724432266Z caller=repair.go:39 component=tsdb msg="found level=info ts=2018-07-12T05:31:15.724476667Z caller=repair.go:39 component=tsdb msg="found level=info ts=2018-07-12T05:31:15.724520869Z caller=repair.go:39 component=tsdb msg="found level=info ts=2018-07-12T05:31:15.72455797Z caller=repair.go:39 component=tsdb msg="found level=info ts=2018-07-12T05:31:15.724601572Z caller=repair.go:39 component=tsdb msg="found level=info ts=2018-07-12T05:31:15.724658474Z caller=repair.go:39 component=tsdb msg="found level=info ts=2018-07-12T05:31:15.724702076Z caller=repair.go:39 component=tsdb msg="found level=info ts=2018-07-12T05:31:15.724741777Z caller=repair.go:39 component=tsdb msg="found level=info ts=2018-07-12T05:31:15.724778579Z caller=repair.go:39 component=tsdb msg="found level=info ts=2018-07-12T05:31:15.751614284Z caller=web.go:415 component=web msg="Start level=info ts=2018-07-12T05:31:15.770081276Z caller=main.go:402 msg="Stopping scrape discovery level=info ts=2018-07-12T05:31:15.770113677Z caller=main.go:416 msg="Stopping notify discovery level=info ts=2018-07-12T05:31:15.770125678Z caller=main.go:438 msg="Stopping scrape manager..."< level=info ts=2018-07-12T05:31:15.770141078Z caller=manager.go:464 component="rule manager" level=info ts=2018-07-12T05:31:15.770154479Z caller=manager.go:470 component="rule manager" level=info ts=2018-07-12T05:31:15.770166579Z caller=notifier.go:512 component=notifier msg="Stopping level=info ts=2018-07-12T05:31:15.770214481Z caller=main.go:398 msg="Scrape discovery manager level=info ts=2018-07-12T05:31:15.770233882Z caller=main.go:412 msg="Notify discovery manager level=info ts=2018-07-12T05:31:15.770297784Z caller=main.go:432 msg="Scrape manager stopped"
level=info ts=2018-07-12T05:31:15.770319985Z caller=main.go:588 msg="Notifier manager stopped"
` NUM SAMPLES NUM CHUNKS NUM SERIES PATH
18629 3604 data/01CJ54J5V66TX4YSMV3RDMK917
18629 3604 data/01CJ54JBTCDV83661ATDJFN034
18629 3604 data/01CJ54JGRPJ55YMZ0WPJA5WTXH
18629 3604 data/01CJ54JVRXSY42VH8EGNKWYPDY
18629 3604 data/01CJ54KBNRWJ7912V1YR9TDXT1
18629 3604 data/01CJ54MF1A7DR8QJ9963KDE65E
18629 3604 data/01CJ54PA2SB46CAT9CG4HDS74T
18629 3604 data/01CJ54R573TKF597WTWMYAQWP7
18629 3604 data/01CJ54T0V70HZ68QQM3NXXAS2X
18629 3604 data/01CJ54W07ZXFEC2KGEJ97C490G
18629 3604 data/01CJ54Y0BA2FKGWXN4EFT8SHSV
18629 3604 data/01CJ54ZVX31VSS5Y91820DFVPM
18629 3604 data/01CJ551V3TC0NPBVD4DCC9B7RK
18629 3604 data/01CJ553QR9543PBZK7PXKTQ9SE
18629 3604 data/01CJ555JRCNMBVSEQV67FVYJRH
18629 3604 data/01CJ557H26AXFVJD14GEE4EW61
18629 3604 data/01CJ559C32XHQ6VQCSQGMQ47K7
18629 3604 data/01CJ55BB52GYD23A7P3VF2Z3BP
18629 3604 data/01CJ55D62K50AS5GHGYKB7980B
18629 3604 data/01CJ55F11B9YN0ZHJ6VTMGDFX2
18629 3604 data/01CJ55GW6TQ63JKJB5K66P5ZA6
18629 3604 data/01CJ55JRYGV6FSSWZY0D468ZTA
18629 3604 data/01CJ55MM20CT1KC8FKKEMR2N3E
18629 3604 data/01CJ55PJW6YY7TCM2P5T33VS3G
18629 3604 data/01CJ55RE6VBA2J0SGSW6HYQC1G
18629 3604 data/01CJ55T9XXV5DAYRRX25G6NTAP
18629 3604 data/01CJ55W532VFKDYGQQD4SJEQG7
18629 3604 data/01CJ55Y419P5FSV4MNSH1JAGBG
18629 3604 data/01CJ55ZZ56PSDKE2SQ6GZ66JGT
18629 3604 data/01CJ561T7X3TA7KYTHE5VXHH41
18629 3604 data/01CJ563QJ60HKTS7TG6C9989MW
18629 3604 data/01CJ565NFQF77EK0P7NR9MTPR1
18629 3604 data/01CJ567HM7R13A9WVKTSW1ZHDG
18629 3604 data/01CJ569CYMTW0N9Z83HSP3NB25
18629 3604 data/01CJ56BBPTMJKB4V2AYF8C8AST
18629 3604 data/01CJ56DARWDYXEJ70Y4AF35A9C
18629 3604 data/01CJ56FANKEZFWDGTM60F9A0XG
18629 3604 data/01CJ56H8V6Z6RXPM8AZPWTNWSM
18629 3604 data/01CJ56K3WN6DP69CZEP36YJKF7
18629 3604 data/01CJ54J496SFZ7QDHSSZC03G32
18629 3604 data/01CJ56PXGRJMW86HKFARVK68VP
18629 3604 data/01CJ56RSF2GASZ1AV89F6BY042
18629 3604 data/01CJ56TNF20XTTH47YB7XBM4EJ
18629 3604 data/01CJ56WKNYP5KTE6WW4N3WFQG2
18629 3604 data/01CJ56YENRNJQRBX3EFV3WYZHC
18629 3604 data/01CJ570A8YMECEC4ZDZD2SDFNE
18629 3604 data/01CJ5728CMSM828BZBJCJCTJYD
18629 3604 data/01CJ5743ZBZCWB8CK243XW3TTA
18629 3604 data/01CJ576296PEMZMB75NHEYGG2F
18629 3604 data/01CJ5781NM1X4M1Q6HWPW6WG7B
18629 3604 data/01CJ579X6XSP73KAMRV2DYRVFF
18629 3604 data/01CJ57BS2BWBY51H8YQXY9NM3B
18629 3604 data/01CJ57DNKWP6R257T2WF2BW4JD
18629 3604 data/01CJ57FN31PJPFCB28E0DJZFZD
18629 3604 data/01CJ57HH2BK30GSEK29803SQGH
18629 3604 data/01CJ57KCFXHB5HM36S85X3YAZP
18629 3604 data/01CJ57NA6B8V9K865FDNJEZJJQ
18629 3604 data/01CJ57Q5KYXS3SDZ115KDSW23A
18629 3604 data/01CJ57S52CGE2S89JK1N0DRKNR
18629 3604 data/01CJ57V0D8SZRCPT3YECRWF4BA
18629 3604 data/01CJ57WVRE8CQRXFVE3HPFGX25
18629 3604 data/01CJ57YR6WXNHJY0SD4X9ZBT6X
18629 3604 data/01CJ580M77MS3BC4FZBHNKFQ8T
18629 3604 data/01CJ582JPGBJYK9RRKQDSZ5553
18629 3604 data/01CJ584FM8TPV3JSYDC8N4QH2T
18629 3604 data/01CJ586E43A1WY89D7DSYJB8C7
18629 3604 data/01CJ588A9HENY3D3BPKDRC14QE
18629 3604 data/01CJ58A7XB8VS778F0PD45NABZ
18629 3604 data/01CJ58C7WY07MDNXSQ4EJF2RP3
18629 3604 data/01CJ58E6KRCK2VP3V2KYPPEARM
18629 3604 data/01CJ58G6KRY94RRJST7TP2WBFS
18629 3604 data/01CJ58J1ZQETE7WBW5BSKV7XQH
18629 3604 data/01CJ58KXBG6AVTWD7E202403BN
18629 3604 data/01CJ58NTFCSRSRG0ZA62XYS9SN
18629 3604 data/01CJ58QPTR6DMZ511E2D975ZZP
18629 3604 data/01CJ58SMQ3515RYFAV04CGAMNQ
18629 3604 data/01CJ58VGKDZZMXWZYH7MX2QZH3
18629 3604 data/01CJ58XBX6Z1MFPKR6700G9ZYX
18629 3604 data/01CJ58Z8HH42M791KVHQ4XJ1KF
18629 3604 data/01CJ591476RHHKA8JKNPDFREQ3
18629 3604 data/01CJ59346E341E613QPX90FH64
18629 3604 data/01CJ59500WKW82ZEN5CP6BPPXH
18629 3604 data/01CJ596XWEPX1QY6KV6P9C9B2N
18629 3604 data/01CJ598SAY8QNEZA8FZ3CA2ZWP
18629 3604 data/01CJ59AMQKSBBM9CG1HPCNM042
18629 3604 data/01CJ59CHJ2FG7734383Y64ZXXZ
18629 3604 data/01CJ59EE9HCAVKQSZSSPJR095X
18629 3604 data/01CJ59GDA1HQY6WN9CSKY1K2QE
18629 3604 data/01CJ59JBVW66J3WYX79TQ0F1N5
18629 3604 data/01CJ59M7A2WZMGK7KMGY4VYM9P
18629 3604 data/01CJ59P2ZQE330RF08V2AT4KSN
18629 3604 data/01CJ59QZRV5YBQJV99T2XCEJSJ
18629 3604 data/01CJ59SVBD0N6C3BW32W2YYJ58
18629 3604 data/01CJ59VPZ28VYDQNF7BEWZXP4S
18629 3604 data/01CJ59XKX6S1DHF2X6N108F026
18629 3604 data/01CJ59ZRJHAJ1QNGXWZVC7KJQ6
18629 3604 data/01CJ5A1N5M6WHZB9VVNF4KKCFQ
18629 3604 data/01CJ5A3J8JAPWMCEVMDQH3JKPW
18629 3604 data/01CJ5A5FJD2VAKEYM9BBE4K3F6
18629 3604 data/01CJ5A7EK3M0ZF3ESY7E3J5VAM
18629 3604 data/01CJ5A9A4J2GGXS83MFWXBN0FX
18629 3604 data/01CJ5AB8J09GCDKT5A872D6M57
18629 3604 data/01CJ5AD7QEQCMC0QA51VNC32QF
18629 3604 data/01CJ5AF3AVTV4MW1F2JKRCBJ48
18629 3604 data/01CJ5AH0FMAMB054QN866R21DK
18629 3604 data/01CJ5AJZA94W8H2AB20QX8R3F2
18629 3604 data/01CJ5AMWB4S1CW2PH52RQFDDJP
18629 3604 data/01CJ5APQT4XABEF7GT7A1ER8MD
18629 3604 data/01CJ5ARN4YG00S0Z8XDBR2K0QQ
18629 3604 data/01CJ5ATN1BB0J10ZPM4GQE0CEH
18629 3604 data/01CJ5AWGP790Q6RXCQT1KNW08Y
18629 3604 data/01CJ5AYC9RWWPZDF87JXH874BE
18629 3604 data/01CJ5B0AP0J5G1CD49HBVFT2KS
18629 3604 data/01CJ5B26AGDA9JV638XMV9EV53
18629 3604 data/01CJ5B44SXPNDEHZGGKVYJ7NVF
18629 3604 data/01CJ5B61F4PER905ZA1DYCSNZZ
18629 3604 data/01CJ5B7XDN3BVZ9R1PAHG56ZRK
18629 3604 data/01CJ5B9T0NP2JYCTGHF2CVSMG2
18629 3604 data/01CJ5BBR6PEZ7SRVCB3KFD936H
18629 3604 data/01CJ5BDKW448AN4ZM09H421TWD
18629 3604 data/01CJ5BFFYPKBPPS67Y8CE8YGED
18629 3604 data/01CJ5BHCCT9AN0FR7CPH5XERDM
18629 3604 data/01CJ5BKCTM0M7P3TKHE4E0K6PW
18629 3604 data/01CJ5BNC6FC4VNSHRPB5H5ZHQM
18629 3604 data/01CJ5BQ7Z4MJP2D5Q0X1V2NZMF
18629 3604 data/01CJ5BS5256S0MJREP2YG91WKV
18629 3604 data/01CJ5BV6PZNKRCV7TKSSVD0DEP
18629 3604 data/01CJ5BX5J1T75MQ2K7SCKRJH5F
18629 3604 data/01CJ5BZ31ZFZM3VWQ7EWBRNHH5
18629 3604 data/01CJ5C13MZSS5CKNS17YNB90EP
18629 3604 data/01CJ5C2ZVC06A47TSC9WVA4AGG
18629 3604 data/01CJ5C4W5ZD863V5MAS4G78JW9
18629 3604 data/01CJ5C6STQPWJXT895FDA6MH0V
18629 3604 data/01CJ5C8PT24XNY32PDZ9EVRZ4D
18629 3604 data/01CJ5CAM39QA3T7BD0VYSNHQDZ
18629 3604 data/01CJ5CCK52FZNA1JYSNZ89W9D7
18629 3604 data/01CJ5CEJDTM8REY02X40T4BGSV
18629 3604 data/01CJ5CGF5E3EKMWJ25XQ7BZ5DY
18629 3604 data/01CJ5CJBR540JJGPG7RW8KDJJF
18629 3604 data/01CJ5CM7HF7BW5F34S9JB292VK
18629 3604 data/01CJ5CP4FWR6C7XJ162Z85JABG
18629 3604 data/01CJ5CR244KCP4T4VK5DHE1W5G
18629 3604 data/01CJ5CT04S3F185CEXABE5MYA2
18629 3604 data/01CJ5CVYSC41MB50CDP6ZSS6E1
18629 3604 data/01CJ5CXZNY7RFFNXEJM82S44K8
18629 3604 data/01CJ5CZYWCRM0F8DQP9AT3TWMJ
18629 3604 data/01CJ5D269KP079D0CRVKQ9RCJS
18629 3604 data/01CJ5D45V9NAR0NJKRFE49346A
18629 3604 data/01CJ5D63B0GSBB9Y0PG8XSN446
18629 3604 data/01CJ5D80ZZ83AQHHT677VM7CDZ
18629 3604 data/01CJ5D9X01PQ2SHV4J72GR51V5
18629 3604 data/01CJ5DBVJC2DPKNWGJSD3MYJTS
18629 3604 data/01CJ5DDV4N0600G1F6BDXK41PY
18629 3604 data/01CJ5DFVPAB4JBXHENVC9HE6P1
18629 3604 data/01CJ5DK6HNCH7J3BV866DRSR9N
18629 3604 data/01CJ5DN36VB72VPH2AC188G2JZ
18629 3604 data/01CJ5DQ3QSV6CG2DHKJ9S66AJ0
18629 3604 data/01CJ5DS05Z2PCKCTEZ8DDQCT62
18629 3604 data/01CJ5DTX6D50SFJFAKTA8KAVQ0
18629 3604 data/01CJ5DXB5QQGGYX7XC40ZXJGCY
18629 3604 data/01CJ5DZ7DS9AM890WXBMQ1KVQM
18629 3604 data/01CJ5E15CXKH2QYF6PR5N9FNBA
18629 3604 data/01CJ5E31TA67W6968J6F4AM6FX
18629 3604 data/01CJ5E6C43Y456FS95PQ6VVQ68
18629 3604 data/01CJ5E888HECNC32ERF466DRT5
18629 3604 data/01CJ5EA3XNH6A2ZTCKK1Z0D5R9
18629 3604 data/01CJ5EC2S3H7TFHZKSSF1Q1SSR
18629 3604 data/01CJ5EE0BR6DRH3B9RJ2C0P3TA
18629 3604 data/01CJ5EFW0CTCWA31406J19MD09
18629 3604 data/01CJ5EHQH43ZBWF0MX0RVCSNXD
18629 3604 data/01CJ5EKMG22JHG7TGG3X59J4T9
18629 3604 data/01CJ5ENMCZT7MNNVJWF4ZWYBZ7
18629 3604 data/01CJ5EQGDRJAS9BZ8ZD48C3SPX
18629 3604 data/01CJ5ESDX8VPYA1RQMPEQK21QQ
18629 3604 data/01CJ5EV9Z2FCV30T7F59TD5CNG
18629 3604 data/01CJ5EX5F411QDQHV3KMHWA2H4
18629 3604 data/01CJ5EZ1BRYHHWD5DYXRD08Z81
18629 3604 data/01CJ5F0XD27P368QJB5JVRXCMZ
18629 3604 data/01CJ5F2TRSF0PVSB2KPVVASKY8
18629 3604 data/01CJ5F4QH1MAT9G2VA6KRBHDDG
18629 3604 data/01CJ5F6K93EBBXZ664BD1TE46A
18629 3604 data/01CJ5F8ES475PK0M16DSPSNFDD
18629 3604 data/01CJ5FAB6ZTN00CTERGYTXMBPZ
18629 3604 data/01CJ5FC70K2SXBWZZZGBDJQQXS
18629 3604 data/01CJ5FE2JNSEWFGZYF94MWJ8HK
18629 3604 data/01CJ5FG16HE72DHK64BN92RY86
18629 3604 data/01CJ5FJ06N00J5KT7RJB6EZSBW
18629 3604 data/01CJ5FKW0CK2M86RYW7ZYK4TSQ
18629 3604 data/01CJ5FNQXKFQ1JM5HC4NYGK9BM
18629 3604 data/01CJ5FQKEYZXN6Z05K522AKHWN
18629 3604 data/01CJ5FSJC097N3CYVHWG8HTD9H
18629 3604 data/01CJ5FVF3CMHVF61N2R68VAZDV
18629 3604 data/01CJ5FXBPR4VPCNHQBM2YVJBGS
18629 3604 data/01CJ5FZAHZN74CZ15CFZ6SVZB0
18629 3604 data/01CJ5G18KVA0VD0D2WHQGWS1QJ
18629 3604 data/01CJ5G3479W1FW0WR4AHQ2H8Y9
18629 3604 data/01CJ5G508YAPWTS0P6ZN3WE6D3
18629 3604 data/01CJ5G6VXEB32WQE1RVJW4MHX3
18629 3604 data/01CJ5G8RM31V2CDQ6818PTJ503
18629 3604 data/01CJ5GAME0T059G2T23TNM87TH
18629 3604 data/01CJ5GCH37FCT0AG4Y8VAZCRW5
18629 3604 data/01CJ5GECVPPMYJ1K4DFQ2NZXZ2
18629 3604 data/01CJ5GG8G5N3W73JC8VN1ZVD5C
18629 3604 data/01CJ5GJ66YD82134NEVWS57FVW
18629 3604 data/01CJ5GM4KYE1XF7EGZ239A1QJV
18629 3604 data/01CJ5GP11WRR1B2WHJ2C5H394N
18629 3604 data/01CJ5GQWQPN26F5ZF0AJ3922V6
18629 3604 data/01CJ5GSRN9J4Y6NVG82112HKCR
18629 3604 data/01CJ5GVNJM9H3TVRQ4K58R1ACM
18629 3604 data/01CJ5GXHGJSGX8FFV2MHRJBBZ8
18629 3604 data/01CJ5GZFBHXQ3YG3CZENGPR2E7
18629 3604 data/01CJ5H1B03HZRE18HHN7TRW7XP
18629 3604 data/01CJ5H36KAH6N28HEJQJQEABDH
18629 3604 data/01CJ5H52B7NNK5QV1PX0J1N7V4
18629 3604 data/01CJ5H713PPS77S8JXVWT6WWPQ
18629 3604 data/01CJ5H8ZHAF912PQPGFRG7AS6Y
18629 3604 data/01CJ5HAVXZTMK9XEF15Y7K9WV0
18629 3604 data/01CJ5HCRGG4PYVJRKRJ56AFQF0
18629 3604 data/01CJ5HEQRR8XGXPZZ8HD4YWXE0
18629 3604 data/01CJ5HGQ11NA03BCEPYW46DF34
18629 3604 data/01CJ5HJKQHBD4XQHQEJ6HAHYEB
18629 3604 data/01CJ5HMFM3PZJC2M4HD7TAVBV2
18629 3604 data/01CJ5HPD69QTRQW6MJRS5SXDZR
18629 3604 data/01CJ5HR9GPZ6D2JJQTJP5RV739
18629 3604 data/01CJ5HT91PA53VHNZQ322HGJ3N
18629 3604 data/01CJ5HW5R3WVFWYAHQ5GTC4ZJZ
18629 3604 data/01CJ5HY2VR6YC8N1AWWCNV75YQ
18629 3604 data/01CJ5HZZ4AZJFJBNEJFA3WCAJ7
18629 3604 data/01CJ5J1W43ZR6SY4VEJRD9GHF7
18629 3604 data/01CJ5J3QWR5Y3XDK54B0FDT0R9
18629 3604 data/01CJ5J5RWKJ797302SFFZ534MD
18629 3604 data/01CJ5J7NSZPGTZ2RKMWMQX47FP
18629 3604 data/01CJ5J9HP7DKTJD5SYBFW3KQHE
18629 3604 data/01CJ5JBE0RT66F9QRXMB1H6V89
18629 3604 data/01CJ5JDA0N9RK9K2J50NM2S217
18629 3604 data/01CJ5JF6SMG7VDSXCQMHFYV0P4
18629 3604 data/01CJ5JH39TX5M22V6SYBCBP2KG
18629 3604 data/01CJ5JK1W5XN67VXS5FFVKH6RG
18629 3604 data/01CJ5JMYW2SB5P597M06J8T1HJ
18629 3604 data/01CJ5JPXC3S07Y3YXFM5YM0YH2
18629 3604 data/01CJ5JRS371SM8CKQJAC0HNKH4
18629 3604 data/01CJ5JTNQ0MRJ1BGYRYS1PBNEC
18629 3604 data/01CJ5JWHHDZQWDZHRM58EFVVMS
18629 3604 data/01CJ5JYDC53Z9B9ZAJKK1KH0W7
18629 3604 data/01CJ5K0BG5F02FBK7S08W4FRTZ
18629 3604 data/01CJ5K29XCN00Q94A348MRAWYF
18629 3604 data/01CJ5K45WP64RWCAB37H7P8C7N
18629 3604 data/01CJ5K6232RWJQ9QYS1DW3DMV4
18629 3604 data/01CJ5K808CGTNS03TTQ4T79PVH
18629 3604 data/01CJ5K9W1Z3NCN4YHDCGTDXBXP
18629 3604 data/01CJ5KBVFH4H4JS1675CXSGSFA
18629 3604 data/01CJ5KDS12032C3PQ8R2PSW8XV
18629 3604 data/01CJ5KFNM9BC9JWJNBVJ0EEJ28
18629 3604 data/01CJ5KHHTRDDQBBTDV76EYHDGW
18629 3604 data/01CJ5KKDNEHR2Q7B7NDXPER55S
18629 3604 data/01CJ5KN8JR8XKM77Y6G8V6HWV3
18629 3604 data/01CJ5KQ858T5ZF6EZN61M61NDV
18629 3604 data/01CJ5KS3C8N6D4QTXAR0CWSRKE
18629 3604 data/01CJ5KTYAEAM08BXHPGYKJ5R71
18629 3604 data/01CJ5KWSQ3W8SR62SWJXQS61GK
18629 3604 data/01CJ5KYNHB9RV2NGNZ876ZHGA9
18629 3604 data/01CJ5M0K3JHBK92MVRCE4FZ5SQ
18629 3604 data/01CJ5M2E00SPH1K5WA57NCWKSE
18629 3604 data/01CJ5M48Y39PTKX246HHNEPSSN
18629 3604 data/01CJ5M656B4SWK0DC0Z9EQGKF8
18629 3604 data/01CJ5M80A08MWEJPSWA7VKP29J
18629 3604 data/01CJ5M9YNMRM7DYEQSTXYHATTE
18629 3604 data/01CJ5MBV1EGFFXNQ5VH1YZSME7
18629 3604 data/01CJ5MDP0ANFVJ08R5W4K36T32
18629 3604 data/01CJ5MFH16E9ANREJYXDK53FWD
18629 3604 data/01CJ5MHCD3SGPR7MNPKRDEDFE9
18629 3604 data/01CJ5MKBHX4KRVJ8Q6C28S4D3X
18629 3604 data/01CJ5MN6HN9C534KF2XJP1RWKC
18629 3604 data/01CJ5MQ4BQ8AC2ZSHNGSRG0GZC
18629 3604 data/01CJ5MRZEW6D47XGFAKQW8KQK2
18629 3604 data/01CJ5MTWN7WVGFBPFESEAVRVQ2
18629 3604 data/01CJ5MWQT21Y6PHPQWVMJNEHYM
18629 3604 data/01CJ5MYJVK48TA98QTHXZVTN52
18629 3604 data/01CJ5N0FFXANSBCFF15891EV43
18629 3604 data/01CJ5N2ANZGFBWJR9WP0G8NVMF
18629 3604 data/01CJ5N46NB0EXM6A9HDTSYHQA8
18629 3604 data/01CJ5N628GNQTTSF2W38VXKSCF
18629 3604 data/01CJ5N7X72015A8FSQMFBQRN3A
18629 3604 data/01CJ5N9RB8WX5NEMDVFVVHZNNQ
18629 3604 data/01CJ5NBK9BMWH5YRS0EK9AV92D
18629 3604 data/01CJ5NDEY4E1APQ3RZ3KWQ2BAD
18629 3604 data/01CJ5NFC929EE2FV02QCF4D5BH
18629 3604 data/01CJ5NH77VACYZV4E54DMZGD6A
18629 3604 data/01CJ5NK4XN165Z8PEC80EPBJBT
18629 3604 data/01CJ5NN02XSQWJ0ABKERCA3TAA
18629 3604 data/01CJ5NPV2PST0QS1VRK2MZX3XH
18629 3604 data/01CJ5NRSZMVXWJ4S1EQV67P3ZP
18629 3604 data/01CJ5NTMVK0E67D9NMTPJY6NY4
18629 3604 data/01CJ5NWFXV2EXSRPW679SKFDN0
18629 3604 data/01CJ5NYF7YH0XDDZBC419WW3M3
18629 3604 data/01CJ5P0E2HWVM8GNMT7EFRX2H3
18629 3604 data/01CJ5P296BBVN8BG1H6P36AJCE
18629 3604 data/01CJ5P448ZWR8Z98GHRTDD7XFE
18629 3604 data/01CJ5P631960H6PTYNWQKS20HM
18629 3604 data/01CJ5P7Y1BANGJXDKK9PJAA7DD
18629 3604 data/01CJ5P9WV0TVEH2TA57DH2K0XF
18629 3604 data/01CJ5PBWDGPFKAPNWS36W2SRN1
18629 3604 data/01CJ5PDRKTRTFK7X10PZESDWNQ
18629 3604 data/01CJ5PFKK9N6MZGBW2H1MYHEHR
18629 3604 data/01CJ5PHEJWVFM1YQ0XSZ2F4Z8C
18629 3604 data/01CJ5PKBV7GF53SFVQ26JR00EE
18629 3604 data/01CJ5PN6SF1XE2MQDA1QZSBYG1
18629 3604 data/01CJ5PQ1R16CTMS1QXMQ59JPGV
18629 3604 data/01CJ5PRZ1DJFZ01XNWM9SQZGGE
18629 3604 data/01CJ5PTTMCACVEZ86M79SN4BDG
18629 3604 data/01CJ5PWNMZNZFZG3DCCVWQ0APW
18629 3604 data/01CJ5PYGMHRCCHJ37PMRWDVTPS
18629 3604 data/01CJ5Q0C46HDHWXB7W2JVJT1QZ
18629 3604 data/01CJ5Q277AK5SM6HJD70MVVPPQ
18629 3604 data/01CJ5Q425R66Z7CWF486BYS11K
18629 3604 data/01CJ5Q5X5ZQ3QNZBZ050XKPPPS
18629 3604 data/01CJ5Q7TDWSH5FX7KEA5HPXSF8
18629 3604 data/01CJ5Q9S9HNHMV7QRF3YQJBK9R
18629 3604 data/01CJ5QBNQ2537BWE8Z15HGTRMZ
18629 3604 data/01CJ5QDGQEB2X9BA9RH3RTR26A
18629 3604 data/01CJ5QFBNTZY7PPPWDZSSBKD4D
18629 3604 data/01CJ5QH6NZC65DN0F7W1B60RW6
18629 3604 data/01CJ5QK205N91E11FQTESZSMFA
18629 3604 data/01CJ5QN0KWZ4ZSYCQYRQEQDTFZ
18629 3604 data/01CJ5QPVHPSYK0P4GKVFKJP6FM
18629 3604 data/01CJ5QRQQTY8A72R3F1MV3QSM2
18629 3604 data/01CJ5QTMDATDK3FV27K0B0QHGV
18629 3604 data/01CJ5QWFH4SCFJ62AARKZBD132
18629 3604 data/01CJ5QYD2YK9YEX9PSZJ4VFP7M
18629 3604 data/01CJ5R0937ZJDMGQF96ZZ39V6N
18629 3604 data/01CJ5R240J56F49NTFZZ80QXF8
18629 3604 data/01CJ5R3YZM687T6TPCGMKD405A
18629 3604 data/01CJ5R5T2QQXFSNC9E5R502RN0
18629 3604 data/01CJ5R7PVVDG3023S725XGR08A
18629 3604 data/01CJ5R9NJV9WFJWVS15DXEGPZ9
18629 3604 data/01CJ5RBGS5J9ENEQBPSJSGX0NV
18629 3604 data/01CJ5RDFQHAM186AYNJXEDMJAE
18629 3604 data/01CJ5RFASVTJSNFBBKNS4VNE05
18629 3604 data/01CJ5RH5QGEQWPDKQYNB94DE9W
18629 3604 data/01CJ5RK34JQD4V35ZSBFP321KQ
18629 3604 data/01CJ5RMYZ5WR9FJ18165ED7BPF
18629 3604 data/01CJ5RPTW2RE2HEPCP0EYZT9DG
18629 3604 data/01CJ5RRQQV09JKE2PZ9PQ7HQHE
18629 3604 data/01CJ5RTJS6BAFK9K4SVKMVXXP2
18629 3604 data/01CJ5RWEQH8ETTG1Y6YB28XHKG
18629 3604 data/01CJ5RY9RT9J66A8N9MMXVVZZN
18629 3604 data/01CJ5S04R3XQ2A82TXX1B55ET9
18629 3604 data/01CJ5S24M853AQDFQ4EXWMCJA0
18629 3604 data/01CJ5S3ZKWKBSEWMSJZS0462WH
18629 3604 data/01CJ5S5V3GXN5KR8Y5KGWHXG30
18629 3604 data/01CJ5S7S4S5E4XG9TPMD81H4JZ
18629 3604 data/01CJ5S9PVT6WEC5SPCNV6MFBB4
18629 3604 data/01CJ5SBHV7F10S4H63MNZFE4MK
18629 3604 data/01CJ5SDE3JXNH9DPTR8T7NBYJ1
18629 3604 data/01CJ5SFCSXKW5KD2JV901ZFP1X
18629 3604 data/01CJ5SH81BHXN6QHVFXNS0PPMX
18629 3604 data/01CJ5SK31GGXS0JXT2MGD94VYY
18629 3604 data/01CJ5SMY2X8E2Y0K7DWHZDAXE7
18629 3604 data/01CJ5SPW30DWW7FRPTDK7818DA
18629 3604 data/01CJ5SRQ715G4EYWATG37E9SC4
18629 3604 data/01CJ5STNMFHJSKV16FQDMDC6KZ
18629 3604 data/01CJ5SWGNMMP46JK2TYMCX4VY1
18629 3604 data/01CJ5SYBY7Z43SF3Y0Z30E2YPG
18629 3604 data/01CJ5T0719MR6K00SR3422RZQN
18629 3604 data/01CJ5T2205DQJQQSWBTA9R4HA7
18629 3604 data/01CJ5T3X07N6V6M958HGBHD4AV
18629 3604 data/01CJ5T5QYFXKJ2X4SXVM3DB1KR
18629 3604 data/01CJ5T7JX7M8H6M6D2VWAREVXJ
18629 3604 data/01CJ5T9E1FFV9JCFAEW81KCQKB
18629 3604 data/01CJ5TBA1W66XYJYVQ0RYJFPRH
18629 3604 data/01CJ5TD5ZF8MCQNWJ5PGFMK7M6
18629 3604 data/01CJ5TF0Y8TAT9B0T37ERMC5NZ
18629 3604 data/01CJ5TGVW0MCJD45MCPGWMBYB8
18629 3604 data/01CJ5TJTDRB579KS6ZZM8JFRTJ
18629 3604 data/01CJ5TMNE7X929HE32R7WXXR2G
18629 3604 data/01CJ5TPGGF3KPV26B7JA85VF4W
18629 3604 data/01CJ5TREMWEHZDHETXND7EEYRE
18629 3604 data/01CJ5TT9KTR6ZA046FX5APWA89
18629 3604 data/01CJ5TW4QBNDMDEMXYV7K529GP
18629 3604 data/01CJ5TXZMEPDY4EX6R8GV80P4Y
18629 3604 data/01CJ5TZY1CY7Z3AAH2NM4QDPG7
18629 3604 data/01CJ5V1S0EK00A08WG3W9K6BEQ
18629 3604 data/01CJ5V3PZVQC5VZTKMCK5KTED4
18629 3604 data/01CJ5V5J1P9SNBSKG45MXG7HDC
18629 3604 data/01CJ5V7CYTHJW1HBJHJAA2KC2G
18629 3604 data/01CJ5V9B9543GCZN2CT7K713V8
18629 3604 data/01CJ5VB9EJABPCFQSB7HYSPXAV
18629 3604 data/01CJ5VD4F3M713BJKM81JTKZGC
18629 3604 data/01CJ5VEZZJE01KMPG3Y54Q3Q06
18629 3604 data/01CJ5VGV8NDPDZ8FVRH43QRKPE
18629 3604 data/01CJ5VJPB6XA44AR5RPPQ4651G
18629 3604 data/01CJ5VMJVNK1M0XKMGH8KNA6WS
18629 3604 data/01CJ5VPFXYWS8B8FZH6BY9EANT
18629 3604 data/01CJ5VRAY5ZAD2WJZ7SRM22X2V
18629 3604 data/01CJ5VT64X07ZH6CEK1D589AMH
18629 3604 data/01CJ5VW14AS2D38S1X0JBXDB3E
18629 3604 data/01CJ5VXW5QQ5S4HEBH5MDEV6BA
18629 3604 data/01CJ5VZQ8H0EE3MN54ZR5D27MP
18629 3604 data/01CJ5W1JGQBP361SBQGRDTBVP3
18629 3604 data/01CJ5W3GFC8VJ032NY3KSR266V
18629 3604 data/01CJ5W5E71F2AADM41VFXF5Y35
18629 3604 data/01CJ5W79KNAGMYBM5KG3J0AMJF
18629 3604 data/01CJ5W94SNSMSRF1J1616CMMCN
18629 3604 data/01CJ5WB2KZX0WTKFY0NTMVRDQF
18629 3604 data/01CJ5WCXKPF6BXR3RWCC6DATCB
18629 3604 data/01CJ5WEVFCQ280P2SK4S0BVQSC
18629 3604 data/01CJ5WGQ1PV4WHKNM3HV5TTEMZ
18629 3604 data/01CJ5WJJ7R5SYXCENSCD1FMESK
18629 3604 data/01CJ5WMDBRJTKPR6ZVNJS45H5T
18629 3604 data/01CJ5WP8B614MNT7ZCKTQTWZ5P
18629 3604 data/01CJ5WR6970N2R48361RC1E9M4
18629 3604 data/01CJ5WT1HJJ0GJAB0F3ES2R4BR
18629 3604 data/01CJ5WW0AX8PM7XFVKMSWJG4SR
18629 3604 data/01CJ5WXVSSPD0K998Y3DZBVAR1
18629 3604 data/01CJ5WZPZ4S4PH435T1EFEM2AJ
18629 3604 data/01CJ5X1HZQRFWQEB2B7MZ3X2MB
18629 3604 data/01CJ5X3CZ1E8TSAA21020M5JVD
18629 3604 data/01CJ5X5B5NZTGMZPS7YCQKMK1X
18629 3604 data/01CJ5X76HHQ4NN2NQRGGJNPVKA
18629 3604 data/01CJ5X93CPC8Q5VV3G8ZZP0E6B
18629 3604 data/01CJ5XAZQNR9N7JCHEW70KCPV9
18629 3604 data/01CJ5XCYQ12MCNNNP5PZV4BPMV
18629 3604 data/01CJ5XESPDSFC11B377KJQHEP6
18629 3604 data/01CJ5XGMPJF29EP4JS72B453B8
18629 3604 data/01CJ5XJFY353M0CW4YCXJAS3EK
18629 3604 data/01CJ5XMB43A4G1K64M754HFHCM
18629 3604 data/01CJ5XP634WP7G1QEG1QWJ6DFS
18629 3604 data/01CJ5XR3G44FC9C7NXN4AB9Y7Q
18629 3604 data/01CJ5XT14X52EEAZD15QR7YPBT
18629 3604 data/01CJ5XVXB90DXHED0DY0Y4RKZC
18629 3604 data/01CJ5XXV435NRC1RN9PV6V2K80
18629 3604 data/01CJ5XZQ3V0KD7P1KTVZQXMTQC
18629 3604 data/01CJ5Y1J6EJMEQTVFGKZTF997H
18629 3604 data/01CJ5Y3D51HKM8XXA6AVAE6ZDF
version="(version=2.3.1, branch=HEAD, revision=188ca45bd85ce843071e768d855722a9d9dabe03)"
user=root@82ef94f1b8f7, date=20180619-15:56:22)"
0.1.el7.x86_64 #1 SMP Thu Jan 29 18:05:33 UTC 2015 x86_64 HCMDEMAZIND01 (none))"
"
healthy block" mint=1530547200000 maxt=1530576000000 ulid=01CHF863S201TBMD667KNN3TE3
healthy block" mint=1530576000000 maxt=1530640800000 ulid=01CHGZ3YMXBMM09NDKTJA2P2BH
healthy block" mint=1530640800000 maxt=1530705600000 ulid=01CHJWXF056J7YPX5ZP2SSYGW4
healthy block" mint=1530705600000 maxt=1530770400000 ulid=01CHMTQ21WA532AY7P68XZ02F8
healthy block" mint=1530770400000 maxt=1530835200000 ulid=01CHPRGN1E7W6VYQV032B95ZAA
healthy block" mint=1530835200000 maxt=1530900000000 ulid=01CHRPA0DHK5GTW7GV65GKTQC0
healthy block" mint=1530900000000 maxt=1530964800000 ulid=01CHTM3JCMQZQ5KJ66G8H5H4GE
healthy block" mint=1530964800000 maxt=1531029600000 ulid=01CHWHX2Y0CXWFWYD7C41MH7YB
healthy block" mint=1531029600000 maxt=1531094400000 ulid=01CHYFPPFQASSQ6N649EN75803
healthy block" mint=1531094400000 maxt=1531159200000 ulid=01CJ0DG8HAPPYAFDPY4KCH6ATM
healthy block" mint=1531159200000 maxt=1531224000000 ulid=01CJ2B9QHK9Z4909NNNB2S9SDR
healthy block" mint=1531288800000 maxt=1531296000000 ulid=01CJ493678EFRDSFZNPQDHXNAP
healthy block" mint=1531224000000 maxt=1531288800000 ulid=01CJ4936XK9AE031X8133MXDVT
healthy block" mint=1531296000000 maxt=1531303200000 ulid=01CJ4FYXFG0V5VM0G2FQXT6G3A
healthy block" mint=1531310400000 maxt=1531317600000 ulid=01CJ4XPD0BPWZKPJNW08JPC0DF
healthy block" mint=1531288800000 maxt=1531310400000 ulid=01CJ4XPDNXD5AWTB44DZ0QTGQG
healthy block" mint=1531317600000 maxt=1531324800000 ulid=01CJ56MZZ638P31GJX3NS9SW4N
listening for connections" address=0.0.0.0:9090
manager..."
manager..."
br> msg="Stopping rule manager..."
msg="Rule manager stopped"
notification manager..."
stopped"
stopped"
> invalid block sequence: block time ranges overlap: [mint: 1531288800000, maxt: 1531296000000, range: 2h0m0s, blocks: 2]: <ulid: 01CJ493678EFRDSFZNPQDHXNAP, mint: 1531288800000, maxt: 1531296000000, range: 2h0m0s>, <ulid: 01CJ4XPDNXD5AWTB44DZ0QTGQG, mint: 1531288800000, maxt: 1531310400000, range: 6h0m0s>\n[mint: 1531296000000, maxt: 1531303200000, range: 2h0m0s, blocks: 2]: <ulid: 01CJ4XPDNXD5AWTB44DZ0QTGQG, mint: 1531288800000, maxt: 1531310400000, range: 6h0m0s>, <ulid: 01CJ4FYXFG0V5VM0G2FQXT6G3A, mint: 1531296000000, maxt: 1531303200000, range: 2h0m0s>"

tvvignesh · 2018-07-12T05:39:14Z

Oh. It started working. I had to run the tool again and again and again and it kept deleting blocks. Now it works. Any way to prevent this from happening again?

krasi-georgiev · 2018-07-12T07:20:57Z

the 2.3.2 release has a fix for that.
#4370

Misterhex · 2018-08-13T03:36:04Z

Using 2.3.2 but still getting the same issue.

goroutine profile: total 42
5 @ 0x1532162 0x155ef66 0x155977f 0x1570230 0x459d11
#	0x1532161	github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/chunkenc.(*xorAppender).Append+0x891	/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/chunkenc/xor.go:131
#	0x155ef65	github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*memSeries).append+0x125		/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/head.go:1229
#	0x155977e	github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*Head).processWALSamples+0x16e	/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/head.go:219
#	0x157022f	github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*Head).ReadWAL.func1+0x5f		/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/head.go:256

5 @ 0x42ca7a 0x42cb2e 0x404262 0x403f5b 0x1559837 0x1570230 0x459d11
#	0x1559836	github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*Head).processWALSamples+0x226	/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/head.go:209
#	0x157022f	github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*Head).ReadWAL.func1+0x5f		/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/head.go:256

2 @ 0x40a1de 0x155e6a0 0x155971a 0x1570230 0x459d11
#	0x155e69f	github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*stripeSeries).getByID+0x7f		/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/head.go:1049
#	0x1559719	github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*Head).processWALSamples+0x109	/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/head.go:214
#	0x157022f	github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*Head).ReadWAL.func1+0x5f		/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/head.go:256

2 @ 0x470961 0x155e6bb 0x155971a 0x1570230 0x459d11
#	0x470960	sync.(*RWMutex).RUnlock+0x90										/usr/local/go/src/sync/rwmutex.go:62
#	0x155e6ba	github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*stripeSeries).getByID+0x9a		/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/head.go:1050
#	0x1559719	github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*Head).processWALSamples+0x109	/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/head.go:214
#	0x157022f	github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*Head).ReadWAL.func1+0x5f		/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/head.go:256

1 @ 0x1532651 0x1531c74 0x155ef66 0x155977f 0x1570230 0x459d11
#	0x1532650	github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/chunkenc.(*xorAppender).writeVDelta+0x4e0	/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/chunkenc/xor.go:188
#	0x1531c73	github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/chunkenc.(*xorAppender).Append+0x3a3		/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/chunkenc/xor.go:175
#	0x155ef65	github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*memSeries).append+0x125			/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/head.go:1229
#	0x155977e	github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*Head).processWALSamples+0x16e		/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/head.go:219
#	0x157022f	github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*Head).ReadWAL.func1+0x5f			/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/head.go:256

1 @ 0x15585ad 0x156ed7b 0x156d3b7 0x1568bef 0x1559be1 0x1550748 0x1577a33 0x1716f66 0x724dc7 0x459d11
#	0x15585ac	github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*decbuf).be64+0xec			/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/encoding_helpers.go:130
#	0x156ed7a	github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*walReader).decodeSamples+0x12a	/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/wal.go:1172
#	0x156d3b6	github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*walReader).Read+0x516		/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/wal.go:961
#	0x1568bee	github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*repairingWALReader).Read+0x5e	/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/wal.go:261
#	0x1559be0	github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*Head).ReadWAL+0x320		/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/head.go:308
#	0x1550747	github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.Open+0x4f7				/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/db.go:235
#	0x1577a32	github.com/prometheus/prometheus/storage/tsdb.Open+0x292						/go/src/github.com/prometheus/prometheus/storage/tsdb/tsdb.go:143
#	0x1716f65	main.main.func20+0x1f5											/go/src/github.com/prometheus/prometheus/cmd/prometheus/main.go:534
#	0x724dc6	github.com/prometheus/prometheus/vendor/github.com/oklog/oklog/pkg/group.(*Group).Run.func1+0x26	/go/src/github.com/prometheus/prometheus/vendor/github.com/oklog/oklog/pkg/group/group.go:38

1 @ 0x155e6d4 0x155971a 0x1570230 0x459d11
#	0x155e6d3	github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*stripeSeries).getByID+0xb3		/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/head.go:1045
#	0x1559719	github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*Head).processWALSamples+0x109	/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/head.go:214
#	0x157022f	github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*Head).ReadWAL.func1+0x5f		/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/head.go:256

1 @ 0x40f3b2 0x4412b6 0x6fd8f2 0x459d11
#	0x4412b5	os/signal.signal_recv+0xa5	/usr/local/go/src/runtime/sigqueue.go:139
#	0x6fd8f1	os/signal.loop+0x21		/usr/local/go/src/os/signal/signal_unix.go:22

1 @ 0x42ca7a 0x4045ec 0x403f1b 0x166cd2c 0x459d11
#	0x166cd2b	github.com/prometheus/prometheus/prompb.RegisterAdminHandlerFromEndpoint.func1.1+0x4b	/go/src/github.com/prometheus/prometheus/prompb/rpc.pb.gw.go:85

1 @ 0x42ca7a 0x427d9a 0x427417 0x49116b 0x4911ed 0x49204d 0x5171cf 0x52874a 0x169d70a 0x7d843a 0x167dfd0 0x471206 0x471378 0xcbb0bb 0xcbb944 0x167f400 0x16805f9 0x167e893 0x459d11
#	0x427416	internal/poll.runtime_pollWait+0x56									/usr/local/go/src/runtime/netpoll.go:173
#	0x49116a	internal/poll.(*pollDesc).wait+0x9a									/usr/local/go/src/internal/poll/fd_poll_runtime.go:85
#	0x4911ec	internal/poll.(*pollDesc).waitRead+0x3c									/usr/local/go/src/internal/poll/fd_poll_runtime.go:90
#	0x49204c	internal/poll.(*FD).Read+0x17c										/usr/local/go/src/internal/poll/fd_unix.go:157
#	0x5171ce	net.(*netFD).Read+0x4e											/usr/local/go/src/net/fd_unix.go:202
#	0x528749	net.(*conn).Read+0x69											/usr/local/go/src/net/net.go:176
#	0x167dfcf	github.com/prometheus/prometheus/vendor/github.com/cockroachdb/cmux.(*bufferedReader).Read+0x11f	/go/src/github.com/prometheus/prometheus/vendor/github.com/cockroachdb/cmux/buffer.go:42
#	0x471205	io.ReadAtLeast+0x85											/usr/local/go/src/io/io.go:309
#	0x471377	io.ReadFull+0x57											/usr/local/go/src/io/io.go:327
#	0xcbb0ba	github.com/prometheus/prometheus/vendor/golang.org/x/net/http2.readFrameHeader+0x7a			/go/src/github.com/prometheus/prometheus/vendor/golang.org/x/net/http2/frame.go:237
#	0xcbb943	github.com/prometheus/prometheus/vendor/golang.org/x/net/http2.(*Framer).ReadFrame+0xa3			/go/src/github.com/prometheus/prometheus/vendor/golang.org/x/net/http2/frame.go:492
#	0x167f3ff	github.com/prometheus/prometheus/vendor/github.com/cockroachdb/cmux.matchHTTP2Field+0x13f		/go/src/github.com/prometheus/prometheus/vendor/github.com/cockroachdb/cmux/matchers.go:145
#	0x16805f8	github.com/prometheus/prometheus/vendor/github.com/cockroachdb/cmux.HTTP2HeaderField.func1+0x58		/go/src/github.com/prometheus/prometheus/vendor/github.com/cockroachdb/cmux/matchers.go:111
#	0x167e892	github.com/prometheus/prometheus/vendor/github.com/cockroachdb/cmux.(*cMux).serve+0x1f2			/go/src/github.com/prometheus/prometheus/vendor/github.com/cockroachdb/cmux/cmux.go:143

1 @ 0x42ca7a 0x427d9a 0x427417 0x49116b 0x4911ed 0x49204d 0x5171cf 0x52874a 0x57fe48 0x471206 0x471378 0xcbb0bb 0xcbb944 0x160d36f 0x1601597 0x459d11
#	0x427416	internal/poll.runtime_pollWait+0x56									/usr/local/go/src/runtime/netpoll.go:173
#	0x49116a	internal/poll.(*pollDesc).wait+0x9a									/usr/local/go/src/internal/poll/fd_poll_runtime.go:85
#	0x4911ec	internal/poll.(*pollDesc).waitRead+0x3c									/usr/local/go/src/internal/poll/fd_poll_runtime.go:90
#	0x49204c	internal/poll.(*FD).Read+0x17c										/usr/local/go/src/internal/poll/fd_unix.go:157
#	0x5171ce	net.(*netFD).Read+0x4e											/usr/local/go/src/net/fd_unix.go:202
#	0x528749	net.(*conn).Read+0x69											/usr/local/go/src/net/net.go:176
#	0x57fe47	bufio.(*Reader).Read+0x237										/usr/local/go/src/bufio/bufio.go:216
#	0x471205	io.ReadAtLeast+0x85											/usr/local/go/src/io/io.go:309
#	0x471377	io.ReadFull+0x57											/usr/local/go/src/io/io.go:327
#	0xcbb0ba	github.com/prometheus/prometheus/vendor/golang.org/x/net/http2.readFrameHeader+0x7a			/go/src/github.com/prometheus/prometheus/vendor/golang.org/x/net/http2/frame.go:237
#	0xcbb943	github.com/prometheus/prometheus/vendor/golang.org/x/net/http2.(*Framer).ReadFrame+0xa3			/go/src/github.com/prometheus/prometheus/vendor/golang.org/x/net/http2/frame.go:492
#	0x160d36e	github.com/prometheus/prometheus/vendor/google.golang.org/grpc/transport.(*framer).readFrame+0x2e	/go/src/github.com/prometheus/prometheus/vendor/google.golang.org/grpc/transport/http_util.go:608
#	0x1601596	github.com/prometheus/prometheus/vendor/google.golang.org/grpc/transport.(*http2Client).reader+0x46	/go/src/github.com/prometheus/prometheus/vendor/google.golang.org/grpc/transport/http2_client.go:1080

1 @ 0x42ca7a 0x427d9a 0x427417 0x49116b 0x4911ed 0x4935e8 0x517ae2 0x53233e 0x530b29 0x169d323 0x7d7097 0x167e5a8 0x1709c51 0x459d11
#	0x427416	internal/poll.runtime_pollWait+0x56										/usr/local/go/src/runtime/netpoll.go:173
#	0x49116a	internal/poll.(*pollDesc).wait+0x9a										/usr/local/go/src/internal/poll/fd_poll_runtime.go:85
#	0x4911ec	internal/poll.(*pollDesc).waitRead+0x3c										/usr/local/go/src/internal/poll/fd_poll_runtime.go:90
#	0x4935e7	internal/poll.(*FD).Accept+0x1a7										/usr/local/go/src/internal/poll/fd_unix.go:372
#	0x517ae1	net.(*netFD).accept+0x41											/usr/local/go/src/net/fd_unix.go:238
#	0x53233d	net.(*TCPListener).accept+0x2d											/usr/local/go/src/net/tcpsock_posix.go:136
#	0x530b28	net.(*TCPListener).Accept+0x48											/usr/local/go/src/net/tcpsock.go:259
#	0x169d322	github.com/prometheus/prometheus/vendor/golang.org/x/net/netutil.(*limitListener).Accept+0x52			/go/src/github.com/prometheus/prometheus/vendor/golang.org/x/net/netutil/listen.go:30
#	0x7d7096	github.com/prometheus/prometheus/vendor/github.com/mwitkow/go-conntrack.(*connTrackListener).Accept+0x36	/go/src/github.com/prometheus/prometheus/vendor/github.com/mwitkow/go-conntrack/listener_wrapper.go:86
#	0x167e5a7	github.com/prometheus/prometheus/vendor/github.com/cockroachdb/cmux.(*cMux).Serve+0x87				/go/src/github.com/prometheus/prometheus/vendor/github.com/cockroachdb/cmux/cmux.go:124
#	0x1709c50	github.com/prometheus/prometheus/web.(*Handler).Run.func7+0x30							/go/src/github.com/prometheus/prometheus/web/web.go:495

1 @ 0x42ca7a 0x42cb2e 0x404262 0x403f1b 0x1715fae 0x724dc7 0x459d11
#	0x1715fad	main.main.func12+0x5d											/go/src/github.com/prometheus/prometheus/cmd/prometheus/main.go:429
#	0x724dc6	github.com/prometheus/prometheus/vendor/github.com/oklog/oklog/pkg/group.(*Group).Run.func1+0x26	/go/src/github.com/prometheus/prometheus/vendor/github.com/oklog/oklog/pkg/group/group.go:38

1 @ 0x42ca7a 0x42cb2e 0x404262 0x403f1b 0x17162af 0x724dc7 0x459d11
#	0x17162ae	main.main.func14+0x8e											/go/src/github.com/prometheus/prometheus/cmd/prometheus/main.go:453
#	0x724dc6	github.com/prometheus/prometheus/vendor/github.com/oklog/oklog/pkg/group.(*Group).Run.func1+0x26	/go/src/github.com/prometheus/prometheus/vendor/github.com/oklog/oklog/pkg/group/group.go:38

1 @ 0x42ca7a 0x42cb2e 0x404262 0x403f1b 0x1716cc8 0x724dc7 0x459d11
#	0x1716cc7	main.main.func18+0x47											/go/src/github.com/prometheus/prometheus/cmd/prometheus/main.go:517
#	0x724dc6	github.com/prometheus/prometheus/vendor/github.com/oklog/oklog/pkg/group.(*Group).Run.func1+0x26	/go/src/github.com/prometheus/prometheus/vendor/github.com/oklog/oklog/pkg/group/group.go:38

1 @ 0x42ca7a 0x42cb2e 0x404262 0x403f1b 0x171757e 0x724dc7 0x459d11
#	0x171757d	main.main.func24+0x5d											/go/src/github.com/prometheus/prometheus/cmd/prometheus/main.go:584
#	0x724dc6	github.com/prometheus/prometheus/vendor/github.com/oklog/oklog/pkg/group.(*Group).Run.func1+0x26	/go/src/github.com/prometheus/prometheus/vendor/github.com/oklog/oklog/pkg/group/group.go:38

1 @ 0x42ca7a 0x42cb2e 0x404262 0x403f1b 0x724c9c 0x1710ff9 0x42c622 0x459d11
#	0x724c9b	github.com/prometheus/prometheus/vendor/github.com/oklog/oklog/pkg/group.(*Group).Run+0xeb	/go/src/github.com/prometheus/prometheus/vendor/github.com/oklog/oklog/pkg/group/group.go:43
#	0x1710ff8	main.main+0x5458										/go/src/github.com/prometheus/prometheus/cmd/prometheus/main.go:595
#	0x42c621	runtime.main+0x211										/usr/local/go/src/runtime/proc.go:198

1 @ 0x42ca7a 0x42cb2e 0x404262 0x403f5b 0x1516cc0 0x1715a90 0x724dc7 0x459d11
#	0x1516cbf	github.com/prometheus/prometheus/discovery.(*Manager).Run+0x4f						/go/src/github.com/prometheus/prometheus/discovery/manager.go:93
#	0x1715a8f	main.main.func8+0x3f											/go/src/github.com/prometheus/prometheus/cmd/prometheus/main.go:397
#	0x724dc6	github.com/prometheus/prometheus/vendor/github.com/oklog/oklog/pkg/group.(*Group).Run.func1+0x26	/go/src/github.com/prometheus/prometheus/vendor/github.com/oklog/oklog/pkg/group/group.go:38

1 @ 0x42ca7a 0x42cb2e 0x404262 0x403f5b 0x1516cc0 0x1715d10 0x724dc7 0x459d11
#	0x1516cbf	github.com/prometheus/prometheus/discovery.(*Manager).Run+0x4f						/go/src/github.com/prometheus/prometheus/discovery/manager.go:93
#	0x1715d0f	main.main.func10+0x3f											/go/src/github.com/prometheus/prometheus/cmd/prometheus/main.go:411
#	0x724dc6	github.com/prometheus/prometheus/vendor/github.com/oklog/oklog/pkg/group.(*Group).Run.func1+0x26	/go/src/github.com/prometheus/prometheus/vendor/github.com/oklog/oklog/pkg/group/group.go:38

1 @ 0x42ca7a 0x42cb2e 0x404262 0x403f5b 0x15717bf 0x459d11
#	0x15717be	github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*walReader).Read.func1+0x7e	/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/wal.go:901

1 @ 0x42ca7a 0x42cb2e 0x404262 0x403f5b 0x168143b 0x161f8ac 0x1709bdf 0x459d11
#	0x168143a	github.com/prometheus/prometheus/vendor/github.com/cockroachdb/cmux.muxListener.Accept+0x5a	/go/src/github.com/prometheus/prometheus/vendor/github.com/cockroachdb/cmux/cmux.go:184
#	0x161f8ab	github.com/prometheus/prometheus/vendor/google.golang.org/grpc.(*Server).Serve+0x16b		/go/src/github.com/prometheus/prometheus/vendor/google.golang.org/grpc/server.go:432
#	0x1709bde	github.com/prometheus/prometheus/web.(*Handler).Run.func6+0x3e					/go/src/github.com/prometheus/prometheus/web/web.go:492

1 @ 0x42ca7a 0x42cb2e 0x404262 0x403f5b 0x168143b 0x68ef95 0x1709b5f 0x459d11
#	0x168143a	github.com/prometheus/prometheus/vendor/github.com/cockroachdb/cmux.muxListener.Accept+0x5a	/go/src/github.com/prometheus/prometheus/vendor/github.com/cockroachdb/cmux/cmux.go:184
#	0x68ef94	net/http.(*Server).Serve+0x1a4									/usr/local/go/src/net/http/server.go:2770
#	0x1709b5e	github.com/prometheus/prometheus/web.(*Handler).Run.func5+0x3e					/go/src/github.com/prometheus/prometheus/web/web.go:489

1 @ 0x42ca7a 0x42cb2e 0x404262 0x403f5b 0x7c4a6b 0x459d11
#	0x7c4a6a	github.com/prometheus/prometheus/vendor/github.com/golang/glog.(*loggingT).flushDaemon+0x8a	/go/src/github.com/prometheus/prometheus/vendor/github.com/golang/glog/glog.go:879

1 @ 0x42ca7a 0x43c590 0x156bdce 0x459d11
#	0x156bdcd	github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb.(*SegmentWAL).run+0x36d	/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/tsdb/wal.go:704

1 @ 0x42ca7a 0x43c590 0x1601ef2 0x459d11
#	0x1601ef1	github.com/prometheus/prometheus/vendor/google.golang.org/grpc/transport.(*http2Client).controller+0x121	/go/src/github.com/prometheus/prometheus/vendor/google.golang.org/grpc/transport/http2_client.go:1168

1 @ 0x42ca7a 0x43c590 0x161a6b0 0x1626fbf 0x459d11
#	0x161a6af	github.com/prometheus/prometheus/vendor/google.golang.org/grpc.(*addrConn).transportMonitor+0x1bf	/go/src/github.com/prometheus/prometheus/vendor/google.golang.org/grpc/clientconn.go:908
#	0x1626fbe	github.com/prometheus/prometheus/vendor/google.golang.org/grpc.(*ClientConn).resetAddrConn.func1+0x1ae	/go/src/github.com/prometheus/prometheus/vendor/google.golang.org/grpc/clientconn.go:637

1 @ 0x42ca7a 0x43c590 0x1705501 0x1717440 0x724dc7 0x459d11
#	0x1705500	github.com/prometheus/prometheus/web.(*Handler).Run+0xed0						/go/src/github.com/prometheus/prometheus/web/web.go:498
#	0x171743f	main.main.func22+0x3f											/go/src/github.com/prometheus/prometheus/cmd/prometheus/main.go:563
#	0x724dc6	github.com/prometheus/prometheus/vendor/github.com/oklog/oklog/pkg/group.(*Group).Run.func1+0x26	/go/src/github.com/prometheus/prometheus/vendor/github.com/oklog/oklog/pkg/group/group.go:38

1 @ 0x42ca7a 0x43c590 0x17157f1 0x724dc7 0x459d11
#	0x17157f0	main.main.func6+0x120											/go/src/github.com/prometheus/prometheus/cmd/prometheus/main.go:375
#	0x724dc6	github.com/prometheus/prometheus/vendor/github.com/oklog/oklog/pkg/group.(*Group).Run.func1+0x26	/go/src/github.com/prometheus/prometheus/vendor/github.com/oklog/oklog/pkg/group/group.go:38

1 @ 0x42ca7a 0x43c590 0x17169d3 0x724dc7 0x459d11
#	0x17169d2	main.main.func16+0x112											/go/src/github.com/prometheus/prometheus/cmd/prometheus/main.go:486
#	0x724dc6	github.com/prometheus/prometheus/vendor/github.com/oklog/oklog/pkg/group.(*Group).Run.func1+0x26	/go/src/github.com/prometheus/prometheus/vendor/github.com/oklog/oklog/pkg/group/group.go:38

1 @ 0x42ca7a 0x43c590 0x456a24 0x459d11
#	0x42ca79	runtime.gopark+0x119		/usr/local/go/src/runtime/proc.go:291
#	0x43c58f	runtime.selectgo+0xe4f		/usr/local/go/src/runtime/select.go:392
#	0x456a23	runtime.ensureSigM.func1+0x1f3	/usr/local/go/src/runtime/signal_unix.go:549

1 @ 0x6857b0 0x459d11
#	0x6857b0	net/http.(*connReader).backgroundRead+0x0	/usr/local/go/src/net/http/server.go:667

1 @ 0x6f4cd8 0x6f4ae0 0x6f15b4 0x6fcded 0x6fd171 0x17039da 0x68bdb4 0x1689f4d 0x68bdb4 0x1689af5 0x169d112 0x16990f1 0x169cdbc 0x68db20 0x168248b 0x68bdb4 0x68ebcc 0x68add1 0x459d11
#	0x6f4cd7	runtime/pprof.writeRuntimeProfile+0x97														/usr/local/go/src/runtime/pprof/pprof.go:679
#	0x6f4adf	runtime/pprof.writeGoroutine+0x9f														/usr/local/go/src/runtime/pprof/pprof.go:641
#	0x6f15b3	runtime/pprof.(*Profile).WriteTo+0x3e3														/usr/local/go/src/runtime/pprof/pprof.go:310
#	0x6fcdec	net/http/pprof.handler.ServeHTTP+0x20c														/usr/local/go/src/net/http/pprof/pprof.go:243
#	0x6fd170	net/http/pprof.Index+0x1d0															/usr/local/go/src/net/http/pprof/pprof.go:254
#	0x17039d9	github.com/prometheus/prometheus/web.serveDebug+0x199												/go/src/github.com/prometheus/prometheus/web/web.go:349
#	0x68bdb3	net/http.HandlerFunc.ServeHTTP+0x43														/usr/local/go/src/net/http/server.go:1947
#	0x1689f4c	github.com/prometheus/prometheus/vendor/github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerResponseSize.func1+0xec	/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/client_golang/prometheus/promhttp/instrument_server.go:196
#	0x68bdb3	net/http.HandlerFunc.ServeHTTP+0x43														/usr/local/go/src/net/http/server.go:1947
#	0x1689af4	github.com/prometheus/prometheus/vendor/github.com/prometheus/client_golang/prometheus/promhttp.InstrumentHandlerDuration.func2+0xb4		/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/client_golang/prometheus/promhttp/instrument_server.go:76
#	0x169d111	github.com/prometheus/prometheus/vendor/github.com/prometheus/common/route.(*Router).handle.func1+0x221						/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/common/route/route.go:60
#	0x16990f0	github.com/prometheus/prometheus/vendor/github.com/julienschmidt/httprouter.(*Router).ServeHTTP+0x6d0						/go/src/github.com/prometheus/prometheus/vendor/github.com/julienschmidt/httprouter/router.go:299
#	0x169cdbb	github.com/prometheus/prometheus/vendor/github.com/prometheus/common/route.(*Router).ServeHTTP+0x4b						/go/src/github.com/prometheus/prometheus/vendor/github.com/prometheus/common/route/route.go:98
#	0x68db1f	net/http.(*ServeMux).ServeHTTP+0x12f														/usr/local/go/src/net/http/server.go:2337
#	0x168248a	github.com/prometheus/prometheus/vendor/github.com/opentracing-contrib/go-stdlib/nethttp.Middleware.func2+0x3aa					/go/src/github.com/prometheus/prometheus/vendor/github.com/opentracing-contrib/go-stdlib/nethttp/server.go:74
#	0x68bdb3	net/http.HandlerFunc.ServeHTTP+0x43														/usr/local/go/src/net/http/server.go:1947
#	0x68ebcb	net/http.serverHandler.ServeHTTP+0xbb														/usr/local/go/src/net/http/server.go:2694
#	0x68add0	net/http.(*conn).serve+0x650															/usr/local/go/src/net/http/server.go:1830

krasi-georgiev · 2018-08-13T09:32:46Z

@Misterhex would you mind opening a new issue and include a more detailed report with minimal config and steps to reproduce and I can look into it.

lock · 2019-03-22T11:28:30Z

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

fabxc closed this as completed Sep 22, 2017

cauwulixuan mentioned this issue Jan 20, 2018

Opening storage failed" err="invalid block sequence" #3714

Closed

lock bot locked and limited conversation to collaborators Mar 22, 2019

Prometheus 2.0 fails to start up after couple of restarts #3191

Prometheus 2.0 fails to start up after couple of restarts #3191

Comments

JorritSalverda commented Sep 19, 2017

fabxc commented Sep 19, 2017 • edited

JorritSalverda commented Sep 19, 2017

JorritSalverda commented Sep 20, 2017

AndreaGiardini commented Sep 20, 2017

JorritSalverda commented Sep 20, 2017

fabxc commented Sep 20, 2017 • edited

fabxc commented Sep 20, 2017

JorritSalverda commented Sep 20, 2017

fabxc commented Sep 20, 2017

JorritSalverda commented Sep 20, 2017 • edited

fabxc commented Sep 20, 2017

JorritSalverda commented Sep 21, 2017

JorritSalverda commented Sep 21, 2017

JorritSalverda commented Sep 22, 2017

fabxc commented Sep 22, 2017

cauwulixuan commented Jan 20, 2018 • edited

Kirchen99 commented Apr 3, 2018

adilnaimi commented Apr 4, 2018

gouthamve commented Apr 4, 2018

adilnaimi commented Apr 4, 2018

gouthamve commented Apr 4, 2018

adilnaimi commented Apr 4, 2018

yanc0 commented May 30, 2018

krasi-georgiev commented May 30, 2018

yanc0 commented May 30, 2018

krasi-georgiev commented May 30, 2018

kaspernissen commented Jun 20, 2018

kaspernissen commented Jun 20, 2018

brian-brazil commented Jun 20, 2018

kaspernissen commented Jun 20, 2018

Neru007 commented Jun 21, 2018 • edited

RetroDreams commented Jun 28, 2018 • edited

brian-brazil commented Jun 28, 2018

tvvignesh commented Jul 12, 2018

tvvignesh commented Jul 12, 2018

krasi-georgiev commented Jul 12, 2018

Misterhex commented Aug 13, 2018 • edited

krasi-georgiev commented Aug 13, 2018

lock bot commented Mar 22, 2019

fabxc commented Sep 19, 2017 •

edited

fabxc commented Sep 20, 2017 •

edited

JorritSalverda commented Sep 20, 2017 •

edited

cauwulixuan commented Jan 20, 2018 •

edited

Neru007 commented Jun 21, 2018 •

edited

RetroDreams commented Jun 28, 2018 •

edited

Misterhex commented Aug 13, 2018 •

edited