Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cannot run the image by follwing readme #4

Open
irzhywau opened this issue Jan 17, 2020 · 2 comments
Open

Cannot run the image by follwing readme #4

irzhywau opened this issue Jan 17, 2020 · 2 comments

Comments

@irzhywau
Copy link

I tried to setup this exporter beside a httpd container using docker-compose. server status is available on http://httpd/status
and here is the httpd-metrics container that uses the image

  httpd-metrics:
    image: sergeykudrenko/prometheus-apache-exporter:latest
    environment:
      APACHE_EXPORTER_NAME: httpd
      APACHE_EXPORTER_URL: http://httpd/status?auto
      APACHE_EXPORTER_CLUSTERS: '{"source":"http-server"}'
      APACHE_URL_SUBSTRACT_RULES: '["?",";"," HTTP", "/img/"]'
    networks:
    - portal
    ports:
    - 9090:9345
httpd-metrics_1  | Traceback (most recent call last):
httpd-metrics_1  |   File "application.py", line 7, in <module>
httpd-metrics_1  |     REGISTRY.register(Collector())
httpd-metrics_1  |   File "/usr/local/lib/python3.6/site-packages/prometheus_client/registry.py", line 24, in register
httpd-metrics_1  |     names = self._get_names(collector)
httpd-metrics_1  |   File "/usr/local/lib/python3.6/site-packages/prometheus_client/registry.py", line 64, in _get_names
httpd-metrics_1  |     for metric in desc_func():
httpd-metrics_1  |   File "/prometheus-apache-exporter/collector.py", line 256, in collect
httpd-metrics_1  |     tmp_str = root.xpath("/html/body/dl[2]/dt[%d]" % x)[0].text.strip()
httpd-metrics_1  | IndexError: list index out of range
portal_httpd-metrics_1 exited with code 1

What is the mess here @SergeyKudrenko
@SergeyKudrenko
Copy link
Owner

Hello @irzhywau
Sorry for late reply and thanks for your feedback.
I'm not very familiar with docker-compose, because I run it in kubernetes environment.
I've tried to reproduce this error, but it did not work out.
Could you please provide me with your /server-status source page? This exporter parses a page in order to collect metrics, thus it could be dependent on the page layout.

Anyway, please find my scripts to run apache exporter with httpd:2.4

 sergeykudrenko@mac   ~  11:33:13  
$ mkdir tmp && cd tmp

 sergeykudrenko@mac   ~/tmp  11:33:18  
$ docker pull httpd:2.4
2.4: Pulling from library/httpd
Digest: sha256:769018135ba22d3a7a2b91cb89b8de711562cdf51ad6621b2b9b13e95f3798de
Status: Downloaded newer image for httpd:2.4
docker.io/library/httpd:2.4

 sergeykudrenko@mac   ~/tmp  11:33:33  
$ docker run --rm httpd:2.4 cat /usr/local/apache2/conf/httpd.conf > my-httpd.conf

 sergeykudrenko@mac   ~/tmp  11:33:43  
$ echo '<Location /server-status>
    SetHandler server-status
    Require all granted
</Location>' >> my-httpd.conf

 sergeykudrenko@mac   ~/tmp  11:33:54  
$ echo 'FROM httpd:2.4
COPY ./my-httpd.conf /usr/local/apache2/conf/httpd.conf' > dockerfile

 sergeykudrenko@mac   ~/tmp  11:34:07  
$ docker build -t apache-test -f Dockerfile .
Sending build context to Docker daemon  23.55kB
Step 1/2 : FROM httpd:2.4
 ---> c2aa7e16edd8
Step 2/2 : COPY ./my-httpd.conf /usr/local/apache2/conf/httpd.conf
 ---> c6526c374383
Successfully built c6526c374383
Successfully tagged apache-test:latest

 sergeykudrenko@mac   ~/tmp  11:34:42  
$ docker network create --subnet=192.168.1.100/29 apache-net
9b893daf4c0756ec4454e51ac0fd1a978b9d374ebd6ffedc81ea3d9c6e479a69

 sergeykudrenko@mac   ~/tmp  11:34:49  
$ docker run -dit --name my-apache-app --network apache-net -p 8080:80 -v "$PWD":/usr/local/apache2/htdocs/ apache-test
63632d91fac01ffa9103f6e58bbb6525d355961b7814f0b95d7de75baab0c088

 sergeykudrenko@mac   ~/tmp  11:35:02  
$ curl -XGET 'http://localhost:8080/server-status' | head -5
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3882  100  3882    0     0   758k      0 --:--:-- --:--:-- --:--:--  758k
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<html><head>
<title>Apache Status</title>
</head><body>
<h1>Apache Server Status for localhost (via 192.168.1.98)</h1>

 sergeykudrenko@mac   ~/tmp  11:35:30  
$ echo 'http://localhost:8080/server-status' > vegeta -duration=120 -rate=3/1s

 sergeykudrenko@mac   ~/tmp  11:35:45  
$ docker pull sergeykudrenko/prometheus-apache-exporter
Using default tag: latest
latest: Pulling from sergeykudrenko/prometheus-apache-exporter
Digest: sha256:527ed4eb67b47a4fccc08fc41e58df8942fe4d5638ad0495074b28ee2423187c
Status: Image is up to date for sergeykudrenko/prometheus-apache-exporter:latest
docker.io/sergeykudrenko/prometheus-apache-exporter:latest

 sergeykudrenko@mac   ~/tmp  11:37:56  
$ docker run -dit \
-p 9345:9345 \
-e APACHE_EXPORTER_NAME='my-apache-app' \
-e APACHE_EXPORTER_URL='http://my-apache-app:80/server-status' \
-e APACHE_EXPORTER_CLUSTERS='{"cluster1":"/html/body/table[5]/tr"}' \
-e APACHE_URL_SUBSTRACT_RULES='["?",";"," HTTP", "/img/"]' \
--network apache-net \
--name apache-exporter sergeykudrenko/prometheus-apache-exporter:latest
8bc262b44c9b2f5184e3c39b2d033dd18d644e145f03747a40df2c9590f52931

 sergeykudrenko@mac   ~/tmp  11:38:01  
$ curl -XGET 'http://localhost:9345/metrics'
# HELP python_gc_collected_objects Objects collected during gc
# TYPE python_gc_collected_objects histogram
python_gc_collected_objects_bucket{generation="1",le="500.0"} 1.0
python_gc_collected_objects_bucket{generation="1",le="1000.0"} 1.0
python_gc_collected_objects_bucket{generation="1",le="5000.0"} 1.0
python_gc_collected_objects_bucket{generation="1",le="10000.0"} 1.0
python_gc_collected_objects_bucket{generation="1",le="50000.0"} 1.0
python_gc_collected_objects_bucket{generation="1",le="+Inf"} 1.0
python_gc_collected_objects_count{generation="1"} 1.0
python_gc_collected_objects_sum{generation="1"} 0.0
python_gc_collected_objects_bucket{generation="0",le="500.0"} 2.0
python_gc_collected_objects_bucket{generation="0",le="1000.0"} 2.0
python_gc_collected_objects_bucket{generation="0",le="5000.0"} 2.0
python_gc_collected_objects_bucket{generation="0",le="10000.0"} 2.0
python_gc_collected_objects_bucket{generation="0",le="50000.0"} 2.0
python_gc_collected_objects_bucket{generation="0",le="+Inf"} 2.0
python_gc_collected_objects_count{generation="0"} 2.0
python_gc_collected_objects_sum{generation="0"} 0.0
# TYPE python_gc_collected_objects_created gauge
python_gc_collected_objects_created{generation="1"} 1.579945081617887e+09
python_gc_collected_objects_created{generation="0"} 1.5799450816253972e+09
# HELP python_gc_uncollectable_objects Uncollectable object found during GC
# TYPE python_gc_uncollectable_objects histogram
python_gc_uncollectable_objects_bucket{generation="1",le="500.0"} 1.0
python_gc_uncollectable_objects_bucket{generation="1",le="1000.0"} 1.0
python_gc_uncollectable_objects_bucket{generation="1",le="5000.0"} 1.0
python_gc_uncollectable_objects_bucket{generation="1",le="10000.0"} 1.0
python_gc_uncollectable_objects_bucket{generation="1",le="50000.0"} 1.0
python_gc_uncollectable_objects_bucket{generation="1",le="+Inf"} 1.0
python_gc_uncollectable_objects_count{generation="1"} 1.0
python_gc_uncollectable_objects_sum{generation="1"} 0.0
python_gc_uncollectable_objects_bucket{generation="0",le="500.0"} 2.0
python_gc_uncollectable_objects_bucket{generation="0",le="1000.0"} 2.0
python_gc_uncollectable_objects_bucket{generation="0",le="5000.0"} 2.0
python_gc_uncollectable_objects_bucket{generation="0",le="10000.0"} 2.0
python_gc_uncollectable_objects_bucket{generation="0",le="50000.0"} 2.0
python_gc_uncollectable_objects_bucket{generation="0",le="+Inf"} 2.0
python_gc_uncollectable_objects_count{generation="0"} 2.0
python_gc_uncollectable_objects_sum{generation="0"} 0.0
# TYPE python_gc_uncollectable_objects_created gauge
python_gc_uncollectable_objects_created{generation="1"} 1.579945081617919e+09
python_gc_uncollectable_objects_created{generation="0"} 1.5799450816254525e+09
# HELP python_gc_duration_seconds Time spent in garbage collection
# TYPE python_gc_duration_seconds histogram
python_gc_duration_seconds_bucket{generation="1",le="0.005"} 1.0
python_gc_duration_seconds_bucket{generation="1",le="0.01"} 1.0
python_gc_duration_seconds_bucket{generation="1",le="0.025"} 1.0
python_gc_duration_seconds_bucket{generation="1",le="0.05"} 1.0
python_gc_duration_seconds_bucket{generation="1",le="0.075"} 1.0
python_gc_duration_seconds_bucket{generation="1",le="0.1"} 1.0
python_gc_duration_seconds_bucket{generation="1",le="0.25"} 1.0
python_gc_duration_seconds_bucket{generation="1",le="0.5"} 1.0
python_gc_duration_seconds_bucket{generation="1",le="0.75"} 1.0
python_gc_duration_seconds_bucket{generation="1",le="1.0"} 1.0
python_gc_duration_seconds_bucket{generation="1",le="2.5"} 1.0
python_gc_duration_seconds_bucket{generation="1",le="5.0"} 1.0
python_gc_duration_seconds_bucket{generation="1",le="7.5"} 1.0
python_gc_duration_seconds_bucket{generation="1",le="10.0"} 1.0
python_gc_duration_seconds_bucket{generation="1",le="+Inf"} 1.0
python_gc_duration_seconds_count{generation="1"} 1.0
python_gc_duration_seconds_sum{generation="1"} 0.0005419254302978516
python_gc_duration_seconds_bucket{generation="0",le="0.005"} 2.0
python_gc_duration_seconds_bucket{generation="0",le="0.01"} 2.0
python_gc_duration_seconds_bucket{generation="0",le="0.025"} 2.0
python_gc_duration_seconds_bucket{generation="0",le="0.05"} 2.0
python_gc_duration_seconds_bucket{generation="0",le="0.075"} 2.0
python_gc_duration_seconds_bucket{generation="0",le="0.1"} 2.0
python_gc_duration_seconds_bucket{generation="0",le="0.25"} 2.0
python_gc_duration_seconds_bucket{generation="0",le="0.5"} 2.0
python_gc_duration_seconds_bucket{generation="0",le="0.75"} 2.0
python_gc_duration_seconds_bucket{generation="0",le="1.0"} 2.0
python_gc_duration_seconds_bucket{generation="0",le="2.5"} 2.0
python_gc_duration_seconds_bucket{generation="0",le="5.0"} 2.0
python_gc_duration_seconds_bucket{generation="0",le="7.5"} 2.0
python_gc_duration_seconds_bucket{generation="0",le="10.0"} 2.0
python_gc_duration_seconds_bucket{generation="0",le="+Inf"} 2.0
python_gc_duration_seconds_count{generation="0"} 2.0
python_gc_duration_seconds_sum{generation="0"} 0.00021076202392578125
# TYPE python_gc_duration_seconds_created gauge
python_gc_duration_seconds_created{generation="1"} 1.5799450816178222e+09
python_gc_duration_seconds_created{generation="0"} 1.579945081625326e+09
# HELP python_info Python platform information
# TYPE python_info gauge
python_info{implementation="CPython",major="3",minor="6",patchlevel="9",version="3.6.9"} 1.0
# HELP process_virtual_memory_bytes Virtual memory size in bytes.
# TYPE process_virtual_memory_bytes gauge
process_virtual_memory_bytes 3.6634624e+07
# HELP process_resident_memory_bytes Resident memory size in bytes.
# TYPE process_resident_memory_bytes gauge
process_resident_memory_bytes 3.2677888e+07
# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
# TYPE process_start_time_seconds gauge
process_start_time_seconds 1.5799450805e+09
# HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
# TYPE process_cpu_seconds_total counter
process_cpu_seconds_total 0.37
# HELP process_open_fds Number of open file descriptors.
# TYPE process_open_fds gauge
process_open_fds 10.0
# HELP process_max_fds Maximum number of open file descriptors.
# TYPE process_max_fds gauge
process_max_fds 1.048576e+06
# HELP apache_accesses_total Total requests served count since startup
# TYPE apache_accesses_total counter
apache_accesses_total{exporter_name="my-apache-app"} 24.0
# HELP apache_traffic_bytes_total Total bytes transfered since startup
# TYPE apache_traffic_bytes_total counter
apache_traffic_bytes_total{exporter_name="my-apache-app"} 165888.0
# HELP apache_balancer_requests_total Total requests count
# TYPE apache_balancer_requests_total counter
# HELP apache_balancer_write_bytes_total Total bytes written
# TYPE apache_balancer_write_bytes_total counter
# HELP apache_balancer_read_bytes_total Total bytes read
# TYPE apache_balancer_read_bytes_total counter
# HELP apache_requests_per_second Requests per second
# TYPE apache_requests_per_second gauge
apache_requests_per_second{exporter_name="my-apache-app"} 0.124
# HELP apache_io_bytes_per_second Bytes write/read per second
# TYPE apache_io_bytes_per_second gauge
apache_io_bytes_per_second{exporter_name="my-apache-app"} 859.0
# HELP apache_io_bytes_per_request Bytes write/read  per request
# TYPE apache_io_bytes_per_request gauge
apache_io_bytes_per_request{exporter_name="my-apache-app"} 6963.2
# HELP apache_balancer_route_ok Balancing status of the route is OK
# TYPE apache_balancer_route_ok gauge
# HELP apache_balancer_route_disabled Balancing status of the route is DISABLED
# TYPE apache_balancer_route_disabled gauge
# HELP apache_balancer_route_error Balancing status of the route is ERROR
# TYPE apache_balancer_route_error gauge
# HELP apache_balancer_route_unknown Balancing status of the route is UNKNOWN
# TYPE apache_balancer_route_unknown gauge
# HELP apache_scoreboard_current Count of workers grouped by status
# TYPE apache_scoreboard_current gauge
apache_scoreboard_current{exporter_name="my-apache-app",status="Waiting for Connection"} 98.0
apache_scoreboard_current{exporter_name="my-apache-app",status="Reading Request"} 1.0
apache_scoreboard_current{exporter_name="my-apache-app",status="Sending Reply"} 1.0
apache_scoreboard_current{exporter_name="my-apache-app",status="Open slot"} 300.0
# HELP apache_latest_scrape_duration_seconds Latest scrape duration in seconds
# TYPE apache_latest_scrape_duration_seconds gauge
apache_latest_scrape_duration_seconds{exporter_name="my-apache-app",metric_name="apache_accesses_total"} 0.000237
apache_latest_scrape_duration_seconds{exporter_name="my-apache-app",metric_name="apache_traffic_bytes_total"} 0.000237
apache_latest_scrape_duration_seconds{exporter_name="my-apache-app",metric_name="apache_requests_per_second"} 0.00025
apache_latest_scrape_duration_seconds{exporter_name="my-apache-app",metric_name="apache_io_bytes_per_second"} 0.00025
apache_latest_scrape_duration_seconds{exporter_name="my-apache-app",metric_name="apache_io_bytes_per_request"} 0.00025
apache_latest_scrape_duration_seconds{exporter_name="my-apache-app",metric_name="apache_scoreboard_current"} 0.000189
apache_latest_scrape_duration_seconds{exporter_name="my-apache-app",metric_name="apache_balancer_route_ok"} 4.6e-05
apache_latest_scrape_duration_seconds{exporter_name="my-apache-app",metric_name="apache_balancer_route_disabled"} 4.6e-05
apache_latest_scrape_duration_seconds{exporter_name="my-apache-app",metric_name="apache_balancer_route_error"} 4.6e-05
apache_latest_scrape_duration_seconds{exporter_name="my-apache-app",metric_name="apache_balancer_route_unknown"} 4.6e-05
apache_latest_scrape_duration_seconds{exporter_name="my-apache-app",metric_name="apache_balancer_requests_total"} 4.6e-05
apache_latest_scrape_duration_seconds{exporter_name="my-apache-app",metric_name="apache_balancer_write_bytes_total"} 4.6e-05
apache_latest_scrape_duration_seconds{exporter_name="my-apache-app",metric_name="apache_balancer_read_bytes_total"} 4.6e-05
apache_latest_scrape_duration_seconds{exporter_name="my-apache-app",metric_name="apache_endpoint_response_time_seconds"} 5.1e-05
# HELP apache_operation_duration_seconds Operation duration in seconds
# TYPE apache_operation_duration_seconds gauge
apache_operation_duration_seconds{exporter_name="my-apache-app",operation="load_page"} 0.00458
apache_operation_duration_seconds{exporter_name="my-apache-app",operation="parse_page"} 0.0011

Still I found a minor bug with a parsing server-status page in httpd:2.4. Gonna fix it soon.

@lopb
Copy link

lopb commented Sep 19, 2022 •
edited
Loading

Hello there,

I think the error @irzhywau saw (list index out of range) is related to the APACHE_EXPORTER_CLUSTERS parameter, I share the same error and couldn't reproduce the above steps to replicate a working environment.

But I was going to ask two other things:

1 - The "apache_endpoint_response_time_seconds" metric, how is it getting genereted since the mod_status doesn't seem to have this by default? Do you parse the access.log (similar to apachetop) in some way?

2 - on the collector.py file, line 667, there is the following line:
self.endpoint_stats = os.environ['APACHE_ENDPOINT_STATISTICS']
What is this?

Thank you, @SergeyKudrenko !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@irzhywau @SergeyKudrenko @lopb