Error 500 on /metrics page with Google Cloud Memorystore #315

mcinquin · 2019-09-26T12:32:32Z

Describe the problem
Redis_exporter is configured to monitor Google Cloud Memorystore based on Redis.
When i try to get the /metrics page, i have an error 500 as result.

EDIT:
The CONFIG command is blocked by Google in their redis version

What version of redis_exporter are you running?
[ ] 0.3x.x
[X] 1.x.x
https://github.com/oliver006/redis_exporter/releases/download/v1.2.1/redis_exporter-v1.2.1.linux-amd64.tar.gz

Running the exporter
/usr/local/bin/redis_exporter --redis-only-metrics --include-system-metrics --redis.addr=X.X.X.X

Expected behavior
The /metrics page available and accessible.

EDIT:
Do not throw an exception if the CONFIG command or another doesn't work

Did you upgrade from 0.3x.x to 1.0 and are scraping multiple hosts?
No

Screenshots
Log with debug parameter

DEBU[0000] Enabling debug output                        
DEBU[0000] keys: []main.dbKeyPair(nil)                  
DEBU[0000] singleKeys: []main.dbKeyPair(nil)            
INFO[0000] Providing metrics at :9121/metrics           
DEBU[0000] Configured redis addr: "X.X.X.X"          
DEBU[0023] Trying DialURL(): redis://X.X.X.X         
DEBU[0023] connected to: X.X.X.X                     
DEBU[0023] c.Do() - running command: CLIENT [SETNAME redis_exporter] 
DEBU[0023] c.Do() - err: ERR unknown command `CLIENT`, with args beginning with: `SETNAME`, `redis_exporter`,  
DEBU[0023] c.Do() - done                                
ERRO[0023] Couldn't set client name, err: ERR unknown command `CLIENT`, with args beginning with: `SETNAME`, `redis_exporter`,  
DEBU[0023] Redis CONFIG err: ERR unknown command `CONFIG`, with args beginning with: `GET`, `*`,  
DEBU[0023] c.Do() - running command: INFO [ALL]         
DEBU[0023] c.Do() - done                                
DEBU[0023] info: # Server                               
DEBU[0023] info: redis_version:4.0.14                   
DEBU[0023] info: redis_git_sha1:00000000                
DEBU[0023] info: redis_git_dirty:0                      
DEBU[0023] info: redis_build_id:a9dfc400783db9fd        
DEBU[0023] info: redis_mode:standalone                  
DEBU[0023] info: os:Linux 4.14.94+ x86_64               
DEBU[0023] info: arch_bits:64                           
DEBU[0023] info: multiplexing_api:epoll                 
DEBU[0023] info: atomicvar_api:atomic-builtin           
DEBU[0023] info: gcc_version:6.3.0                      
DEBU[0023] info: process_id:1                           
DEBU[0023] info: run_id:04ffff67345d3201766e73ae398db0c729d5f78d 
DEBU[0023] info: tcp_port:6379                          
DEBU[0023] info: uptime_in_seconds:2071899              
DEBU[0023] info: uptime_in_days:23                      
DEBU[0023] info: hz:10                                  
DEBU[0023] info: lru_clock:9219892                      
DEBU[0023] info: executable:/data/redis-server          
DEBU[0023] info: config_file:/usr/local/etc/redis/redis.conf 
DEBU[0023] info:                                        
DEBU[0023] info: # Clients                              
DEBU[0023] info: connected_clients:10                   
DEBU[0023] info: client_longest_output_list:0           
DEBU[0023] info: client_biggest_input_buf:0             
DEBU[0023] info: blocked_clients:0                      
DEBU[0023] info:                                        
DEBU[0023] info: # Memory                               
DEBU[0023] info: used_memory:121377240                  
DEBU[0023] info: used_memory_human:115.75M              
DEBU[0023] info: used_memory_rss:12390400               
DEBU[0023] info: used_memory_rss_human:11.82M           
DEBU[0023] info: used_memory_peak:121558616             
DEBU[0023] info: used_memory_peak_human:115.93M         
DEBU[0023] info: used_memory_peak_perc:99.85%           
DEBU[0023] info: used_memory_overhead:118445490         
DEBU[0023] info: used_memory_startup:786680             
DEBU[0023] info: used_memory_dataset:2931750            
DEBU[0023] info: used_memory_dataset_perc:2.43%         
DEBU[0023] info: total_system_memory:3153059840         
DEBU[0023] info: total_system_memory_human:2.94G        
DEBU[0023] info: used_memory_lua:37888                  
DEBU[0023] info: used_memory_lua_human:37.00K           
DEBU[0023] info: maxmemory:1073741824                   
DEBU[0023] info: maxmemory_human:1.00G                  
DEBU[0023] info: maxmemory_policy:volatile-lru          
DEBU[0023] info: mem_fragmentation_ratio:0.10           
DEBU[0023] info: mem_allocator:jemalloc-4.0.3           
DEBU[0023] info: active_defrag_running:0                
DEBU[0023] info: lazyfree_pending_objects:0             
DEBU[0023] info:                                        
DEBU[0023] info: # Persistence                          
DEBU[0023] info: loading:0                              
DEBU[0023] info: rdb_changes_since_last_save:0          
DEBU[0023] info: rdb_bgsave_in_progress:0               
DEBU[0023] info: rdb_last_save_time:1567429081          
DEBU[0023] info: rdb_last_bgsave_status:ok              
DEBU[0023] info: rdb_last_bgsave_time_sec:-1            
DEBU[0023] info: rdb_current_bgsave_time_sec:-1         
DEBU[0023] info: rdb_last_cow_size:258048               
DEBU[0023] info: aof_enabled:0                          
DEBU[0023] info: aof_rewrite_in_progress:0              
DEBU[0023] info: aof_rewrite_scheduled:0                
DEBU[0023] info: aof_last_rewrite_time_sec:-1           
DEBU[0023] info: aof_current_rewrite_time_sec:-1        
DEBU[0023] info: aof_last_bgrewrite_status:ok           
DEBU[0023] info: aof_last_write_status:ok               
DEBU[0023] info: aof_last_cow_size:0                    
DEBU[0023] info:                                        
DEBU[0023] info: # Stats                                
DEBU[0023] info: total_connections_received:71685       
DEBU[0023] info: total_commands_processed:6663695       
DEBU[0023] info: instantaneous_ops_per_sec:3            
DEBU[0023] info: total_net_input_bytes:158057472        
DEBU[0023] info: total_net_output_bytes:806350920       
DEBU[0023] info: instantaneous_input_kbps:0.08          
DEBU[0023] info: instantaneous_output_kbps:0.18         
DEBU[0023] info: rejected_connections:0                 
DEBU[0023] info: sync_full:1                            
DEBU[0023] info: sync_partial_ok:0                      
DEBU[0023] info: sync_partial_err:0                     
DEBU[0023] info: expired_keys:0                         
DEBU[0023] info: expired_stale_perc:0.00                
DEBU[0023] info: expired_time_cap_reached_count:0       
DEBU[0023] info: evicted_keys:0                         
DEBU[0023] info: keyspace_hits:0                        
DEBU[0023] info: keyspace_misses:0                      
DEBU[0023] info: pubsub_channels:1                      
DEBU[0023] info: pubsub_patterns:0                      
DEBU[0023] info: latest_fork_usec:271                   
DEBU[0023] info: migrate_cached_sockets:0               
DEBU[0023] info: slave_expires_tracked_keys:0           
DEBU[0023] info: active_defrag_hits:0                   
DEBU[0023] info: active_defrag_misses:0                 
DEBU[0023] info: active_defrag_key_hits:0               
DEBU[0023] info: active_defrag_key_misses:0             
DEBU[0023] info:                                        
DEBU[0023] info: # Replication                          
DEBU[0023] info: role:master                            
DEBU[0023] info: connected_slaves:1                     
DEBU[0023] info: slave0:ip=X.X.X.X,port=6379,state=online,offset=5785738,lag=0 
DEBU[0023] info: master_replid:01154cc45a16bb9d28efafab2747c8443f35e2b1 
DEBU[0023] info: master_replid2:0000000000000000000000000000000000000000 
DEBU[0023] info: master_repl_offset:5785738             
DEBU[0023] info: second_repl_offset:-1                  
DEBU[0023] info: repl_backlog_active:1                  
DEBU[0023] info: repl_backlog_size:107374182            
DEBU[0023] info: repl_backlog_first_byte_offset:1       
DEBU[0023] info: repl_backlog_histlen:5785738           
DEBU[0023] info:                                        
DEBU[0023] info: # CPU                                  
DEBU[0023] info: used_cpu_sys:2856.29                   
DEBU[0023] info: used_cpu_user:1254.31                  
DEBU[0023] info: used_cpu_sys_children:0.00             
DEBU[0023] info: used_cpu_user_children:0.00            
DEBU[0023] info:                                        
DEBU[0023] info: # Commandstats                         
DEBU[0023] info: cmdstat_role:calls=4143707,usec=102640226,usec_per_call=24.77 
DEBU[0023] info: cmdstat_psync:calls=4,usec=103,usec_per_call=25.75 
DEBU[0023] info: cmdstat_latency:calls=2544,usec=17829,usec_per_call=7.01 
DEBU[0023] info: cmdstat_ping:calls=29326,usec=79016,usec_per_call=2.69 
DEBU[0023] info: cmdstat_keys:calls=1,usec=33,usec_per_call=33.00 
DEBU[0023] info: cmdstat_info:calls=313398,usec=24035173,usec_per_call=76.69 
DEBU[0023] info: cmdstat_slaveof:calls=34534,usec=70997,usec_per_call=2.06 
DEBU[0023] info: cmdstat_subscribe:calls=16,usec=284,usec_per_call=17.75 
DEBU[0023] info: cmdstat_slowlog:calls=5088,usec=107901,usec_per_call=21.21 
DEBU[0023] info: cmdstat_replconf:calls=2065934,usec=3467737,usec_per_call=1.68 
DEBU[0023] info: cmdstat_config:calls=69086,usec=25363935,usec_per_call=367.14 
DEBU[0023] info: cmdstat_select:calls=57,usec=132,usec_per_call=2.32 
DEBU[0023] info:                                        
DEBU[0023] info: # Cluster                              
DEBU[0023] info: cluster_enabled:0                      
DEBU[0023] info:                                        
DEBU[0023] info: # Keyspace                             
DEBU[0023] info:                                        
DEBU[0023] info: # oom-prevention                       
DEBU[0023] info: oom_prevention_on:No                   
DEBU[0023] info: current_useconds_with_oom_prevention_on:0 
DEBU[0023] info: total_useconds_with_oom_prevention_on:0 
DEBU[0023] info: oom_prevention_threshold:1256277934    
DEBU[0023] info: oom_prevention_threshold_human:1.17G   
DEBU[0023] info: used_memory_total:12390400             
DEBU[0023] info: used_memory_total_human:11.82M         
DEBU[0023] info: peak_used_memory_total:12390400        
DEBU[0023] info: peak_used_memory_total_human:11.82M    
DEBU[0023] info: used_memory_rss:12390400               
DEBU[0023] info: used_memory_rss_human:11.82M           
DEBU[0023] info: used_memory_aof:0                      
DEBU[0023] info: used_memory_aof_human:0B               
DEBU[0023] info: used_memory_rdb:0                      
DEBU[0023] info: used_memory_rdb_human:0B               
DEBU[0023] info:                                        
DEBU[0023] c.Do() - running command: LATENCY [LATEST]   
DEBU[0023] c.Do() - done                                
DEBU[0023] keys: []main.dbKeyPair(nil)                  
DEBU[0023] e.singleKeys: []main.dbKeyPair(nil)          
DEBU[0023] e.keys: []main.dbKeyPair(nil)                
DEBU[0023] allKeys: []main.dbKeyPair{}                  
DEBU[0023] scrapeRedisHost() done

The text was updated successfully, but these errors were encountered:

oliver006 · 2019-09-26T14:27:30Z

Thanks for the detailed bug report, that's much appreciated.

From the log output, I can't really tell why it's failing. In fact, seeing scrapeRedisHost() done in the very end makes it look like the scrape successfully finished. Is there no other output? No stack trace maybe?
I don't have easy access to a Google Memorystore instance right now so reproducing on my end is a little tricky.

mcinquin · 2019-09-26T14:33:38Z

There is no more output or stack trace.
I have a error 500 for each scrape on the metrics page
500 Internal Server Error 2019-09-26 14:12:30 ERROR 500: Internal Server Error.

I think the issue can be related the issue #13 .
On many Cloud provider some REDIS command are blocked. A workaround can be to not throw an exception if the command doesn't respond, ignore it and continue the process.

I have full access to Google Cloud Memorystore and ready to make tests if you want.

oliver006 · 2019-09-26T18:39:21Z

Thanks. You're right, some cloud redis instances don't allow for certain CONFIG commands but in the logs you see that the INFO [ALL] succeeds so the exporter should be able to pull at least some info out of the redis instance and not just crash with a 500.

But what still throws me off is that scrapeRedisHost() done suggests that the scrape finishes and that there's trace or anything.

When you observe the 500, is that via curl/wget or in the prometheus targets page?

mcinquin · 2019-09-26T19:33:47Z

With Prometheus:
server returned HTTP status 500 Internal Server Error

With wget (debug activated):

wget -d http://127.0.0.1:9121/metrics
DEBUG output created by Wget 1.19.4 on linux-gnu.

Reading HSTS entries from /home/mcinquin/.wget-hsts
URI encoding = ‘UTF-8’
Converted file name 'metrics' (UTF-8) -> 'metrics' (UTF-8)
--2019-09-26 19:29:29--  http://127.0.0.1:9121/metrics
Connecting to 127.0.0.1:9121... connected.
Created socket 3.
Releasing 0x000056311c067e60 (new refcount 0).
Deleting unused 0x000056311c067e60.

---request begin---
GET /metrics HTTP/1.1
User-Agent: Wget/1.19.4 (linux-gnu)
Accept: */*
Accept-Encoding: identity
Host: 127.0.0.1:9121
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response... 
---response begin---
HTTP/1.1 500 Internal Server Error
Content-Type: text/plain; charset=utf-8
X-Content-Type-Options: nosniff
Date: Thu, 26 Sep 2019 19:29:29 GMT
Content-Length: 183

---response end---
500 Internal Server Error
Registered socket 3 for persistent reuse.
URI content encoding = ‘utf-8’
Skipping 183 bytes of body: [An error has occurred while serving metrics:

collected metric "redis_memory_used_rss_bytes" { gauge:<value:1.2660736e+07 > } was collected before with the same name and label values
] done.
2019-09-26 19:29:29 ERROR 500: Internal Server Error.

wget with debug mode shows more relevant information !

oliver006 · 2019-09-27T14:57:29Z

Great, I see the error now:

collected metric "redis_memory_used_rss_bytes" { gauge:<value:1.2660736e+07 > } was collected before with the same name and label values

Let me dig into why this metric is being collected twice.

oliver006 · 2019-09-27T15:03:26Z

Ugh, I see it now, there's a section called # oom-prevention which has another entry for used_memory_rss, hence the double recording of the metric.

mcinquin · 2019-09-27T19:49:00Z

Glad to read that the problem is identified. So what are the options to fix that?

oliver006 · 2019-09-27T20:59:32Z

So what are the options to fix that?

I was thinking about keeping track of what was recorded already, I'll try to code something up over the next couple of days.

mcinquin · 2019-09-27T21:11:57Z

Great news !
Feel free to contact me if you want me to run tests on Memorystore.

oliver006 · 2019-10-02T23:53:51Z

I opened PR #318 to fix this.
I build and pushed oliver006/redis_exporter:tst to the docker registry - can you give it a try and see if it works and if yes i'll merge the PR and cut a proper release?

adrilo · 2019-10-03T11:48:37Z

@oliver006 I had the same problem. I just tested it and it worked :)
Feel free to merge and cut a new release whenever you can.

oliver006 · 2019-10-03T15:08:21Z

Thanks for the feedback @adrilo - merging.

oliver006 · 2019-10-03T15:23:15Z

Released as https://github.com/oliver006/redis_exporter/releases/tag/v1.3.0

mcinquin · 2019-10-04T07:40:23Z

Hi,

Sorry I couldn't answer earlier.
I just tested with the release 1.3.0 and i still have the same issue

/usr/local/bin/redis_exporter -redis.addr X.X.X.X:6379 --redis-only-metrics -debug
INFO[0000] Redis Metrics Exporter v1.3.0    build date: 2019-10-03-15:14:39    sha1: 2aadaf75ed0a32d57b24427a401c810eb780091f    Go: go1.13.1    GOOS: linux    GOARCH: amd64 
DEBU[0000] Enabling debug output                        
DEBU[0000] keys: []main.dbKeyPair(nil)                  
DEBU[0000] singleKeys: []main.dbKeyPair(nil)            
INFO[0000] Providing metrics at :9121/metrics           
DEBU[0000] Configured redis addr: "X.X.X.X:6379"     
DEBU[0002] Trying DialURL(): redis://X.X.X.X:6379    
DEBU[0002] connected to: X.X.X.X:6379                
DEBU[0002] c.Do() - running command: CLIENT [SETNAME redis_exporter] 
DEBU[0002] c.Do() - err: ERR unknown command `CLIENT`, with args beginning with: `SETNAME`, `redis_exporter`,  
DEBU[0002] c.Do() - done                                
ERRO[0002] Couldn't set client name, err: ERR unknown command `CLIENT`, with args beginning with: `SETNAME`, `redis_exporter`,  
DEBU[0002] Redis CONFIG err: ERR unknown command `CONFIG`, with args beginning with: `GET`, `*`,  
DEBU[0002] c.Do() - running command: INFO [ALL]         
DEBU[0002] c.Do() - done                                
DEBU[0002] info: # Server                               
DEBU[0002] info: redis_version:4.0.14                   
DEBU[0002] info: redis_git_sha1:00000000                
DEBU[0002] info: redis_git_dirty:0                      
DEBU[0002] info: redis_build_id:a9dfc400783db9fd        
DEBU[0002] info: redis_mode:standalone                  
DEBU[0002] info: os:Linux 4.14.94+ x86_64               
DEBU[0002] info: arch_bits:64                           
DEBU[0002] info: multiplexing_api:epoll                 
DEBU[0002] info: atomicvar_api:atomic-builtin           
DEBU[0002] info: gcc_version:6.3.0                      
DEBU[0002] info: process_id:1                           
DEBU[0002] info: run_id:04ffff67345d3201766e73ae398db0c729d5f78d 
DEBU[0002] info: tcp_port:6379                          
DEBU[0002] info: uptime_in_seconds:2745387              
DEBU[0002] info: uptime_in_days:31                      
DEBU[0002] info: hz:10                                  
DEBU[0002] info: lru_clock:9893380                      
DEBU[0002] info: executable:/data/redis-server          
DEBU[0002] info: config_file:/usr/local/etc/redis/redis.conf 
DEBU[0002] info:                                        
DEBU[0002] info: # Clients                              
DEBU[0002] info: connected_clients:10                   
DEBU[0002] info: client_longest_output_list:0           
DEBU[0002] info: client_biggest_input_buf:0             
DEBU[0002] info: blocked_clients:0                      
DEBU[0002] info:                                        
DEBU[0002] info: # Memory                               
DEBU[0002] info: used_memory:121398968                  
DEBU[0002] info: used_memory_human:115.78M              
DEBU[0002] info: used_memory_rss:13848576               
DEBU[0002] info: used_memory_rss_human:13.21M           
DEBU[0002] info: used_memory_peak:121667240             
DEBU[0002] info: used_memory_peak_human:116.03M         
DEBU[0002] info: used_memory_peak_perc:99.78%           
DEBU[0002] info: used_memory_overhead:118445834         
DEBU[0002] info: used_memory_startup:786680             
DEBU[0002] info: used_memory_dataset:2953134            
DEBU[0002] info: used_memory_dataset_perc:2.45%         
DEBU[0002] info: total_system_memory:3153059840         
DEBU[0002] info: total_system_memory_human:2.94G        
DEBU[0002] info: used_memory_lua:37888                  
DEBU[0002] info: used_memory_lua_human:37.00K           
DEBU[0002] info: maxmemory:1073741824                   
DEBU[0002] info: maxmemory_human:1.00G                  
DEBU[0002] info: maxmemory_policy:volatile-lru          
DEBU[0002] info: mem_fragmentation_ratio:0.11           
DEBU[0002] info: mem_allocator:jemalloc-4.0.3           
DEBU[0002] info: active_defrag_running:0                
DEBU[0002] info: lazyfree_pending_objects:0             
DEBU[0002] info:                                        
DEBU[0002] info: # Persistence                          
DEBU[0002] info: loading:0                              
DEBU[0002] info: rdb_changes_since_last_save:16         
DEBU[0002] info: rdb_bgsave_in_progress:0               
DEBU[0002] info: rdb_last_save_time:1567429081          
DEBU[0002] info: rdb_last_bgsave_status:ok              
DEBU[0002] info: rdb_last_bgsave_time_sec:-1            
DEBU[0002] info: rdb_current_bgsave_time_sec:-1         
DEBU[0002] info: rdb_last_cow_size:258048               
DEBU[0002] info: aof_enabled:0                          
DEBU[0002] info: aof_rewrite_in_progress:0              
DEBU[0002] info: aof_rewrite_scheduled:0                
DEBU[0002] info: aof_last_rewrite_time_sec:-1           
DEBU[0002] info: aof_current_rewrite_time_sec:-1        
DEBU[0002] info: aof_last_bgrewrite_status:ok           
DEBU[0002] info: aof_last_write_status:ok               
DEBU[0002] info: aof_last_cow_size:0                    
DEBU[0002] info:                                        
DEBU[0002] info: # Stats                                
DEBU[0002] info: total_connections_received:94556       
DEBU[0002] info: total_commands_processed:9054229       
DEBU[0002] info: instantaneous_ops_per_sec:5            
DEBU[0002] info: total_net_input_bytes:212671608        
DEBU[0002] info: total_net_output_bytes:1079425758      
DEBU[0002] info: instantaneous_input_kbps:0.10          
DEBU[0002] info: instantaneous_output_kbps:3.19         
DEBU[0002] info: rejected_connections:0                 
DEBU[0002] info: sync_full:1                            
DEBU[0002] info: sync_partial_ok:0                      
DEBU[0002] info: sync_partial_err:0                     
DEBU[0002] info: expired_keys:0                         
DEBU[0002] info: expired_stale_perc:0.00                
DEBU[0002] info: expired_time_cap_reached_count:0       
DEBU[0002] info: evicted_keys:0                         
DEBU[0002] info: keyspace_hits:0                        
DEBU[0002] info: keyspace_misses:0                      
DEBU[0002] info: pubsub_channels:1                      
DEBU[0002] info: pubsub_patterns:0                      
DEBU[0002] info: latest_fork_usec:271                   
DEBU[0002] info: migrate_cached_sockets:0               
DEBU[0002] info: slave_expires_tracked_keys:0           
DEBU[0002] info: active_defrag_hits:0                   
DEBU[0002] info: active_defrag_misses:0                 
DEBU[0002] info: active_defrag_key_hits:0               
DEBU[0002] info: active_defrag_key_misses:0             
DEBU[0002] info:                                        
DEBU[0002] info: # Replication                          
DEBU[0002] info: role:master                            
DEBU[0002] info: connected_slaves:1                     
DEBU[0002] info: slave0:ip=X.X.X.X,port=6379,state=online,offset=7682091,lag=0 
DEBU[0002] info: master_replid:01154cc45a16bb9d28efafab2747c8443f35e2b1 
DEBU[0002] info: master_replid2:0000000000000000000000000000000000000000 
DEBU[0002] info: master_repl_offset:7682091             
DEBU[0002] info: second_repl_offset:-1                  
DEBU[0002] info: repl_backlog_active:1                  
DEBU[0002] info: repl_backlog_size:107374182            
DEBU[0002] info: repl_backlog_first_byte_offset:1       
DEBU[0002] info: repl_backlog_histlen:7682091           
DEBU[0002] info:                                        
DEBU[0002] info: # CPU                                  
DEBU[0002] info: used_cpu_sys:3907.59                   
DEBU[0002] info: used_cpu_user:1716.45                  
DEBU[0002] info: used_cpu_sys_children:0.00             
DEBU[0002] info: used_cpu_user_children:0.00            
DEBU[0002] info:                                        
DEBU[0002] info: # Commandstats                         
DEBU[0002] info: cmdstat_hset:calls=8,usec=52,usec_per_call=6.50 
DEBU[0002] info: cmdstat_zadd:calls=1,usec=13,usec_per_call=13.00 
DEBU[0002] info: cmdstat_role:calls=5490679,usec=138077049,usec_per_call=25.15 
DEBU[0002] info: cmdstat_psync:calls=4,usec=103,usec_per_call=25.75 
DEBU[0002] info: cmdstat_latency:calls=2707,usec=19126,usec_per_call=7.07 
DEBU[0002] info: cmdstat_ping:calls=265302,usec=359554,usec_per_call=1.36 
DEBU[0002] info: cmdstat_keys:calls=1,usec=33,usec_per_call=33.00 
DEBU[0002] info: cmdstat_incr:calls=1,usec=20,usec_per_call=20.00 
DEBU[0002] info: cmdstat_publish:calls=1,usec=5,usec_per_call=5.00 
DEBU[0002] info: cmdstat_info:calls=414950,usec=32502774,usec_per_call=78.33 
DEBU[0002] info: cmdstat_slaveof:calls=45760,usec=91374,usec_per_call=2.00 
DEBU[0002] info: cmdstat_sadd:calls=1,usec=4,usec_per_call=4.00 
DEBU[0002] info: cmdstat_subscribe:calls=98,usec=1137,usec_per_call=11.60 
DEBU[0002] info: cmdstat_slowlog:calls=5414,usec=116570,usec_per_call=21.53 
DEBU[0002] info: cmdstat_replconf:calls=2737451,usec=4726811,usec_per_call=1.73 
DEBU[0002] info: cmdstat_config:calls=91538,usec=33926874,usec_per_call=370.63 
DEBU[0002] info: cmdstat_select:calls=307,usec=479,usec_per_call=1.56 
DEBU[0002] info: cmdstat_exec:calls=5,usec=41,usec_per_call=8.20 
DEBU[0002] info: cmdstat_multi:calls=1,usec=0,usec_per_call=0.00 
DEBU[0002] info:                                        
DEBU[0002] info: # Cluster                              
DEBU[0002] info: cluster_enabled:0                      
DEBU[0002] info:                                        
DEBU[0002] info: # Keyspace                             
DEBU[0002] info: db8:keys=7,expires=0,avg_ttl=0         
DEBU[0002] info:                                        
DEBU[0002] info: # oom-prevention                       
DEBU[0002] info: oom_prevention_on:No                   
DEBU[0002] info: current_useconds_with_oom_prevention_on:0 
DEBU[0002] info: total_useconds_with_oom_prevention_on:0 
DEBU[0002] info: oom_prevention_threshold:1256277934    
DEBU[0002] info: oom_prevention_threshold_human:1.17G   
DEBU[0002] info: used_memory_total:13848576             
DEBU[0002] info: used_memory_total_human:13.21M         
DEBU[0002] info: peak_used_memory_total:13885440        
DEBU[0002] info: peak_used_memory_total_human:13.24M    
DEBU[0002] info: used_memory_rss:13848576               
DEBU[0002] info: used_memory_rss_human:13.21M           
DEBU[0002] info: used_memory_aof:0                      
DEBU[0002] info: used_memory_aof_human:0B               
DEBU[0002] info: used_memory_rdb:0                      
DEBU[0002] info: used_memory_rdb_human:0B               
DEBU[0002] info:                                        
DEBU[0002] c.Do() - running command: LATENCY [LATEST]   
DEBU[0002] c.Do() - done                                
DEBU[0002] keys: []main.dbKeyPair(nil)                  
DEBU[0002] e.singleKeys: []main.dbKeyPair(nil)          
DEBU[0002] e.keys: []main.dbKeyPair(nil)                
DEBU[0002] allKeys: []main.dbKeyPair{}                  
DEBU[0002] scrapeRedisHost() done

wget -d http://Y.Y.Y.Y:9121/metrics
DEBUG output created by Wget 1.19.4 on linux-gnu.

Reading HSTS entries from /root/.wget-hsts
URI encoding = ‘UTF-8’
Converted file name 'metrics' (UTF-8) -> 'metrics' (UTF-8)
--2019-10-04 07:34:28--  http://Y.Y.Y.Y:9121/metrics
Connecting to Y.Y.Y.Y:9121... connected.
Created socket 3.
Releasing 0x000055c18014be70 (new refcount 0).
Deleting unused 0x000055c18014be70.

---request begin---
GET /metrics HTTP/1.1
User-Agent: Wget/1.19.4 (linux-gnu)
Accept: */*
Accept-Encoding: identity
Host: Y.Y.Y.Y:9121
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response... 
---response begin---
HTTP/1.1 500 Internal Server Error
Content-Type: text/plain; charset=utf-8
X-Content-Type-Options: nosniff
Date: Fri, 04 Oct 2019 07:34:28 GMT
Content-Length: 183

---response end---
500 Internal Server Error
Registered socket 3 for persistent reuse.
URI content encoding = ‘utf-8’
Skipping 183 bytes of body: [An error has occurred while serving metrics:

collected metric "redis_memory_used_rss_bytes" { gauge:<value:1.3848576e+07 > } was collected before with the same name and label values
] done.
2019-10-04 07:34:28 ERROR 500: Internal Server Error.

oliver006 · 2019-10-05T02:21:11Z

Thanks for the report.
I have a vague idea what's going on but I need more time to dig into this. I suspect the error isn't thrown at the time NewConstMetric() is being called but downstream, when the prometheus library picks up the metrics from the channel.
I'll update here when I have another fix, should be in the next couple of days.

mcinquin · 2019-10-05T08:11:02Z

No problem.
Thank you for your investment.

oliver006 · 2019-10-07T19:22:33Z

I think I can fix this by using promhttp.HandlerOpts{ErrorHandling: promhttp.ContinueOnError} when registering the promhttp HTTP handler.

@Shini31 - I build and pushed up docker image oliver006/redis_exporter:tst-v2, can you give it a try when you have a moment and let me know if it addresses the issue?

mcinquin · 2019-10-08T10:24:48Z

Great news !!

wget -d http://127.0.0.1:9121/metrics
DEBUG output created by Wget 1.19.4 on linux-gnu.

Reading HSTS entries from /root/.wget-hsts
URI encoding = ‘UTF-8’
Converted file name 'metrics' (UTF-8) -> 'metrics' (UTF-8)
--2019-10-08 10:20:31--  http://127.0.0.1:9121/metrics
Connecting to 127.0.0.1:9121... connected.
Created socket 3.
Releasing 0x000055f4413cfec0 (new refcount 0).
Deleting unused 0x000055f4413cfec0.

---request begin---
GET /metrics HTTP/1.1
User-Agent: Wget/1.19.4 (linux-gnu)
Accept: */*
Accept-Encoding: identity
Host: 127.0.0.1:9121
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response... 
---response begin---
HTTP/1.1 200 OK
Content-Type: text/plain; version=0.0.4; charset=utf-8
Date: Tue, 08 Oct 2019 10:20:31 GMT
Transfer-Encoding: chunked

---response end---
200 OK
Registered socket 3 for persistent reuse.
URI content encoding = ‘utf-8’
Length: unspecified [text/plain]
Saving to: ‘metrics.2’

metrics.2                                                                  [ <=>                                                                                                                                                                       ]  12.23K  --.-KB/s    in 0s      

2019-10-08 10:20:31 (81.0 MB/s) - ‘metrics.2’ saved [12523]

time="2019-10-08T10:19:00Z" level=info msg="Redis Metrics Exporter tst-v2    build date: 2019-10-07-19:21:18    sha1: tst-v2    Go: go1.13    GOOS: linux    GOARCH: amd64"
time="2019-10-08T10:19:00Z" level=debug msg="Enabling debug output"
time="2019-10-08T10:19:00Z" level=debug msg="keys: []main.dbKeyPair(nil)"
time="2019-10-08T10:19:00Z" level=debug msg="singleKeys: []main.dbKeyPair(nil)"
time="2019-10-08T10:19:00Z" level=info msg="Providing metrics at :9121/metrics"
time="2019-10-08T10:19:00Z" level=debug msg="Configured redis addr: \"X.X.X.X:6379\""
time="2019-10-08T10:19:04Z" level=debug msg="Trying DialURL(): redis://X.X.X.X:6379"
time="2019-10-08T10:19:04Z" level=debug msg="connected to: X.X.X.X:6379"
time="2019-10-08T10:19:04Z" level=debug msg="c.Do() - running command: CLIENT [SETNAME redis_exporter]"
time="2019-10-08T10:19:04Z" level=debug msg="c.Do() - err: ERR unknown command `CLIENT`, with args beginning with: `SETNAME`, `redis_exporter`, "
time="2019-10-08T10:19:04Z" level=debug msg="c.Do() - done"
time="2019-10-08T10:19:04Z" level=error msg="Couldn't set client name, err: ERR unknown command `CLIENT`, with args beginning with: `SETNAME`, `redis_exporter`, "
time="2019-10-08T10:19:04Z" level=debug msg="Redis CONFIG err: ERR unknown command `CONFIG`, with args beginning with: `GET`, `*`, "
time="2019-10-08T10:19:04Z" level=debug msg="c.Do() - running command: INFO [ALL]"
time="2019-10-08T10:19:04Z" level=debug msg="c.Do() - done"
time="2019-10-08T10:19:04Z" level=debug msg="info: # Server"
time="2019-10-08T10:19:04Z" level=debug msg="info: redis_version:4.0.14"
time="2019-10-08T10:19:04Z" level=debug msg="info: redis_git_sha1:00000000"
time="2019-10-08T10:19:04Z" level=debug msg="info: redis_git_dirty:0"
time="2019-10-08T10:19:04Z" level=debug msg="info: redis_build_id:a9dfc400783db9fd"
time="2019-10-08T10:19:04Z" level=debug msg="info: redis_mode:standalone"
time="2019-10-08T10:19:04Z" level=debug msg="info: os:Linux 4.14.94+ x86_64"
time="2019-10-08T10:19:04Z" level=debug msg="info: arch_bits:64"
time="2019-10-08T10:19:04Z" level=debug msg="info: multiplexing_api:epoll"
time="2019-10-08T10:19:04Z" level=debug msg="info: atomicvar_api:atomic-builtin"
time="2019-10-08T10:19:04Z" level=debug msg="info: gcc_version:6.3.0"
time="2019-10-08T10:19:04Z" level=debug msg="info: process_id:1"
time="2019-10-08T10:19:04Z" level=debug msg="info: run_id:04ffff67345d3201766e73ae398db0c729d5f78d"
time="2019-10-08T10:19:04Z" level=debug msg="info: tcp_port:6379"
time="2019-10-08T10:19:04Z" level=debug msg="info: uptime_in_seconds:3100863"
time="2019-10-08T10:19:04Z" level=debug msg="info: uptime_in_days:35"
time="2019-10-08T10:19:04Z" level=debug msg="info: hz:10"
time="2019-10-08T10:19:04Z" level=debug msg="info: lru_clock:10248856"
time="2019-10-08T10:19:04Z" level=debug msg="info: executable:/data/redis-server"
time="2019-10-08T10:19:04Z" level=debug msg="info: config_file:/usr/local/etc/redis/redis.conf"
time="2019-10-08T10:19:04Z" level=debug msg="info: "
time="2019-10-08T10:19:04Z" level=debug msg="info: # Clients"
time="2019-10-08T10:19:04Z" level=debug msg="info: connected_clients:10"
time="2019-10-08T10:19:04Z" level=debug msg="info: client_longest_output_list:0"
time="2019-10-08T10:19:04Z" level=debug msg="info: client_biggest_input_buf:0"
time="2019-10-08T10:19:04Z" level=debug msg="info: blocked_clients:0"
time="2019-10-08T10:19:04Z" level=debug msg="info: "
time="2019-10-08T10:19:04Z" level=debug msg="info: # Memory"
time="2019-10-08T10:19:04Z" level=debug msg="info: used_memory:121398976"
time="2019-10-08T10:19:04Z" level=debug msg="info: used_memory_human:115.78M"
time="2019-10-08T10:19:04Z" level=debug msg="info: used_memory_rss:14000128"
time="2019-10-08T10:19:04Z" level=debug msg="info: used_memory_rss_human:13.35M"
time="2019-10-08T10:19:04Z" level=debug msg="info: used_memory_peak:121667240"
time="2019-10-08T10:19:04Z" level=debug msg="info: used_memory_peak_human:116.03M"
time="2019-10-08T10:19:04Z" level=debug msg="info: used_memory_peak_perc:99.78%"
time="2019-10-08T10:19:04Z" level=debug msg="info: used_memory_overhead:118445834"
time="2019-10-08T10:19:04Z" level=debug msg="info: used_memory_startup:786680"
time="2019-10-08T10:19:04Z" level=debug msg="info: used_memory_dataset:2953142"
time="2019-10-08T10:19:04Z" level=debug msg="info: used_memory_dataset_perc:2.45%"
time="2019-10-08T10:19:04Z" level=debug msg="info: total_system_memory:3153059840"
time="2019-10-08T10:19:04Z" level=debug msg="info: total_system_memory_human:2.94G"
time="2019-10-08T10:19:04Z" level=debug msg="info: used_memory_lua:37888"
time="2019-10-08T10:19:04Z" level=debug msg="info: used_memory_lua_human:37.00K"
time="2019-10-08T10:19:04Z" level=debug msg="info: maxmemory:1073741824"
time="2019-10-08T10:19:04Z" level=debug msg="info: maxmemory_human:1.00G"
time="2019-10-08T10:19:04Z" level=debug msg="info: maxmemory_policy:volatile-lru"
time="2019-10-08T10:19:04Z" level=debug msg="info: mem_fragmentation_ratio:0.12"
time="2019-10-08T10:19:04Z" level=debug msg="info: mem_allocator:jemalloc-4.0.3"
time="2019-10-08T10:19:04Z" level=debug msg="info: active_defrag_running:0"
time="2019-10-08T10:19:04Z" level=debug msg="info: lazyfree_pending_objects:0"
time="2019-10-08T10:19:04Z" level=debug msg="info: "
time="2019-10-08T10:19:04Z" level=debug msg="info: # Persistence"
time="2019-10-08T10:19:04Z" level=debug msg="info: loading:0"
time="2019-10-08T10:19:04Z" level=debug msg="info: rdb_changes_since_last_save:16"
time="2019-10-08T10:19:04Z" level=debug msg="info: rdb_bgsave_in_progress:0"
time="2019-10-08T10:19:04Z" level=debug msg="info: rdb_last_save_time:1567429081"
time="2019-10-08T10:19:04Z" level=debug msg="info: rdb_last_bgsave_status:ok"
time="2019-10-08T10:19:04Z" level=debug msg="info: rdb_last_bgsave_time_sec:-1"
time="2019-10-08T10:19:04Z" level=debug msg="info: rdb_current_bgsave_time_sec:-1"
time="2019-10-08T10:19:04Z" level=debug msg="info: rdb_last_cow_size:258048"
time="2019-10-08T10:19:04Z" level=debug msg="info: aof_enabled:0"
time="2019-10-08T10:19:04Z" level=debug msg="info: aof_rewrite_in_progress:0"
time="2019-10-08T10:19:04Z" level=debug msg="info: aof_rewrite_scheduled:0"
time="2019-10-08T10:19:04Z" level=debug msg="info: aof_last_rewrite_time_sec:-1"
time="2019-10-08T10:19:04Z" level=debug msg="info: aof_current_rewrite_time_sec:-1"
time="2019-10-08T10:19:04Z" level=debug msg="info: aof_last_bgrewrite_status:ok"
time="2019-10-08T10:19:04Z" level=debug msg="info: aof_last_write_status:ok"
time="2019-10-08T10:19:04Z" level=debug msg="info: aof_last_cow_size:0"
time="2019-10-08T10:19:04Z" level=debug msg="info: "
time="2019-10-08T10:19:04Z" level=debug msg="info: # Stats"
time="2019-10-08T10:19:04Z" level=debug msg="info: total_connections_received:106440"
time="2019-10-08T10:19:04Z" level=debug msg="info: total_commands_processed:10205115"
time="2019-10-08T10:19:04Z" level=debug msg="info: instantaneous_ops_per_sec:6"
time="2019-10-08T10:19:04Z" level=debug msg="info: total_net_input_bytes:239926856"
time="2019-10-08T10:19:04Z" level=debug msg="info: total_net_output_bytes:1224461433"
time="2019-10-08T10:19:04Z" level=debug msg="info: instantaneous_input_kbps:0.28"
time="2019-10-08T10:19:04Z" level=debug msg="info: instantaneous_output_kbps:2.68"
time="2019-10-08T10:19:04Z" level=debug msg="info: rejected_connections:0"
time="2019-10-08T10:19:04Z" level=debug msg="info: sync_full:1"
time="2019-10-08T10:19:04Z" level=debug msg="info: sync_partial_ok:0"
time="2019-10-08T10:19:04Z" level=debug msg="info: sync_partial_err:0"
time="2019-10-08T10:19:04Z" level=debug msg="info: expired_keys:0"
time="2019-10-08T10:19:04Z" level=debug msg="info: expired_stale_perc:0.00"
time="2019-10-08T10:19:04Z" level=debug msg="info: expired_time_cap_reached_count:0"
time="2019-10-08T10:19:04Z" level=debug msg="info: evicted_keys:0"
time="2019-10-08T10:19:04Z" level=debug msg="info: keyspace_hits:0"
time="2019-10-08T10:19:04Z" level=debug msg="info: keyspace_misses:0"
time="2019-10-08T10:19:04Z" level=debug msg="info: pubsub_channels:1"
time="2019-10-08T10:19:04Z" level=debug msg="info: pubsub_patterns:0"
time="2019-10-08T10:19:04Z" level=debug msg="info: latest_fork_usec:271"
time="2019-10-08T10:19:04Z" level=debug msg="info: migrate_cached_sockets:0"
time="2019-10-08T10:19:04Z" level=debug msg="info: slave_expires_tracked_keys:0"
time="2019-10-08T10:19:04Z" level=debug msg="info: active_defrag_hits:0"
time="2019-10-08T10:19:04Z" level=debug msg="info: active_defrag_misses:0"
time="2019-10-08T10:19:04Z" level=debug msg="info: active_defrag_key_hits:0"
time="2019-10-08T10:19:04Z" level=debug msg="info: active_defrag_key_misses:0"
time="2019-10-08T10:19:04Z" level=debug msg="info: "
time="2019-10-08T10:19:04Z" level=debug msg="info: # Replication"
time="2019-10-08T10:19:04Z" level=debug msg="info: role:master"
time="2019-10-08T10:19:04Z" level=debug msg="info: connected_slaves:1"
time="2019-10-08T10:19:04Z" level=debug msg="info: slave0:ip=X.X.X.X,port=6379,state=online,offset=8674537,lag=1"
time="2019-10-08T10:19:04Z" level=debug msg="info: master_replid:01154cc45a16bb9d28efafab2747c8443f35e2b1"
time="2019-10-08T10:19:04Z" level=debug msg="info: master_replid2:0000000000000000000000000000000000000000"
time="2019-10-08T10:19:04Z" level=debug msg="info: master_repl_offset:8674537"
time="2019-10-08T10:19:04Z" level=debug msg="info: second_repl_offset:-1"
time="2019-10-08T10:19:04Z" level=debug msg="info: repl_backlog_active:1"
time="2019-10-08T10:19:04Z" level=debug msg="info: repl_backlog_size:107374182"
time="2019-10-08T10:19:04Z" level=debug msg="info: repl_backlog_first_byte_offset:1"
time="2019-10-08T10:19:04Z" level=debug msg="info: repl_backlog_histlen:8674537"
time="2019-10-08T10:19:04Z" level=debug msg="info: "
time="2019-10-08T10:19:04Z" level=debug msg="info: # CPU"
time="2019-10-08T10:19:04Z" level=debug msg="info: used_cpu_sys:4482.81"
time="2019-10-08T10:19:04Z" level=debug msg="info: used_cpu_user:1981.51"
time="2019-10-08T10:19:04Z" level=debug msg="info: used_cpu_sys_children:0.00"
time="2019-10-08T10:19:04Z" level=debug msg="info: used_cpu_user_children:0.00"
time="2019-10-08T10:19:04Z" level=debug msg="info: "
time="2019-10-08T10:19:04Z" level=debug msg="info: # Commandstats"
time="2019-10-08T10:19:04Z" level=debug msg="info: cmdstat_hset:calls=8,usec=52,usec_per_call=6.50"
time="2019-10-08T10:19:04Z" level=debug msg="info: cmdstat_zadd:calls=1,usec=13,usec_per_call=13.00"
time="2019-10-08T10:19:04Z" level=debug msg="info: cmdstat_role:calls=6201614,usec=159280849,usec_per_call=25.68"
time="2019-10-08T10:19:04Z" level=debug msg="info: cmdstat_psync:calls=4,usec=103,usec_per_call=25.75"
time="2019-10-08T10:19:04Z" level=debug msg="info: cmdstat_latency:calls=2710,usec=19137,usec_per_call=7.06"
time="2019-10-08T10:19:04Z" level=debug msg="info: cmdstat_ping:calls=279610,usec=378219,usec_per_call=1.35"
time="2019-10-08T10:19:04Z" level=debug msg="info: cmdstat_keys:calls=1,usec=33,usec_per_call=33.00"
time="2019-10-08T10:19:04Z" level=debug msg="info: cmdstat_incr:calls=1,usec=20,usec_per_call=20.00"
time="2019-10-08T10:19:04Z" level=debug msg="info: cmdstat_publish:calls=1,usec=5,usec_per_call=5.00"
time="2019-10-08T10:19:04Z" level=debug msg="info: cmdstat_info:calls=468304,usec=37116434,usec_per_call=79.26"
time="2019-10-08T10:19:04Z" level=debug msg="info: cmdstat_slaveof:calls=51685,usec=102442,usec_per_call=1.98"
time="2019-10-08T10:19:04Z" level=debug msg="info: cmdstat_sadd:calls=1,usec=4,usec_per_call=4.00"
time="2019-10-08T10:19:04Z" level=debug msg="info: cmdstat_subscribe:calls=107,usec=1225,usec_per_call=11.45"
time="2019-10-08T10:19:04Z" level=debug msg="info: cmdstat_slowlog:calls=5420,usec=116630,usec_per_call=21.52"
time="2019-10-08T10:19:04Z" level=debug msg="info: cmdstat_replconf:calls=3091920,usec=5430438,usec_per_call=1.76"
time="2019-10-08T10:19:04Z" level=debug msg="info: cmdstat_config:calls=103388,usec=38486323,usec_per_call=372.25"
time="2019-10-08T10:19:04Z" level=debug msg="info: cmdstat_select:calls=334,usec=514,usec_per_call=1.54"
time="2019-10-08T10:19:04Z" level=debug msg="info: cmdstat_exec:calls=5,usec=41,usec_per_call=8.20"
time="2019-10-08T10:19:04Z" level=debug msg="info: cmdstat_multi:calls=1,usec=0,usec_per_call=0.00"
time="2019-10-08T10:19:04Z" level=debug msg="info: "
time="2019-10-08T10:19:04Z" level=debug msg="info: # Cluster"
time="2019-10-08T10:19:04Z" level=debug msg="info: cluster_enabled:0"
time="2019-10-08T10:19:04Z" level=debug msg="info: "
time="2019-10-08T10:19:04Z" level=debug msg="info: # Keyspace"
time="2019-10-08T10:19:04Z" level=debug msg="info: db8:keys=7,expires=0,avg_ttl=0"
time="2019-10-08T10:19:04Z" level=debug msg="info: "
time="2019-10-08T10:19:04Z" level=debug msg="info: # oom-prevention"
time="2019-10-08T10:19:04Z" level=debug msg="info: oom_prevention_on:No"
time="2019-10-08T10:19:04Z" level=debug msg="info: current_useconds_with_oom_prevention_on:0"
time="2019-10-08T10:19:04Z" level=debug msg="info: total_useconds_with_oom_prevention_on:0"
time="2019-10-08T10:19:04Z" level=debug msg="info: oom_prevention_threshold:1256277934"
time="2019-10-08T10:19:04Z" level=debug msg="info: oom_prevention_threshold_human:1.17G"
time="2019-10-08T10:19:04Z" level=debug msg="info: used_memory_total:14000128"
time="2019-10-08T10:19:04Z" level=debug msg="info: used_memory_total_human:13.35M"
time="2019-10-08T10:19:04Z" level=debug msg="info: peak_used_memory_total:14303232"
time="2019-10-08T10:19:04Z" level=debug msg="info: peak_used_memory_total_human:13.64M"
time="2019-10-08T10:19:04Z" level=debug msg="info: used_memory_rss:14000128"
time="2019-10-08T10:19:04Z" level=debug msg="info: used_memory_rss_human:13.35M"
time="2019-10-08T10:19:04Z" level=debug msg="info: used_memory_aof:0"
time="2019-10-08T10:19:04Z" level=debug msg="info: used_memory_aof_human:0B"
time="2019-10-08T10:19:04Z" level=debug msg="info: used_memory_rdb:0"
time="2019-10-08T10:19:04Z" level=debug msg="info: used_memory_rdb_human:0B"
time="2019-10-08T10:19:04Z" level=debug msg="info: "
time="2019-10-08T10:19:04Z" level=debug msg="c.Do() - running command: LATENCY [LATEST]"
time="2019-10-08T10:19:04Z" level=debug msg="c.Do() - done"
time="2019-10-08T10:19:04Z" level=debug msg="keys: []main.dbKeyPair(nil)"
time="2019-10-08T10:19:04Z" level=debug msg="e.singleKeys: []main.dbKeyPair(nil)"
time="2019-10-08T10:19:04Z" level=debug msg="e.keys: []main.dbKeyPair(nil)"
time="2019-10-08T10:19:04Z" level=debug msg="allKeys: []main.dbKeyPair{}"
time="2019-10-08T10:19:04Z" level=debug msg="scrapeRedisHost() done"

oliver006 · 2019-10-08T14:14:35Z

Ok, thanks for your help, that's great. Let me get the fix in and cut a new release.

oliver006 · 2019-10-08T14:31:09Z

Released as https://github.com/oliver006/redis_exporter/releases/tag/v1.3.1

mcinquin · 2019-10-08T20:11:23Z

Thank you for your helpfulness and determination.

apurvabhandari · 2020-07-13T12:11:57Z

Facing same issue still in v1.9.0 CentOS 7 for Redis 4.0

This will exit after sometime, maybe due to the ERROR in output.
I have tried with Redis 3.2 it works well.

[root@prometheus redis_exporter-v1.9.0.linux-amd64]# ./redis_exporter --redis-only-metrics --include-system-metrics --redis.addr=redis://10.x.x.x:6379 INFO[0000] Redis Metrics Exporter v1.9.0 build date: 2020-07-07-02:25:01 sha1: 3e15a27ef8657e77100a55f10895e1a4fee70f79 Go: go1.14.4 GOOS: linux GOARCH: amd64 INFO[0000] Providing metrics at :9121/metrics ERRO[0009] Couldn't set client name, err: ERR unknown command CLIENT, with args beginning with: SETNAME, redis_exporter, ERRO[0039] Couldn't set client name, err: ERR unknown command CLIENT, with args beginning with: SETNAME, redis_exporter, ERRO[0054] Couldn't set client name, err: ERR unknown command CLIENT, with args beginning with: SETNAME, redis_exporter,

oliver006 · 2020-07-13T13:49:54Z

Did you try using the --set-client-name=false command line flag?

apurvabhandari · 2020-07-13T14:13:04Z

Thanks. This is the fix. It's working!

mcinquin added the bug label Sep 26, 2019

mcinquin assigned oliver006 Sep 26, 2019

mcinquin changed the title ~~Error 500 on /metrics page with Google Memorystore~~ Error 500 on /metrics page with Google Cloud Memorystore Sep 26, 2019

oliver006 mentioned this issue Oct 2, 2019

don't panic on error for NewConstMetric(), catch it and continue #318

Merged

oliver006 closed this as completed in #318 Oct 3, 2019

oliver006 reopened this Oct 5, 2019

oliver006 mentioned this issue Oct 8, 2019

use promhttp.ContinueOnError to not fail on duplicate metrics #321

Merged

oliver006 closed this as completed in #321 Oct 8, 2019

ChapChap mentioned this issue Apr 7, 2022

[prometheus-redis-exporter] memorystore set-client-name extra args helper prometheus-community/helm-charts#1958

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error 500 on /metrics page with Google Cloud Memorystore #315

Error 500 on /metrics page with Google Cloud Memorystore #315

mcinquin commented Sep 26, 2019 •

edited

oliver006 commented Sep 26, 2019

mcinquin commented Sep 26, 2019

oliver006 commented Sep 26, 2019

mcinquin commented Sep 26, 2019

oliver006 commented Sep 27, 2019 •

edited

oliver006 commented Sep 27, 2019

mcinquin commented Sep 27, 2019

oliver006 commented Sep 27, 2019

mcinquin commented Sep 27, 2019

oliver006 commented Oct 2, 2019

adrilo commented Oct 3, 2019

oliver006 commented Oct 3, 2019

oliver006 commented Oct 3, 2019

mcinquin commented Oct 4, 2019

oliver006 commented Oct 5, 2019 •

edited

mcinquin commented Oct 5, 2019

oliver006 commented Oct 7, 2019

mcinquin commented Oct 8, 2019

oliver006 commented Oct 8, 2019

oliver006 commented Oct 8, 2019

mcinquin commented Oct 8, 2019

apurvabhandari commented Jul 13, 2020

oliver006 commented Jul 13, 2020

apurvabhandari commented Jul 13, 2020

Error 500 on /metrics page with Google Cloud Memorystore #315

Error 500 on /metrics page with Google Cloud Memorystore #315

Comments

mcinquin commented Sep 26, 2019 • edited

oliver006 commented Sep 26, 2019

mcinquin commented Sep 26, 2019

oliver006 commented Sep 26, 2019

mcinquin commented Sep 26, 2019

oliver006 commented Sep 27, 2019 • edited

oliver006 commented Sep 27, 2019

mcinquin commented Sep 27, 2019

oliver006 commented Sep 27, 2019

mcinquin commented Sep 27, 2019

oliver006 commented Oct 2, 2019

adrilo commented Oct 3, 2019

oliver006 commented Oct 3, 2019

oliver006 commented Oct 3, 2019

mcinquin commented Oct 4, 2019

oliver006 commented Oct 5, 2019 • edited

mcinquin commented Oct 5, 2019

oliver006 commented Oct 7, 2019

mcinquin commented Oct 8, 2019

oliver006 commented Oct 8, 2019

oliver006 commented Oct 8, 2019

mcinquin commented Oct 8, 2019

apurvabhandari commented Jul 13, 2020

oliver006 commented Jul 13, 2020

apurvabhandari commented Jul 13, 2020

mcinquin commented Sep 26, 2019 •

edited

oliver006 commented Sep 27, 2019 •

edited

oliver006 commented Oct 5, 2019 •

edited