can't add a new key into hash during iteration #1804

jl2005 · 2017-12-27T09:55:26Z

fluentd or td-agent version.

fluentd-0.12.42
Environment information, e.g. OS.

Kubernetes DaemonSet
Your configuration

   <source>
      @type tail
      tag kubernetes.*
      path /var/log/containers/*.log
      pos_file /var/log/fluentd-containers.log.pos
      format json
      time_key not-parse-time
      read_from_head true
    </source>

Your problem explanation. If you have an error logs, write it together.

2017-12-26 07:41:05 +0000 [error]: can't add a new key into hash during iteration
  2017-12-26 07:41:05 +0000 [error]: /var/lib/gems/2.3.0/gems/fluentd-0.12.42/lib/fluent/plugin/in_tail.rb:247:in `block in start_watchers'
  2017-12-26 07:41:05 +0000 [error]: /var/lib/gems/2.3.0/gems/fluentd-0.12.42/lib/fluent/plugin/in_tail.rb:228:in `each'
  2017-12-26 07:41:05 +0000 [error]: /var/lib/gems/2.3.0/gems/fluentd-0.12.42/lib/fluent/plugin/in_tail.rb:228:in `start_watchers'
  2017-12-26 07:41:05 +0000 [error]: /var/lib/gems/2.3.0/gems/fluentd-0.12.42/lib/fluent/plugin/in_tail.rb:214:in `refresh_watchers'
  2017-12-26 07:41:05 +0000 [error]: /var/lib/gems/2.3.0/gems/fluentd-0.12.42/lib/fluent/plugin/in_tail.rb:577:in `on_timer'
  2017-12-26 07:41:05 +0000 [error]: /var/lib/gems/2.3.0/gems/cool.io-1.5.3/lib/cool.io/loop.rb:88:in `run_once'
  2017-12-26 07:41:05 +0000 [error]: /var/lib/gems/2.3.0/gems/cool.io-1.5.3/lib/cool.io/loop.rb:88:in `run'
  2017-12-26 07:41:05 +0000 [error]: /var/lib/gems/2.3.0/gems/fluentd-0.12.42/lib/fluent/plugin/in_tail.rb:316:in `run'
2017-12-26 11:14:05 +0000 [error]: can't add a new key into hash during iteration
  2017-12-26 11:14:05 +0000 [error]: suppressed same stacktrace

The text was updated successfully, but these errors were encountered:

repeatedly · 2017-12-29T01:07:11Z

Does this happen during shutdown?

jl2005 · 2017-12-31T04:08:21Z

Does this happen during shutdown?

@repeatedly No, although there is this error log, this container is still running.

repeatedly · 2018-01-09T03:46:24Z

That's weird. in_tail plugin works on one thread.
So this error should not happen. Hmm...
btw, we can ignore this error because next refresh find new file again.
Will add the code.

jl2005 · 2018-02-06T05:47:14Z

hi, @repeatedly When I print the fluentd stack trace, I found two in_tail threads. Will this be the case if there will be more than one thread.

/tmp/sigdump-9.log

Sigdump at 2018-02-06 03:45:45 +0000 process 9 (/usr/local/bin/fluentd)
  Thread #<Thread:0x007f542a17e1b8> status=run priority=0
      /var/lib/gems/2.3.0/gems/sigdump-0.2.4/lib/sigdump.rb:52:in `backtrace'
      /var/lib/gems/2.3.0/gems/sigdump-0.2.4/lib/sigdump.rb:52:in `dump_backtrace'
      /var/lib/gems/2.3.0/gems/sigdump-0.2.4/lib/sigdump.rb:34:in `block in dump_all_thread_backtrace'
      /var/lib/gems/2.3.0/gems/sigdump-0.2.4/lib/sigdump.rb:33:in `each'
      /var/lib/gems/2.3.0/gems/sigdump-0.2.4/lib/sigdump.rb:33:in `dump_all_thread_backtrace'
      /var/lib/gems/2.3.0/gems/sigdump-0.2.4/lib/sigdump.rb:16:in `block in dump'
      /var/lib/gems/2.3.0/gems/sigdump-0.2.4/lib/sigdump.rb:136:in `open'
      /var/lib/gems/2.3.0/gems/sigdump-0.2.4/lib/sigdump.rb:136:in `_open_dump_path'
      /var/lib/gems/2.3.0/gems/sigdump-0.2.4/lib/sigdump.rb:14:in `dump'
      /var/lib/gems/2.3.0/gems/sigdump-0.2.4/lib/sigdump.rb:7:in `block in setup'
      /var/lib/gems/2.3.0/gems/cool.io-1.5.3/lib/cool.io/loop.rb:88:in `run_once'
      /var/lib/gems/2.3.0/gems/cool.io-1.5.3/lib/cool.io/loop.rb:88:in `run'
      /var/lib/gems/2.3.0/gems/fluentd-0.12.42/lib/fluent/engine.rb:199:in `run'
      /var/lib/gems/2.3.0/gems/fluentd-0.12.42/lib/fluent/supervisor.rb:570:in `run_engine'
      /var/lib/gems/2.3.0/gems/fluentd-0.12.42/lib/fluent/supervisor.rb:176:in `block in start'
      /var/lib/gems/2.3.0/gems/fluentd-0.12.42/lib/fluent/supervisor.rb:366:in `main_process'
      /var/lib/gems/2.3.0/gems/fluentd-0.12.42/lib/fluent/supervisor.rb:170:in `start'
      /var/lib/gems/2.3.0/gems/fluentd-0.12.42/lib/fluent/command/fluentd.rb:173:in `<top (required)>'
      /usr/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
      /usr/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
      /var/lib/gems/2.3.0/gems/fluentd-0.12.42/bin/fluentd:8:in `<top (required)>'
      /usr/local/bin/fluentd:22:in `load'
      /usr/local/bin/fluentd:22:in `<main>'
  Thread #<Thread:0x007f54227f3fa0> status=sleep priority=0
      /var/lib/gems/2.3.0/gems/fluent-plugin-detect-exceptions-0.0.9/lib/fluent/plugin/out_detect_exceptions.rb:115:in `sleep'
      /var/lib/gems/2.3.0/gems/fluent-plugin-detect-exceptions-0.0.9/lib/fluent/plugin/out_detect_exceptions.rb:115:in `block (2 levels) in check_flush_loop'
      /var/lib/gems/2.3.0/gems/fluent-plugin-detect-exceptions-0.0.9/lib/fluent/plugin/out_detect_exceptions.rb:114:in `loop'
      /var/lib/gems/2.3.0/gems/fluent-plugin-detect-exceptions-0.0.9/lib/fluent/plugin/out_detect_exceptions.rb:114:in `block in check_flush_loop'
      /var/lib/gems/2.3.0/gems/fluent-plugin-detect-exceptions-0.0.9/lib/fluent/plugin/out_detect_exceptions.rb:113:in `synchronize'
      /var/lib/gems/2.3.0/gems/fluent-plugin-detect-exceptions-0.0.9/lib/fluent/plugin/out_detect_exceptions.rb:113:in `check_flush_loop'
  Thread #<Thread:0x007f54228865a8> status=sleep priority=0
      /var/lib/gems/2.3.0/gems/fluentd-0.12.42/lib/fluent/output.rb:174:in `sleep'
      /var/lib/gems/2.3.0/gems/fluentd-0.12.42/lib/fluent/output.rb:174:in `wait'
      /var/lib/gems/2.3.0/gems/fluentd-0.12.42/lib/fluent/output.rb:174:in `cond_wait'
      /var/lib/gems/2.3.0/gems/fluentd-0.12.42/lib/fluent/output.rb:158:in `run'
  Thread #<Thread:0x007f54228863c8> status=sleep priority=0
      /var/lib/gems/2.3.0/gems/fluentd-0.12.42/lib/fluent/output.rb:174:in `sleep'
      /var/lib/gems/2.3.0/gems/fluentd-0.12.42/lib/fluent/output.rb:174:in `wait'
      /var/lib/gems/2.3.0/gems/fluentd-0.12.42/lib/fluent/output.rb:174:in `cond_wait'
      /var/lib/gems/2.3.0/gems/fluentd-0.12.42/lib/fluent/output.rb:158:in `run'
  Thread #<Thread:0x007f54228861e8> status=sleep priority=0
      /var/lib/gems/2.3.0/gems/fluentd-0.12.42/lib/fluent/output.rb:174:in `sleep'
      /var/lib/gems/2.3.0/gems/fluentd-0.12.42/lib/fluent/output.rb:174:in `wait'
      /var/lib/gems/2.3.0/gems/fluentd-0.12.42/lib/fluent/output.rb:174:in `cond_wait'
      /var/lib/gems/2.3.0/gems/fluentd-0.12.42/lib/fluent/output.rb:158:in `run'
  Thread #<Thread:0x007f5422886008> status=sleep priority=0
      /var/lib/gems/2.3.0/gems/fluentd-0.12.42/lib/fluent/output.rb:174:in `sleep'
      /var/lib/gems/2.3.0/gems/fluentd-0.12.42/lib/fluent/output.rb:174:in `wait'
      /var/lib/gems/2.3.0/gems/fluentd-0.12.42/lib/fluent/output.rb:174:in `cond_wait'
      /var/lib/gems/2.3.0/gems/fluentd-0.12.42/lib/fluent/output.rb:158:in `run'
  Thread #<Thread:0x007f5422b0e988> status=sleep priority=0
      /var/lib/gems/2.3.0/gems/cool.io-1.5.3/lib/cool.io/loop.rb:88:in `run_once'
      /var/lib/gems/2.3.0/gems/cool.io-1.5.3/lib/cool.io/loop.rb:88:in `run'
      /var/lib/gems/2.3.0/gems/fluentd-0.12.42/lib/fluent/plugin/in_tail.rb:316:in `run'
  Thread #<Thread:0x007f5423fd4f20> status=sleep priority=0
      /var/lib/gems/2.3.0/gems/cool.io-1.5.3/lib/cool.io/loop.rb:88:in `run_once'
      /var/lib/gems/2.3.0/gems/cool.io-1.5.3/lib/cool.io/loop.rb:88:in `run'
      /var/lib/gems/2.3.0/gems/fluentd-0.12.42/lib/fluent/plugin/in_tail.rb:316:in `run'
  Thread #<Thread:0x007f5423fd4d68> status=sleep priority=0
      /var/lib/gems/2.3.0/gems/fluent-plugin-systemd-0.0.9/lib/fluent/plugin/systemd/pos_writer.rb:43:in `sleep'
      /var/lib/gems/2.3.0/gems/fluent-plugin-systemd-0.0.9/lib/fluent/plugin/systemd/pos_writer.rb:43:in `work'
  Thread #<Thread:0x007f5423fd4ae8> status=sleep priority=0
      /var/lib/gems/2.3.0/gems/fluent-plugin-systemd-0.0.9/lib/fluent/plugin/in_systemd.rb:97:in `sleep'
      /var/lib/gems/2.3.0/gems/fluent-plugin-systemd-0.0.9/lib/fluent/plugin/in_systemd.rb:97:in `watch'
      /var/lib/gems/2.3.0/gems/fluent-plugin-systemd-0.0.9/lib/fluent/plugin/in_systemd.rb:70:in `run'
  Thread #<Thread:0x007f5423fc9378> status=sleep priority=0
      /usr/lib/ruby/2.3.0/webrick/server.rb:177:in `select'
      /usr/lib/ruby/2.3.0/webrick/server.rb:177:in `block in start'
      /usr/lib/ruby/2.3.0/webrick/server.rb:33:in `start'
      /usr/lib/ruby/2.3.0/webrick/server.rb:164:in `start'
      /var/lib/gems/2.3.0/gems/fluentd-0.12.42/lib/fluent/plugin/in_monitor_agent.rb:265:in `block in start'
  Thread #<Thread:0x007f5423fbf3a0> status=sleep priority=0
      /usr/lib/ruby/2.3.0/webrick/server.rb:177:in `select'
      /usr/lib/ruby/2.3.0/webrick/server.rb:177:in `block in start'
      /usr/lib/ruby/2.3.0/webrick/server.rb:33:in `start'
      /usr/lib/ruby/2.3.0/webrick/server.rb:164:in `start'
      /var/lib/gems/2.3.0/gems/fluent-plugin-prometheus-0.3.0/lib/fluent/plugin/in_prometheus.rb:33:in `block in start'
  Thread #<Thread:0x007f5423fbee78> status=sleep priority=0
      /var/lib/gems/2.3.0/gems/cool.io-1.5.3/lib/cool.io/loop.rb:88:in `run_once'
      /var/lib/gems/2.3.0/gems/cool.io-1.5.3/lib/cool.io/loop.rb:88:in `run'
      /var/lib/gems/2.3.0/gems/fluent-plugin-prometheus-0.3.0/lib/fluent/plugin/in_prometheus_monitor.rb:82:in `run'
  Thread #<Thread:0x007f5423fbeb08> status=sleep priority=0
      /var/lib/gems/2.3.0/gems/cool.io-1.5.3/lib/cool.io/loop.rb:88:in `run_once'
      /var/lib/gems/2.3.0/gems/cool.io-1.5.3/lib/cool.io/loop.rb:88:in `run'
      /var/lib/gems/2.3.0/gems/fluent-plugin-prometheus-0.3.0/lib/fluent/plugin/in_prometheus_output_monitor.rb:109:in `run'
  Thread #<Thread:0x007f5423fbe590> status=sleep priority=0
      /var/lib/gems/2.3.0/gems/cool.io-1.5.3/lib/cool.io/loop.rb:88:in `run_once'
      /var/lib/gems/2.3.0/gems/cool.io-1.5.3/lib/cool.io/loop.rb:88:in `run'
      /var/lib/gems/2.3.0/gems/fluent-plugin-prometheus-0.3.0/lib/fluent/plugin/in_prometheus_tail_monitor.rb:79:in `run'
  Thread #<Thread:0x007f5423fbe220> status=sleep priority=0
      /var/lib/gems/2.3.0/gems/fluentd-0.12.42/lib/fluent/engine.rb:167:in `sleep'
      /var/lib/gems/2.3.0/gems/fluentd-0.12.42/lib/fluent/engine.rb:167:in `log_event_loop'
  Thread #<WEBrick::Utils::TimeoutHandler::Thread:0x007f54246f7190> status=sleep priority=0
      /usr/lib/ruby/2.3.0/webrick/utils.rb:177:in `pop'
      /usr/lib/ruby/2.3.0/webrick/utils.rb:177:in `block in initialize'
  GC stat:
      count: 334
      heap_allocated_pages: 760
      heap_sorted_length: 761
      heap_allocatable_pages: 0
      heap_available_slots: 309764
      heap_live_slots: 308816
      heap_free_slots: 948
      heap_final_slots: 0
      heap_marked_slots: 172388
      heap_swept_slots: 75749
      heap_eden_pages: 760
      heap_tomb_pages: 0
      total_allocated_pages: 760
      total_freed_pages: 0
      total_allocated_objects: 45082857
      total_freed_objects: 44774041
      malloc_increase_bytes: 29792
      malloc_increase_bytes_limit: 16777216
      minor_gc_count: 296
      major_gc_count: 38
      remembered_wb_unprotected_objects: 5343
      remembered_wb_unprotected_objects_limit: 6834
      old_objects: 164461
      old_objects_limit: 268088
      oldmalloc_increase_bytes: 1166080
      oldmalloc_increase_bytes_limit: 16777216
  Built-in objects:
   309,764: TOTAL
   144,898: T_STRING
    44,480: T_ARRAY
    35,995: T_IMEMO
    25,820: T_DATA
    21,803: T_OBJECT
    17,671: T_HASH
     7,699: T_NODE
     3,817: T_CLASS
     2,633: T_STRUCT
     1,152: FREE
       974: T_REGEXP
       696: T_MATCH
       563: T_BIGNUM
       450: T_FILE
       371: T_SYMBOL
       263: T_MODULE
       262: T_ICLASS
       183: T_RATIONAL
        33: T_FLOAT
         1: T_COMPLEX

repeatedly · 2018-02-07T15:49:26Z

If your configuration has only one tail, it seems the problem.
Does this two threads happen in all servers?

jl2005 · 2018-02-08T02:33:52Z

Sorry, because there are two tail in my configuration, there are two threads here.
I'm sorry!

jbrownsc · 2018-03-13T20:18:04Z

Hi @repeatedly I am seeing this issue as well with fluentd 1.1.1, but I see it on startup, every time. Things work for the initial set of containers and logs, but then fluentd's timer is secretly not running and NO refresh occurs (I think?). I believe the source of my problem is a non thread-safe clause in https://github.com/fluent/fluent-plugin-prometheus/blob/master/lib/fluent/plugin/in_prometheus_tail_monitor.rb#L72 .

lruslan · 2018-03-29T14:11:44Z

hi @repeatedly we also experience this issue with fluentd-1.1.2

2018-03-29 14:01:11 +0000 [error]: #0 unexpected error error_class=RuntimeError error="can't add a new key into hash during iteration"
  2018-03-29 14:01:11 +0000 [error]: #0 /var/lib/gems/2.3.0/gems/fluentd-1.1.2/lib/fluent/plugin/in_tail.rb:297:in `block in start_watchers'
  2018-03-29 14:01:11 +0000 [error]: #0 /var/lib/gems/2.3.0/gems/fluentd-1.1.2/lib/fluent/plugin/in_tail.rb:278:in `each'
  2018-03-29 14:01:11 +0000 [error]: #0 /var/lib/gems/2.3.0/gems/fluentd-1.1.2/lib/fluent/plugin/in_tail.rb:278:in `start_watchers'
  2018-03-29 14:01:11 +0000 [error]: #0 /var/lib/gems/2.3.0/gems/fluentd-1.1.2/lib/fluent/plugin/in_tail.rb:258:in `refresh_watchers'
  2018-03-29 14:01:11 +0000 [error]: #0 /var/lib/gems/2.3.0/gems/fluentd-1.1.2/lib/fluent/plugin/in_tail.rb:193:in `start'
  2018-03-29 14:01:11 +0000 [error]: #0 /var/lib/gems/2.3.0/gems/fluentd-1.1.2/lib/fluent/root_agent.rb:165:in `block in start'
  2018-03-29 14:01:11 +0000 [error]: #0 /var/lib/gems/2.3.0/gems/fluentd-1.1.2/lib/fluent/root_agent.rb:154:in `block (2 levels) in lifecycle'
  2018-03-29 14:01:11 +0000 [error]: #0 /var/lib/gems/2.3.0/gems/fluentd-1.1.2/lib/fluent/root_agent.rb:153:in `each'
  2018-03-29 14:01:11 +0000 [error]: #0 /var/lib/gems/2.3.0/gems/fluentd-1.1.2/lib/fluent/root_agent.rb:153:in `block in lifecycle'
  2018-03-29 14:01:11 +0000 [error]: #0 /var/lib/gems/2.3.0/gems/fluentd-1.1.2/lib/fluent/root_agent.rb:140:in `each'
  2018-03-29 14:01:11 +0000 [error]: #0 /var/lib/gems/2.3.0/gems/fluentd-1.1.2/lib/fluent/root_agent.rb:140:in `lifecycle'
  2018-03-29 14:01:11 +0000 [error]: #0 /var/lib/gems/2.3.0/gems/fluentd-1.1.2/lib/fluent/root_agent.rb:164:in `start'
  2018-03-29 14:01:11 +0000 [error]: #0 /var/lib/gems/2.3.0/gems/fluentd-1.1.2/lib/fluent/engine.rb:274:in `start'
  2018-03-29 14:01:11 +0000 [error]: #0 /var/lib/gems/2.3.0/gems/fluentd-1.1.2/lib/fluent/engine.rb:219:in `run'
  2018-03-29 14:01:11 +0000 [error]: #0 /var/lib/gems/2.3.0/gems/fluentd-1.1.2/lib/fluent/supervisor.rb:774:in `run_engine'
  2018-03-29 14:01:11 +0000 [error]: #0 /var/lib/gems/2.3.0/gems/fluentd-1.1.2/lib/fluent/supervisor.rb:523:in `block in run_worker'
  2018-03-29 14:01:11 +0000 [error]: #0 /var/lib/gems/2.3.0/gems/fluentd-1.1.2/lib/fluent/supervisor.rb:699:in `main_process'
  2018-03-29 14:01:11 +0000 [error]: #0 /var/lib/gems/2.3.0/gems/fluentd-1.1.2/lib/fluent/supervisor.rb:518:in `run_worker'
  2018-03-29 14:01:11 +0000 [error]: #0 /var/lib/gems/2.3.0/gems/fluentd-1.1.2/lib/fluent/command/fluentd.rb:316:in `<top (required)>'
  2018-03-29 14:01:11 +0000 [error]: #0 /usr/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
  2018-03-29 14:01:11 +0000 [error]: #0 /usr/lib/ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
  2018-03-29 14:01:11 +0000 [error]: #0 /var/lib/gems/2.3.0/gems/fluentd-1.1.2/bin/fluentd:8:in `<top (required)>'
  2018-03-29 14:01:11 +0000 [error]: #0 /usr/local/bin/fluentd:22:in `load'
  2018-03-29 14:01:11 +0000 [error]: #0 /usr/local/bin/fluentd:22:in `<main>'
2018-03-29 14:01:11 +0000 [error]: #0 unexpected error error_class=RuntimeError error="can't add a new key into hash during iteration"
  2018-03-29 14:01:11 +0000 [error]: #0 suppressed same stacktrace

This is source section from fluentd configuration:

<source>
      @type tail
      path /var/log/fluentd-payload-logger/**/*.log
      pos_file /var/log/fluentd-payload-logger.pos
      tag payload-logger
      read_lines_limit 100
      read_from_head true
      refresh_interval 1
      <parse>
        @type json
      </parse>
    </source>

repeatedly · 2018-03-30T11:51:42Z

Hmm... Does longer refresh_interval, e.g. 10 or larger, have same problem?
Very small refresh_interval may conflict start and refresh_interval threads.

lruslan · 2018-04-18T08:31:52Z

hi @repeatedly thanks for the suggestion I've changed interval to 10 and still see same behaviour:

    <source>
      @type tail
      path /var/log/fluentd-payload-logger/**/*.log
      pos_file /var/log/fluentd-payload-logger.pos
      enable_stat_watcher false
      tag payload-logger
      read_lines_limit 100
      read_from_head true
      refresh_interval 10
      <parse>
        @type json
      </parse>
    </source>

Exception:

2018-04-08 11:15:54 +0000 [error]: #0 Unexpected error raised. Stopping the timer. title=:in_tail_refresh_watchers error_class=RuntimeError error="can't add a new key into hash during iteration"
  2018-04-08 11:15:54 +0000 [error]: #0 /var/lib/gems/2.3.0/gems/fluentd-1.1.2/lib/fluent/plugin/in_tail.rb:297:in `block in start_watchers'
  2018-04-08 11:15:54 +0000 [error]: #0 /var/lib/gems/2.3.0/gems/fluentd-1.1.2/lib/fluent/plugin/in_tail.rb:278:in `each'
  2018-04-08 11:15:54 +0000 [error]: #0 /var/lib/gems/2.3.0/gems/fluentd-1.1.2/lib/fluent/plugin/in_tail.rb:278:in `start_watchers'
  2018-04-08 11:15:54 +0000 [error]: #0 /var/lib/gems/2.3.0/gems/fluentd-1.1.2/lib/fluent/plugin/in_tail.rb:258:in `refresh_watchers'
  2018-04-08 11:15:54 +0000 [error]: #0 /var/lib/gems/2.3.0/gems/fluentd-1.1.2/lib/fluent/plugin_helper/timer.rb:80:in `on_timer'
  2018-04-08 11:15:54 +0000 [error]: #0 /var/lib/gems/2.3.0/gems/cool.io-1.5.3/lib/cool.io/loop.rb:88:in `run_once'
  2018-04-08 11:15:54 +0000 [error]: #0 /var/lib/gems/2.3.0/gems/cool.io-1.5.3/lib/cool.io/loop.rb:88:in `run'
  2018-04-08 11:15:54 +0000 [error]: #0 /var/lib/gems/2.3.0/gems/fluentd-1.1.2/lib/fluent/plugin_helper/event_loop.rb:93:in `block in start'
  2018-04-08 11:15:54 +0000 [error]: #0 /var/lib/gems/2.3.0/gems/fluentd-1.1.2/lib/fluent/plugin_helper/thread.rb:78:in `block in thread_create'
2018-04-08 11:15:54 +0000 [error]: #0 Timer detached. title=:in_tail_refresh_watchers

jl2005 · 2018-04-28T01:28:11Z

@repeatedly @jbrownsc @lruslan

This conflict is cause by prometheus_tail_monitor concurrent access to Fluent::Plugin::TailInput

Detail Info see

fluent/fluent-plugin-prometheus#49

Signed-off-by: 焦龙 <jiaolongdy@163.com>

change read_pos and read_ino. relate #1953 #1804

jl2005 mentioned this issue Apr 28, 2018

prometheus_tail_monitor change position file pos fluent/fluent-plugin-prometheus#49

Closed

jl2005 added a commit to jl2005/fluentd that referenced this issue Apr 28, 2018

change read_pos and read_ino. relate fluent#1953 fluent#1804

7e17b93

Signed-off-by: 焦龙 <jiaolongdy@163.com>

This was referenced Apr 30, 2018

clone variable before iterating to avoid concurrent access fluent/fluent-plugin-prometheus#50

Merged

make FilePositionEntry thread-safe #1966

Closed

repeatedly added a commit that referenced this issue Jun 11, 2018

Merge pull request #1963 from jl2005/master

31fb6a4

change read_pos and read_ino. relate #1953 #1804

repeatedly closed this as completed Jun 13, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

can't add a new key into hash during iteration #1804

can't add a new key into hash during iteration #1804

jl2005 commented Dec 27, 2017 •

edited

Loading

repeatedly commented Dec 29, 2017

jl2005 commented Dec 31, 2017

repeatedly commented Jan 9, 2018

jl2005 commented Feb 6, 2018

repeatedly commented Feb 7, 2018

jl2005 commented Feb 8, 2018

jbrownsc commented Mar 13, 2018 •

edited

Loading

lruslan commented Mar 29, 2018 •

edited

Loading

repeatedly commented Mar 30, 2018

lruslan commented Apr 18, 2018

jl2005 commented Apr 28, 2018 •

edited

Loading

can't add a new key into hash during iteration #1804

can't add a new key into hash during iteration #1804

Comments

jl2005 commented Dec 27, 2017 • edited Loading

repeatedly commented Dec 29, 2017

jl2005 commented Dec 31, 2017

repeatedly commented Jan 9, 2018

jl2005 commented Feb 6, 2018

repeatedly commented Feb 7, 2018

jl2005 commented Feb 8, 2018

jbrownsc commented Mar 13, 2018 • edited Loading

lruslan commented Mar 29, 2018 • edited Loading

repeatedly commented Mar 30, 2018

lruslan commented Apr 18, 2018

jl2005 commented Apr 28, 2018 • edited Loading

jl2005 commented Dec 27, 2017 •

edited

Loading

jbrownsc commented Mar 13, 2018 •

edited

Loading

lruslan commented Mar 29, 2018 •

edited

Loading

jl2005 commented Apr 28, 2018 •

edited

Loading