Skip to content

Commit

Permalink
changed default buffer limit parameters:
Browse files Browse the repository at this point in the history
Fluentd core:
  buffer_chunk_limit 256MB -> 8MB
  buffer_queue_limit 128 -> 256

buf_memory:
  buffer_chunk_limit 32MB -> 8MB
  buffer_queue_limit 32 -> 64

out_file:
  buffer_chunk_limit -> 256MB
  • Loading branch information
frsyuki committed May 30, 2012
1 parent 0aa12a5 commit f70fa37
Show file tree
Hide file tree
Showing 3 changed files with 11 additions and 5 deletions.
7 changes: 5 additions & 2 deletions lib/fluent/buffer.rb
Original file line number Diff line number Diff line change
Expand Up @@ -127,8 +127,11 @@ def enable_parallel(b=true)
@parallel_pop = b
end

config_param :buffer_chunk_limit, :size, :default => 256*1024*1024
config_param :buffer_queue_limit, :integer, :default => 128
# This configuration assumes plugins to send records to a remote server.
# Local file based plugins which should provide more reliability and efficiency
# should override buffer_chunk_limit with a larger size.
config_param :buffer_chunk_limit, :size, :default => 8*1024*1024
config_param :buffer_queue_limit, :integer, :default => 256

alias chunk_limit buffer_chunk_limit
alias chunk_limit= buffer_chunk_limit=
Expand Down
6 changes: 3 additions & 3 deletions lib/fluent/plugin/buf_memory.rb
Original file line number Diff line number Diff line change
Expand Up @@ -68,9 +68,9 @@ def initialize
super
end

# overwrite default buffer_chunk_limit and buffer_queue_limit
config_set_default :buffer_chunk_limit, 32*1024*1024
config_set_default :buffer_queue_limit, 32
# Overwrite default BasicBuffer#buffer_queue_limit
# to limit total memory usage upto 512MB.
config_set_default :buffer_queue_limit, 64

def configure(conf)
super
Expand Down
3 changes: 3 additions & 0 deletions lib/fluent/plugin/out_file.rb
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,9 @@ def configure(conf)
conf['buffer_path'] ||= "#{@path}.*"
end

# override default buffer_chunk_limit
conf['buffer_chunk_limit'] ||= '256m'

super

@timef = TimeFormatter.new(@time_format, @localtime)
Expand Down

0 comments on commit f70fa37

Please sign in to comment.