Permalink
Fetching contributors…
Cannot retrieve contributors at this time
75 lines (56 sloc) 2.79 KB

http://groups.google.com/a/elasticsearch.com/group/users/browse_thread/thread/8517d7ccdaa6a72b

We have 3 servers in each data center, with 28M docs consuming 170G
disk (soon to shrink with ES 0.14), handling about 6k req/min for
client queries and 195k document matches/minute for alerting purposes.
With our hardware, we’re hardly taxing them and still averaging
30-35ms response times.

:index_buffer_size => “512m”, :heap_size => ‘11000’, :fd_ping_interval => ‘2s’, :fd_ping_timeout => ‘60s’, :fd_ping_retries => ‘6’, :seeds => ‘10.116.83.97:9300,10.196.190.111:9300,10.112.45.60:9300,10.118.254.64:9300’, :recovery_after_time => ‘10m’, :recovery_after_nodes => 4, :expected_nodes => 4, :refresh_interval => 900,

with 80 primary / 160 active shards in 5 indexes, each shard sized as approx:

14395 /mnt/elasticsearch/data/hoolock/nodes/0/indices/tweet-2009q3pre/10/index
26615 /mnt/elasticsearch/data/hoolock/nodes/0/indices/tweet-2009q4/0/index
9294 /mnt/elasticsearch/data/hoolock/nodes/0/indices/tweet-201004/12/index
12204 /mnt/elasticsearch/data/hoolock/nodes/0/indices/tweet-201005/12/index

after recovering cluster most nodes were at 7.5 – 9.6 GB

http true:

14409 /mnt/elasticsearch/data/hoolock/nodes/0/indices/tweet-2009q3pre/11/index
26573 /mnt/elasticsearch/data/hoolock/nodes/0/indices/tweet-2009q4/11/index
23885 /mnt/elasticsearch/data/hoolock/nodes/0/indices/tweet-2010q1/4/index
9271 /mnt/elasticsearch/data/hoolock/nodes/0/indices/tweet-201004/0/index
12218 /mnt/elasticsearch/data/hoolock/nodes/0/indices/tweet-201005/4/index

13723 /mnt/elasticsearch/data/hoolock/nodes/0/indices/tweet-201006/9/index
15578 /mnt/elasticsearch/data/hoolock/nodes/0/indices/tweet-201007/6/index
1471 /mnt/elasticsearch/data/hoolock/nodes/0/indices/tweet-201008/11/index
915 /mnt/elasticsearch/data/hoolock/nodes/0/indices/tweet-201009/1/index
1908 /mnt/elasticsearch/data/hoolock/nodes/0/indices/tweet-201010/13/index
2026 /mnt/elasticsearch/data/hoolock/nodes/0/indices/tweet-201011/7/index

“tweet-201010” : “num_docs” : 40272985, “tweet-201011” : “num_docs” : 39012255, “tweet-2009q4” : “num_docs” : 577762139, “tweet-201006” : “num_docs” : 288445236, “tweet-201008” : “num_docs” : 30904989, “tweet-201005” : “num_docs” : 242058418, “tweet-201007” : “num_docs” : 311059766, “tweet-2009q3pre” : “num_docs” : 359075858, “tweet-201004” : “num_docs” : 190501768, “tweet-201009” : “num_docs” : 19166922, “tweet-2010q1” : “num_docs” : 368031331,

14409 tweet-2009q3pre
26590 tweet-2009q4
23923 tweet-2010q1
9278 tweet-201004
12216 tweet-201005
13735 tweet-201006
15580 tweet-201007
1472 tweet-201008
916 tweet-201009
1910 tweet-201010
2023 tweet-201011