Logstash error: Error: Permission denied - /var/lib/logstash/sincedb_collector.20212.23238.470530

[darinfisher]

Hi,

I am having the same problem as issue #3311 on CentOS 6.7 w/ Logstash 1.5.4

It has occurred w/o setting the 'sincedb_path' and also with setting it to a specific file
This will run and update the file fine for a period of time, anywhere from 30 minutes to 5 hours and then crash....
{:timestamp=>"2015-10-21T06:42:30.890000-0700", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::File type=>"GA_collector_1", path=>["/mnt/dc1filer02//current//logs/collector/collector.log"], start_position=>"beginning", codec=><LogStash::Codecs::Multiline patterns_dir=>["/etc/logstash/conf.d/patterns"], pattern=>"^%{GATIMESTAMP}|%{IP} ", negate=>true, what=>"previous", max_lines=>2500, charset=>"UTF-8", multiline_tag=>"multiline", max_bytes=>10485760>, debug=>false, stat_interval=>1, discover_interval=>15, sincedb_write_interval=>15, delimiter=>"\n">\n Error: Permission denied - /var/lib/logstash/.sincedb_ea02cd1022d953a4b0364cbd826c437f.18696.19185.765920", :level=>:error}

{:timestamp=>"2015-10-21T07:42:50.868000-0700", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::File type=>"GA_riskengine_1", path=>["/mnt/dc1filer02//current//logs/riskengine/*/riskengine.log"], start_position=>"beginning", sincedb_path=>"/var/lib/logstash/sincedb_riskengine", codec=><LogStash::Codecs::Multiline patterns_dir=>["/etc/logstash/conf.d/patterns"], pattern=>"^%{GATIMESTAMP}|%{IP} ", negate=>true, what=>"previous", max_lines=>2500, charset=>"UTF-8", multiline_tag=>"multiline", max_bytes=>10485760>, debug=>false, stat_interval=>1, discover_interval=>15, sincedb_write_interval=>15, delimiter=>"\n">\n Error: Permission denied - /var/lib/logstash/sincedb_riskengine.25256.21254.235099", :level=>:error}

{:timestamp=>"2015-10-21T08:07:23.659000-0700", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::File type=>"GA_collector_1", path=>["/mnt/dc1filer02//current//logs/collector/collector.log"], start_position=>"beginning", sincedb_path=>"/var/lib/logstash/sincedb_collector", codec=><LogStash::Codecs::Multiline patterns_dir=>["/etc/logstash/conf.d/patterns"], pattern=>"^%{GATIMESTAMP}|%{IP} ", negate=>true, what=>"previous", max_lines=>2500, charset=>"UTF-8", multiline_tag=>"multiline", max_bytes=>10485760>, debug=>false, stat_interval=>1, discover_interval=>15, sincedb_write_interval=>15, delimiter=>"\n">\n Error: Permission denied - /var/lib/logstash/sincedb_collector.20212.23238.470530", :level=>:error}

file {
type => "GA_riskengine_1"
path => "/mnt/dc1filer02//current//logs/riskengine/*/riskengine.log"
start_position => "beginning"
sincedb_path => "/var/lib/logstash/sincedb_riskengine"
codec => multiline {
patterns_dir => "/etc/logstash/conf.d/patterns"
pattern => "^%{GATIMESTAMP}|%{IP} "
negate => true
what => previous
max_lines => 2500
}
}

[root@mon-esc-2 ~]# ls -alt /var/lib/logstash/
total 80
drwxrwxr-x. 2 logstash logstash 4096 Oct 21 09:00 .
-rw-r--r--. 1 logstash logstash 3690 Oct 21 09:00 sincedb_collector
-rw-r--r--. 1 logstash logstash 68702 Oct 21 08:59 sincedb_riskengine
drwxr-xr-x. 22 root root 4096 Oct 21 03:46 ..

Suggestions?

[darinfisher]

No longer utilizing this and there are much better version now.