custom patterns - permissions denied

[ghost]

I setup a custom pattern file in /opt/logstash/patterns/patterns/

when I restart the service logstash will not work/filter logs

when I review the logstash logs in /var/log/ I see this:
The error reported is: \n Permission denied - /opt/logstash/patterns/patterns"}

these are the permissions though:
drw-rw-rw- 2 logstash logstash 4096 Oct 3 10:02 patterns

here is my filter clause:
grok {
break_on_match => "true"
patterns_dir => "./patterns"

[electrical]

Hi,

can you check the permissions of the file it self as well?
And all related directories.

[jsvd]

The directory must have the executable permission set, so it should look like:
drwxrwxrwx 2 logstash logstash 4096 Oct 3 10:02 patterns
instead of
drw-rw-rw- 2 logstash logstash 4096 Oct 3 10:02 patterns

[ghost]

-rw-rw-rw- 1 logstash logstash 331 Oct 3 11:22 custpatterns

Thanks,
Kevin
From: Richard Pijnenburg [mailto:notifications@github.com]
Sent: Friday, October 03, 2014 11:44 AM
To: elasticsearch/logstash
Cc: Kevin McGillicuddy
Subject: Re: [logstash] custom patterns - permissions denied (#1841)

Hi,

can you check the permissions of the file it self as well?
And all related directories.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/1841#issuecomment-57813747.

[ghost]

I tried that before – I did chmod 777 then switch it to 666 – I will switch it back to 777 and left you know

Thanks,
Kevin

From: João Duarte [mailto:notifications@github.com]
Sent: Friday, October 03, 2014 11:49 AM
To: elasticsearch/logstash
Cc: Kevin McGillicuddy
Subject: Re: [logstash] custom patterns - permissions denied (#1841)

The directory must have the executable permission set, so it should look like:
drwxrwxrwx 2 logstash logstash 4096 Oct 3 10:02 patterns
instead of
drw-rw-rw- 2 logstash logstash 4096 Oct 3 10:02 patterns

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/1841#issuecomment-57814521.

[ghost]

Yea same thing:

The error reported is: \n Permission denied - /opt/logstash/patterns/patterns"}

drwxrwxrwx 2 logstash logstash 4096 Oct 3 10:02 patterns
-rwxrwxrwx 1 logstash logstash 331 Oct 3 11:22 custpatterns

Thanks,
Kevin

From: João Duarte [mailto:notifications@github.com]
Sent: Friday, October 03, 2014 11:49 AM
To: elasticsearch/logstash
Cc: Kevin McGillicuddy
Subject: Re: [logstash] custom patterns - permissions denied (#1841)

The directory must have the executable permission set, so it should look like:
drwxrwxrwx 2 logstash logstash 4096 Oct 3 10:02 patterns
instead of
drw-rw-rw- 2 logstash logstash 4096 Oct 3 10:02 patterns

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/1841#issuecomment-57814521.

[jsvd]

Can you read the file as user logstash?

sudo -u logstash "cat <file>"

[ghost]

Yea

root@st-log01:/opt/logstash/patterns/patterns# sudo -u logstash cat custpatterns
GNATTIME (?:\d_.)?\d+-(?:\d_.)?\d+-(?:\d_.)?\d+ (?:\d_.)?\d+:(?:\d_.)?\d+:(?:\d_.)?\d+
FLAGS (?:\d_.)?\d+x(?:\d_.)?\d+
MSG [a-zA-Z]+ [a-zA-Z]+
DURATION (?:\d_.)?\d+
PROTO (?:\d_.)?\d+/\b\w{3}\b
INTERFACE [a-zA-Z]+ ([a-zA-Z]+)
TIME1 (?!<[0-9])(?:2[0123]|[01]?[0-9]):(?:[0-5][0-9]):(?:(?:[0-5]?[0-9]|60)(?:[:.,][0-9]+)?)

Thanks,
Kevin

From: João Duarte [mailto:notifications@github.com]
Sent: Friday, October 03, 2014 11:56 AM
To: elasticsearch/logstash
Cc: Kevin McGillicuddy
Subject: Re: [logstash] custom patterns - permissions denied (#1841)

Can you read the file as user logstash?

sudo -u logstash "cat "

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/1841#issuecomment-57815383.

[jsvd]

What is the current directory where you're running logstash?

it's saying "/opt/logstash/patterns/patterns" is denied, you have a patterns directory inside the patterns dir, that's correct?

[ghost]

Yes I do logstash is running of opt/Logstash/bin

root@st-log01:/opt/logstash/patterns/patterns# ps -ef | grep logstash
logstash 22666 1 29 13:45 ? 00:00:23 /usr/bin/java -Djava.io.tmpdir=/var/lib/logstash -Xmx500m -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -Djava.awt.headless=true -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -jar /opt/logstash/vendor/jar/jruby-complete-1.7.11.jar -I/opt/logstash/lib /opt/logstash/lib/logstash/runner.rb web -l /var/log/logstash/logstash.log
logstash 22799 1 99 13:46 ? 00:00:13 /usr/bin/java -Djava.io.tmpdir=/var/lib/logstash -Xmx500m -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -Djava.awt.headless=true -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -jar /opt/logstash/vendor/jar/jruby-complete-1.7.11.jar -I/opt/logstash/lib /opt/logstash/lib/logstash/runner.rb agent -f /etc/logstash/conf.d -l /var/log/logstash/logstash.log
root 22826 1543 0 13:46 pts/2 00:00:00 grep --color=auto logstash

root@st-log01:/opt/logstash/patterns# ls -l
total 76
-rw-rw-r-- 1 logstash logstash 7518 May 14 21:43 firewalls
-rw-r--r-- 1 logstash logstash 5309 Oct 2 16:00 grok
-rwxrwxr-x 1 logstash logstash 5309 Jun 9 14:53 grok-patterns
-rw-rw-r-- 1 logstash logstash 3142 May 14 21:43 haproxy
-rw-rw-r-- 1 logstash logstash 167 Jun 9 14:53 java
-rw-rw-r-- 1 logstash logstash 1097 Jun 9 14:53 junos
-rw-rw-r-- 1 logstash logstash 1031 Jun 9 14:53 linux-syslog
-rw-rw-r-- 1 logstash logstash 49 May 14 21:43 mcollective
-rw-rw-r-- 1 logstash logstash 190 May 14 21:43 mcollective-patterns
-rw-rw-r-- 1 logstash logstash 414 Jun 9 14:53 mongodb
-rw-rw-r-- 1 logstash logstash 7761 May 14 21:43 nagios
drwxrwxrwx 2 logstash logstash 4096 Oct 3 10:02 patterns
-rw-rw-r-- 1 logstash logstash 142 May 14 21:43 postgresql
-rw-rw-r-- 1 logstash logstash 104 May 14 21:43 redis
-rw-rw-r-- 1 logstash logstash 188 May 14 21:43 ruby

Thanks,
Kevin McGillicuddy
Server Administrator
Sight & Sound Theatres
717-687-4220 x2317
kevin.mcgillicuddy@sight-sound.commailto:kevin.mcgillicuddy@sight-sound.com

[http://www.sight-sound.com/StaticContent/images/signature.gif]http://www.sight-sound.com/
[http://www.sight-sound.com/StaticContent/images/youtube.gif]http://www.youtube.com/user/sightsoundtheatres[http://www.sight-sound.com/StaticContent/images/facebook.gif]http://www.facebook.com/sightsoundtheatres

From: João Duarte [mailto:notifications@github.com]
Sent: Friday, October 03, 2014 12:03 PM
To: elasticsearch/logstash
Cc: Kevin McGillicuddy
Subject: Re: [logstash] custom patterns - permissions denied (#1841)

What is the current directory where you're running logstash?

it's saying "/opt/logstash/patterns/patterns" is denied, you have a patterns directory inside the patterns dir, that's correct?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/1841#issuecomment-57816274.

[ghost]

Yes I do logstash is running of opt/Logstash/bin

root@st-log01:/opt/logstash/patterns/patterns# ps -ef | grep logstash
logstash 22666 1 29 13:45 ? 00:00:23 /usr/bin/java -Djava.io.tmpdir=/var/lib/logstash -Xmx500m -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -Djava.awt.headless=true -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -jar /opt/logstash/vendor/jar/jruby-complete-1.7.11.jar -I/opt/logstash/lib /opt/logstash/lib/logstash/runner.rb web -l /var/log/logstash/logstash.log
logstash 22799 1 99 13:46 ? 00:00:13 /usr/bin/java -Djava.io.tmpdir=/var/lib/logstash -Xmx500m -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -Djava.awt.headless=true -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -jar /opt/logstash/vendor/jar/jruby-complete-1.7.11.jar -I/opt/logstash/lib /opt/logstash/lib/logstash/runner.rb agent -f /etc/logstash/conf.d -l /var/log/logstash/logstash.log
root 22826 1543 0 13:46 pts/2 00:00:00 grep --color=auto logstash

root@st-log01:/opt/logstash/patterns# ls -l
total 76
-rw-rw-r-- 1 logstash logstash 7518 May 14 21:43 firewalls
-rw-r--r-- 1 logstash logstash 5309 Oct 2 16:00 grok
-rwxrwxr-x 1 logstash logstash 5309 Jun 9 14:53 grok-patterns
-rw-rw-r-- 1 logstash logstash 3142 May 14 21:43 haproxy
-rw-rw-r-- 1 logstash logstash 167 Jun 9 14:53 java
-rw-rw-r-- 1 logstash logstash 1097 Jun 9 14:53 junos
-rw-rw-r-- 1 logstash logstash 1031 Jun 9 14:53 linux-syslog
-rw-rw-r-- 1 logstash logstash 49 May 14 21:43 mcollective
-rw-rw-r-- 1 logstash logstash 190 May 14 21:43 mcollective-patterns
-rw-rw-r-- 1 logstash logstash 414 Jun 9 14:53 mongodb
-rw-rw-r-- 1 logstash logstash 7761 May 14 21:43 nagios
drwxrwxrwx 2 logstash logstash 4096 Oct 3 10:02 patterns
-rw-rw-r-- 1 logstash logstash 142 May 14 21:43 postgresql
-rw-rw-r-- 1 logstash logstash 104 May 14 21:43 redis
-rw-rw-r-- 1 logstash logstash 188 May 14 21:43 ruby

Thanks,
Kevin

[MrShine]

There seems to be a general problem with folders placed in the patterns folder. Move your custom patterns folder outside the patterns folder and you should be fine. (Tested with logstash-1.4.2)

[jordansissel]

For Logstash 1.5.0, we've moved all plugins to individual repositories, so I have moved this issue to logstash-plugins/logstash-filter-grok#38. Let's continue the discussion there! :)