tests failing

[ajrpayne]

The libmodsecurity tests pass, and the tests from http://hg.nginx.org/nginx-tests/ pass, but the ModSecurtiy-nginx tests fail for me.

I know the product is in alpha (and I am using the experimental branch), but when I run the tests I get the following results:

$ prove modsecurity*
modsecurity-config-auditlog.t .. nginx: [emerg] "modsecurity_rules" directive Rules error. File:
SecRule ARGS "@Streq root" "id:1,phase:1,auditlog,status:302,redirect:http://www.modsecurity.org"
SecDebugLog /tmp/nginx-test-_hnbNFuBXw/auditlog-debug-root.txt
SecDebugLogLevel 9
SecAuditLogParts AB
SecAuditLog /tmp/nginx-test-_hnbNFuBXw/auditlog-root.txt
SecAuditLogType Serial
SecAuditLogStorageDir /tmp/nginx-test-_hnbNFuBXw/
. Line: 2. Column: 455. syntax error, unexpected end of file, expecting "," or QUOTATION_MARK in /tmp/nginx-test-_hnbNFuBXw/nginx.conf:91
Can't start nginx at lib/Test/Nginx.pm line 274.
modsecurity-config-auditlog.t .. Dubious, test returned 2 (wstat 512, 0x200)
No subtests run
modsecurity-config-debuglog.t .. nginx: [emerg] "modsecurity_rules" directive Rules error. File:
SecRule ARGS "@Streq root" "id:1,phase:1,status:302,redirect:http://www.modsecurity.org"
SecDebugLog /tmp/nginx-test-W5Hd5Xk3DI/debuglog-root.txt
SecDebugLogLevel 9
. Line: 2. Column: 226. syntax error, unexpected end of file, expecting "," or QUOTATION_MARK in /tmp/nginx-test-W5Hd5Xk3DI/nginx.conf:55
Can't start nginx at lib/Test/Nginx.pm line 274.
modsecurity-config-debuglog.t .. Dubious, test returned 2 (wstat 512, 0x200)
No subtests run
modsecurity-config.t ........... nginx: [emerg] "modsecurity_rules" directive Rules error. File:
SecRule ARGS "@Streq root" "id:1,phase:1,status:302,redirect:http://www.modsecurity.org"
. Line: 2. Column: 118. syntax error, unexpected end of file, expecting "," or QUOTATION_MARK in /tmp/nginx-test-Fm9IIs6mvi/nginx.conf:49
Can't start nginx at lib/Test/Nginx.pm line 274.
modsecurity-config.t ........... Dubious, test returned 2 (wstat 512, 0x200)
No subtests run
modsecurity-proxy.t ............ nginx: [emerg] "proxy_read_timeout" directive Rules error. File:
SecRuleEngine On
SecRule ARGS "@Streq redirect301" "id:1,phase:1,status:301,redirect:http://www.modsecurity.org"
SecRule ARGS "@Streq redirect302" "id:1,phase:1,status:302,redirect:http://www.modsecurity.org"
SecRule ARGS "@Streq block401" "id:11,phase:1,status:401,block"
SecRule ARGS "@Streq block403" "id:11,phase:1,status:403,block"
. Line: 3. Column: 397. syntax error, unexpected end of file, expecting "," or QUOTATION_MARK in /tmp/nginx-test-ndF1spIzZj/nginx.conf:79
Can't start nginx at lib/Test/Nginx.pm line 274.
(in cleanup) Can't open error.log: No such file or directory at lib/Test/Nginx.pm line 378.

Looks like your test exited with 2 before it could output anything.

modsecurity-proxy.t ............ Dubious, test returned 2 (wstat 512, 0x200)
Failed 24/24 subtests
modsecurity.t .................. nginx: [emerg] "modsecurity_rules" directive Rules error. File:
SecRuleEngine On
SecRule ARGS "@Streq redirect301" "id:1,phase:1,status:301,redirect:http://www.modsecurity.org"
SecRule ARGS "@Streq redirect302" "id:1,phase:1,status:302,redirect:http://www.modsecurity.org"
SecRule ARGS "@Streq block401" "id:11,phase:1,status:401,block"
SecRule ARGS "@Streq block403" "id:11,phase:1,status:403,block"
. Line: 3. Column: 397. syntax error, unexpected end of file, expecting "," or QUOTATION_MARK in /tmp/nginx-test-IRILiSUeGJ/nginx.conf:74
Can't start nginx at lib/Test/Nginx.pm line 274.
modsecurity.t .................. Dubious, test returned 2 (wstat 512, 0x200)
No subtests run

Test Summary Report

modsecurity-config-auditlog.t (Wstat: 512 Tests: 0 Failed: 0)
Non-zero exit status: 2
Parse errors: No plan found in TAP output
modsecurity-config-debuglog.t (Wstat: 512 Tests: 0 Failed: 0)
Non-zero exit status: 2
Parse errors: No plan found in TAP output
modsecurity-config.t (Wstat: 512 Tests: 0 Failed: 0)
Non-zero exit status: 2
Parse errors: No plan found in TAP output
modsecurity-proxy.t (Wstat: 512 Tests: 0 Failed: 0)
Non-zero exit status: 2
Parse errors: Bad plan. You planned 24 tests but ran 0.
modsecurity.t (Wstat: 512 Tests: 0 Failed: 0)
Non-zero exit status: 2
Parse errors: No plan found in TAP output
Files=5, Tests=0, 2 wallclock secs ( 0.03 usr 0.00 sys + 0.21 cusr 0.07 csys = 0.31 CPU)
Result: FAIL

[zimmerle]

Hi @ajrpayne,

Thank you for the report, i will investigate it.

[LinuxJedi]

I think this was fixed with my patches back in November

[ajrpayne]

Hi @LinuxJedi,

I ran the tests again, and they now appear to be working.

Thank you,

prove modsecurity*
modsecurity-config-auditlog.t .. ok
modsecurity-config-debuglog.t .. ok
modsecurity-config.t ........... ok
modsecurity-proxy.t ............ ok
modsecurity.t .................. ok
All tests successful.

Test Summary Report

modsecurity-config-auditlog.t (Wstat: 0 Tests: 10 Failed: 0)
TODO passed: 10
modsecurity-config-debuglog.t (Wstat: 0 Tests: 4 Failed: 0)
TODO passed: 4
modsecurity-config.t (Wstat: 0 Tests: 10 Failed: 0)
TODO passed: 10
Files=5, Tests=69, 1 wallclock secs ( 0.05 usr 0.00 sys + 0.24 cusr 0.08 csys = 0.37 CPU)
Result: PASS

[zimmerle]

Thank you @LinuxJedi and @ajrpayne.

It seems that our buildbots agrees with you guys.
http://www.modsecurity.org/developers/buildbot/modsecurity-nginx/builders/Linux32%20-%20Nginx%20%28Mainline%29/builds/17

As the tests are working fine, i am closing this issue.