[Monolog Bridge][DX] Add a Monolog activation strategy for ignoring specific HTTP codes #23707

simshaun · 2017-07-28T21:01:53Z

Q	A
Branch?	3.4
Bug fix?	no
New feature?	yes
BC breaks?	no
Deprecations?	no
Tests pass?	yes
Fixed tickets	#9712
License	MIT
Doc PR	symfony/symfony-docs#8235

This PR introduces a Monolog activation strategy that makes it easy to ignore specific HTTP codes. e.g. Stopping logs from being flooded with 403s, 404s, and 405s.

Relevant Symfony integration PR on symfony/monolog-bundle: symfony/monolog-bundle#221

chalasr · 2017-07-28T21:17:20Z

New features should target the 3.4 branch.

simshaun · 2017-07-28T21:21:29Z

Sorry about that. Rebased to 3.4.

How do I get AppVeyor to run again? It says "continuous-integration/appveyor/pr — AppVeyor was unable to build non-mergeable "

chalasr · 2017-07-29T09:54:42Z

Seems like you changed the PR target but not correctly rebased to the 3.4 branch.

simshaun · 2017-07-29T17:59:00Z

I think I fixed the rebase.

The Travis failure seems to be unrelated:

There was 1 failure:
1) Symfony\Component\Lock\Tests\Store\FlockStoreTest::testBlockingLocks
The store saves a locked key.
/home/travis/build/symfony/symfony/src/Symfony/Component/Lock/Tests/Store/BlockingStoreTestTrait.php:57
/home/travis/build/symfony/symfony/.phpunit/phpunit-6.0/phpunit:5
/home/travis/build/symfony/symfony/vendor/symfony/phpunit-bridge/bin/simple-phpunit:192
FAILURES!
Tests: 99, Assertions: 261, Failures: 1.
KO src/Symfony/Component/Lock

nicolas-grekas · 2017-09-26T14:31:15Z

Just wondering: couldn't/shouldn't this be submitted to monolog instead?

simshaun · 2017-09-26T20:10:22Z

Maybe. I did it here since the bridge contains the NotFoundActivationStrategy and this is similar.

nicolas-grekas · 2017-09-26T20:45:01Z

OK my bad it's legit, Symfony namespace here :)

nicolas-grekas · 2017-09-26T20:46:48Z

So, ping @lsmith77 I guess, original report of the feature request.

fabpot · 2017-10-01T17:59:33Z

Looks useful to me, but what about deprecating the NotFoundActivationStrategy in favor of this one, which is more generic (we should add support for the blacklist option though).

fabpot · 2017-10-01T17:57:31Z

src/Symfony/Bridge/Monolog/Handler/FingersCrossed/HttpCodeActivationStrategy.php

+            $isActivated
+            && isset($record['context']['exception'])
+            && $record['context']['exception'] instanceof HttpException
+            && ($request = $this->requestStack->getMasterRequest())


$request is not used

I think you are wrong, $request is used here: https://github.com/symfony/symfony/pull/23707/files#diff-ca46116b05ee5c3e47d1438e580c53eeR53

It wasn't being used before. It is now with the addition of URL blacklisting.

nicolas-grekas · 2017-10-08T16:26:12Z

Moving to 4.1. Rebase on master might be needed, where PHP 7.1 features can be used btw.

pimolo · 2017-12-18T01:40:09Z

src/Symfony/Bridge/Monolog/Handler/FingersCrossed/HttpCodeActivationStrategy.php

+ *
+ * @author Shaun Simmons <shaun@envysphere.com>
+ */
+class HttpCodeActivationStrategy extends ErrorLevelActivationStrategy


It should not extend ErrorLevelActivationStrategy but rather implement ActivationStrategyInterface, since this strategy doesn't deal with the error levels.

Plus, this class could be final IMO.

@pimolo The FingersCrossedHandler leaves it up to the activation strategy object to handle the action level. It defaults to the ErrorLevelActivationStrategy only when it isn't explicitly given an object implementing ActivationStrategyInterface. Because an HttpCodeActivationStrategy object is given to the FingersCrossedHandler when the excluded_http_codes configuration exists, the HttpCodeActivationStrategy becomes responsible for handling the action level, just like the existing NotFoundActivationStrategy does. So, that's why it's extending the ErrorLevelActivationStrategy class.

soullivaneuh · 2018-02-01T12:38:24Z

src/Symfony/Bridge/Monolog/Handler/FingersCrossed/HttpCodeActivationStrategy.php

+ *
+ * @author Shaun Simmons <shaun@envysphere.com>
+ */
+class HttpCodeActivationStrategy extends ErrorLevelActivationStrategy


Plus, this class could be final IMO.

soullivaneuh · 2018-02-01T12:46:14Z

@nicolas-grekas What is the status of the PR?

@simshaun are you sill handling it? If not, I may rework it.

Looks useful to me, but what about deprecating the NotFoundActivationStrategy in favor of this one, which is more generic (we should add support for the blacklist option though).

@fabpot Makes sense to me. We could imagine something like that:

excluded_http_erros:
    403: ^/    # All 403 errors are excluded
    404: ^/admin # All 404 errors are excluded for /admin/*
    400: # Can be still multiple regex. 
        - ^/csrf/
        - ^/foo/bar/

simshaun · 2018-02-01T19:02:34Z

I'm working on adding the URL blacklist. @fabpot, does the configuration format @soullivaneuh showed above look good?

One thing I'm thinking could be improved for user-friendliness: If you want a code to be excluded for all URLs, you just pass true instead of a ^/ regex.

excluded_http_codes:
    403: true           # All 403 errors are excluded
    404: ^/admin        # All 404 errors are excluded for /admin/*
    400:                # Can contain multiple regex
        - ^/csrf/
        - ^/foo/bar/

soullivaneuh · 2018-02-02T15:17:18Z

@simshaun I was thinking about the true option but didn't propose that to simplify the work. But if you think it can be implemented easily then 👍. :-)

javiereguiluz · 2018-02-09T15:08:23Z

Just asking: do we really want to allow excluding HTTP status per URL? If we don't do that, this would be really simple:

excluded_http_codes: [400, 403, 404]

If we really must support excluding per URL, do we really want to mix booleans and regexp strings? As proposed above, putting 403: ^/ would exclude 403 for all URLs, without the need to support, explain, learn and maintain forever the true value as an special option.

And now a proposal that mixes both ideas. What if we add support for URLs, but make them optional? Like this:

excluded_http_codes: [403, 404]

excluded_http_codes: [{ 404: '^/some/section' }]

excluded_http_codes: [403, 404, { 400: ['^/csrf', '^/foo/bar' ] }]

simshaun · 2018-02-09T15:26:50Z

I'm good with that too.

simshaun · 2018-02-18T10:48:44Z

I've updated this PR to support URL blacklists using the YAML format suggested by @javiereguiluz. The other PRs referenced in the OP (MonologBundle and docs) have been updated as well.

I really struggled with making the Configuration tree work with both YAML and XML. I ended up just normalizing the two into a common format that the new activation strategy understands.

It now supports:

YAML:

excluded_http_codes: [403, 404, { 400: ['^/foo', '^/bar'] }]

XML:

<monolog:excluded-http-code code="403" />
<monolog:excluded-http-code code="404" />
<monolog:excluded-http-code code="400">
  <monolog:url>^/foo</monolog:url>
  <monolog:url>^/bar</monolog:url>
</monolog:excluded-http-code>

simshaun · 2018-02-20T19:32:59Z

Looks like the Travis failure is unrelated. Lots of problems with FormExtensionBootstrap4HorizontalLayoutTest

nicolas-grekas · 2018-04-03T07:34:31Z

src/Symfony/Bridge/Monolog/Handler/FingersCrossed/HttpCodeActivationStrategy.php

+                }
+
+                $urlBlacklist = null;
+                if (count($exclusion['url'])) {


If ! empty(...) could prevent an issue when the url index is not defined.

Validation added in the constructor

nicolas-grekas · 2018-04-03T07:35:23Z

src/Symfony/Bridge/Monolog/Handler/FingersCrossed/HttpCodeActivationStrategy.php

+            && ($request = $this->requestStack->getMasterRequest())
+        ) {
+            foreach ($this->exclusions as $exclusion) {
+                if ($record['context']['exception']->getStatusCode() !== $exclusion['code']) {


Would it be useful to ensure that the code key exists, and throw otherwise?

Done in the constructor

fabpot · 2018-04-04T06:34:03Z

Thank you @simshaun.

… for ignoring specific HTTP codes (simshaun, fabpot) This PR was merged into the 4.1-dev branch. Discussion ---------- [Monolog Bridge][DX] Add a Monolog activation strategy for ignoring specific HTTP codes | Q | A | ------------- | --- | Branch? | 3.4 | Bug fix? | no | New feature? | yes | BC breaks? | no | Deprecations? | no | Tests pass? | yes | Fixed tickets | #9712 | License | MIT | Doc PR | symfony/symfony-docs#8235 This PR introduces a Monolog activation strategy that makes it easy to ignore specific HTTP codes. e.g. Stopping logs from being flooded with 403s, 404s, and 405s. Relevant Symfony integration PR on symfony/monolog-bundle: symfony/monolog-bundle#221 Commits ------- 6fc1cc3 added some validation c11a8eb Add a Monolog activation strategy for ignoring specific HTTP codes

…mshaun, fabpot) This PR was merged into the 3.x-dev branch. Discussion ---------- Add configuration for new HttpCodeActivationStrategy Dependent upon symfony/symfony#23707 This PR introduces configuration to integrate a new `HttpCodeActivationStrategy`, offering an easy way to ignore specific HTTP status codes. Commits ------- 30a70a2 tweaked code 61a78a6 Add configuration for new HttpCodeActivationStrategy

…n, javiereguiluz) This PR was merged into the 4.1 branch. Discussion ---------- Document new Monolog HTTP code exclusion feature I'm submitting PRs on symfony/symfony and symfony/monolog-bundle that introduces an easy way of excluding specific HTTP codes from Monolog. This PR documents that new feature, pending its acceptance. Pending PRs: symfony/symfony#23707 symfony/monolog-bundle#221 Commits ------- 30cc7d4 Added the versionadded directive c8c19f2 Update to Symfony 4 paths e19bca2 Fix indentation 17cb416 Update configuration format to support URL blacklist c9f6ee5 Fix title underline length a3b6a01 Document new Monolog HTTP code exclusion feature

fmonts · 2018-06-20T12:21:55Z

The feature is not working as expected.

I added excluded_http_codes: [403, 404]

and I still got:

[2018-06-20 12:18:05] security.INFO: Populated the TokenStorage with an anonymous Token. [] []
[2018-06-20 12:18:05] request.CRITICAL: Uncaught PHP Exception Symfony\Component\Security\Core\Exception\AccessDeniedException: "Access Denied." at /../vendor/symfony/security/Http/Firewall/AccessListener.php line 68 {"exception":"[object] (Symfony\Component\Security\Core\Exception\AccessDeniedException(code: 403): Access Denied. at /.../vendor/symfony/security/Http/Firewall/AccessListener.php:68)"} []
[2018-06-20 12:18:05] security.DEBUG: Access denied, the user is not fully authenticated; redirecting to authentication entry point. {"exception":"[object] (Symfony\Component\Security\Core\Exception\AccessDeniedException(code: 403): Access Denied. at /.../vendor/symfony/security/Http/Firewall/AccessListener.php:68)"} []
[2018-06-20 12:18:05] security.DEBUG: Calling Authentication entry point. [] []

fmonts · 2018-06-20T12:22:27Z

...should I open a new issue?

simshaun · 2018-06-20T19:02:16Z

@fmonts A new issue would be better. Please @ me and I'll try to take a look.

Also, please include the context of your logging config. (I'm mostly just curious if you're using the fingers_crossed type.)

…mshaun, fabpot) This PR was merged into the 3.x-dev branch. Discussion ---------- Add configuration for new HttpCodeActivationStrategy Dependent upon symfony/symfony#23707 This PR introduces configuration to integrate a new `HttpCodeActivationStrategy`, offering an easy way to ignore specific HTTP status codes. Commits ------- 2e7b3fe tweaked code f90e3e6 Add configuration for new HttpCodeActivationStrategy

carsonbot added Status: Needs Review Feature labels Jul 28, 2017

simshaun mentioned this pull request Jul 28, 2017

Add configuration for new HttpCodeActivationStrategy symfony/monolog-bundle#221

Merged

simshaun changed the title ~~Add a Monolog activation strategy for ignoring specific HTTP codes~~ [Monolog Bridge][DX] Add a Monolog activation strategy for ignoring specific HTTP codes Jul 28, 2017

simshaun mentioned this pull request Jul 28, 2017

Document new Monolog HTTP code exclusion feature symfony/symfony-docs#8235

Merged

chalasr added the MonologBridge label Jul 28, 2017

simshaun force-pushed the 2.7-monolog-exclude-http-codes branch from 9a93318 to 7beede2 Compare July 28, 2017 21:20

simshaun changed the base branch from 2.7 to 3.4 July 28, 2017 21:20

chalasr added this to the 3.4 milestone Jul 29, 2017

simshaun force-pushed the 2.7-monolog-exclude-http-codes branch from 33e4c94 to 6a0e3e6 Compare July 29, 2017 17:47

fabpot reviewed Oct 1, 2017

View reviewed changes

nicolas-grekas modified the milestones: 3.4, 4.1 Oct 8, 2017

mtibben mentioned this pull request Dec 17, 2017

[HttpKernel][DX] Allow the log level of http exceptions to be overridden #25533

Closed

carsonbot removed the Status: Needs Review label Dec 18, 2017

pimolo suggested changes Dec 18, 2017

View reviewed changes

carsonbot added the Status: Needs Work label Dec 18, 2017

soullivaneuh suggested changes Feb 1, 2018

View reviewed changes

carsonbot added Status: Reviewed and removed Status: Needs Work labels Apr 3, 2018

fabpot approved these changes Apr 3, 2018

View reviewed changes

nicolas-grekas reviewed Apr 3, 2018

View reviewed changes

fabpot changed the base branch from 3.4 to master April 4, 2018 06:16

fabpot force-pushed the 2.7-monolog-exclude-http-codes branch from f108940 to 067502d Compare April 4, 2018 06:16

Add a Monolog activation strategy for ignoring specific HTTP codes

c11a8eb

fabpot force-pushed the 2.7-monolog-exclude-http-codes branch 2 times, most recently from be3ec97 to 27f9cb2 Compare April 4, 2018 06:26

added some validation

6fc1cc3

fabpot force-pushed the 2.7-monolog-exclude-http-codes branch from 27f9cb2 to 6fc1cc3 Compare April 4, 2018 06:31

nicolas-grekas approved these changes Apr 4, 2018

View reviewed changes

fabpot merged commit 6fc1cc3 into symfony:master Apr 4, 2018

fabpot mentioned this pull request May 7, 2018

Release v4.1.0-BETA1 #27181

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Monolog Bridge][DX] Add a Monolog activation strategy for ignoring specific HTTP codes #23707

[Monolog Bridge][DX] Add a Monolog activation strategy for ignoring specific HTTP codes #23707

simshaun commented Jul 28, 2017 •

edited by javiereguiluz

chalasr commented Jul 28, 2017

simshaun commented Jul 28, 2017 •

edited

chalasr commented Jul 29, 2017

simshaun commented Jul 29, 2017

nicolas-grekas commented Sep 26, 2017

simshaun commented Sep 26, 2017

nicolas-grekas commented Sep 26, 2017

nicolas-grekas commented Sep 26, 2017

fabpot commented Oct 1, 2017

fabpot Oct 1, 2017

soullivaneuh Feb 20, 2018

simshaun Feb 20, 2018

nicolas-grekas commented Oct 8, 2017

pimolo Dec 18, 2017

soullivaneuh Feb 1, 2018

simshaun Feb 1, 2018 •

edited

soullivaneuh Feb 1, 2018

soullivaneuh commented Feb 1, 2018 •

edited

simshaun commented Feb 1, 2018 •

edited

soullivaneuh commented Feb 2, 2018

javiereguiluz commented Feb 9, 2018

simshaun commented Feb 9, 2018 •

edited

simshaun commented Feb 18, 2018

simshaun commented Feb 20, 2018

nicolas-grekas Apr 3, 2018

fabpot Apr 4, 2018

nicolas-grekas Apr 3, 2018

fabpot Apr 4, 2018

fabpot commented Apr 4, 2018

fmonts commented Jun 20, 2018

fmonts commented Jun 20, 2018

simshaun commented Jun 20, 2018

[Monolog Bridge][DX] Add a Monolog activation strategy for ignoring specific HTTP codes #23707

[Monolog Bridge][DX] Add a Monolog activation strategy for ignoring specific HTTP codes #23707

Conversation

simshaun commented Jul 28, 2017 • edited by javiereguiluz

chalasr commented Jul 28, 2017

simshaun commented Jul 28, 2017 • edited

chalasr commented Jul 29, 2017

simshaun commented Jul 29, 2017

nicolas-grekas commented Sep 26, 2017

simshaun commented Sep 26, 2017

nicolas-grekas commented Sep 26, 2017

nicolas-grekas commented Sep 26, 2017

fabpot commented Oct 1, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nicolas-grekas commented Oct 8, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

simshaun Feb 1, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

soullivaneuh commented Feb 1, 2018 • edited

simshaun commented Feb 1, 2018 • edited

soullivaneuh commented Feb 2, 2018

javiereguiluz commented Feb 9, 2018

simshaun commented Feb 9, 2018 • edited

simshaun commented Feb 18, 2018

simshaun commented Feb 20, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fabpot commented Apr 4, 2018

fmonts commented Jun 20, 2018

fmonts commented Jun 20, 2018

simshaun commented Jun 20, 2018

simshaun commented Jul 28, 2017 •

edited by javiereguiluz

simshaun commented Jul 28, 2017 •

edited

simshaun Feb 1, 2018 •

edited

soullivaneuh commented Feb 1, 2018 •

edited

simshaun commented Feb 1, 2018 •

edited

simshaun commented Feb 9, 2018 •

edited