Do not sanity check hash of anonymous requests #289

bdunogier · 2016-03-03T19:02:41Z

Fixes an issue described in #274

The hardcoded anonymous hash might differ from the one used in the sanity check introduced by this PR. This causes cache expirations for us since anonymous requests don't match.

This change excludes anonymous requests, using an AnonymousRequestMatcher, from this sanity check.

ezsystems/ezpublish-kernel#1601 describes the consequences and way to reproduce on ezplatform.

TODO

Update tests
Check inconsistency with what is used for the Vary header

bdunogier · 2016-03-03T21:13:41Z

Travis looks very broken, but I doubt that my changes did that... how stable is the 1.3 branch ? :-)

ddeboer · 2016-03-06T17:04:46Z

The test failures have to do with your introduction of AnonymousRequestMatcher. Just to be sure, I restarted the tests on the 1.3 branch and they are still green.

bdunogier · 2016-03-09T08:55:39Z

Okay, thank you @ddeboer.

dbu · 2016-03-29T11:20:18Z

EventListener/UserContextSubscriber.php

+     * Tests if $request is an anonymous request or not.
+     *
+     * For backward compatibility reasons, true will be returned if no anonymous request matcher was provided.
+     *


please only use Request here, use statements work for annotations too

dbu · 2016-03-29T11:40:49Z

i agree with the solution. if i think this through correctly, things should work unless you manage to re-use a non-anonymous hash with an anonymous request - but that would be a misconfiguration of your application. #274 was about expired sessions, where the server side session has expired, but the session id is still sent in the request. in that case, we find the hash in the cache and do the request with hash and session id, and thus compare hashes (and see that they mismatch, or if the hash was not in the cache, see the "real" anonymous hash, vs the fake hash ez would create on a request without session cookie). @bastnic maybe you can give us your thoughts as well?

(btw, i still feel ezpublish would be better off simply do a hash lookup for anonymous users as well and then cache that - its a low overhead keeping many things simpler. for example, the anonymous cache will be duplicated between completely anonymous users and users that are determined by symfony to be anonymous because the session expired or something.)

the tests seem to say that the configuration for the anonymous matcher is wrong.

bdunogier · 2016-04-29T12:16:10Z

Review remarks taken care of.

dbu · 2016-04-30T07:20:14Z

EventListener/UserContextSubscriber.php

+     * @return bool
+     */
+    private function isAnonymous(Request $request)
+    {


lets remove the isset here and just say $this->anonymousRequestMatcher ? $this-... or if you don't like this null !== $this->anonymousRequestMatcher ?

bdunogier · 2016-07-01T11:59:48Z

Hmm, does anyone know what could cause PHP Warning: Unexpected character in input: '�' (ASCII=18) state=0 in /home/travis/build/FriendsOfSymfony/FOSHttpCacheBundle/vendor/symfony/symfony/src/Symfony/Bridge/Twig/Extension/CodeExtension.php on line 140 ?

Tests pass except for this error.

dbu · 2016-07-01T12:01:14Z

the lowest version test fails. i just re-ran that test for the master branch and it works. can you please try rebasing this branch on master? or check the failures, if they are actually caused by a change in this branch?

dbu · 2016-07-01T12:02:18Z

oh, just noticed this branch is against the 1.3 version. we now have 1.4 and should target 1.4 instead. or indeed master as its a new feature and there will be no 1.5 but 2 hopefully soonish.

bdunogier · 2016-07-01T12:02:47Z

Has 1.3 reached end of life in between ? :)

1.3.x is affected by this issue, as far as I can say.

dbu · 2016-07-01T12:06:11Z

sorry, had lib and bundle open at the same time. the bundle is still at 1.3. so rebase on 1.3 should be the right thing.

bdunogier · 2016-07-01T12:18:11Z

Note that iirc, I've done that yesterday.

$ git l origin/1.3..
79bbdb454b96da7ae46e557baa8834863b2d2034 fixup! Do not generate hash for anonymous requests
e4dca8a55c90ba62af278e5c149ffa59258e3b8a fixup! Do not generate hash for anonymous requests
a2a675baafbd4e189a1df293e25da8ac4b848978 fixup! Do not generate hash for anonymous requests
53764a2a146b68c4fe577862b676f464c1d39014 Do not generate hash for anonymous requests

$ git l ..origin/1.3

bdunogier · 2016-07-01T12:28:47Z

@dbu is there anything I should do in particular to run the 1.3 tests ?

On both by branch and 1.3, after composer update --prefer-lowest:

$ SYMFONY_DEPRECATIONS_HELPER=weak ./vendor/bin/phpunit
... 
26) FOS\HttpCacheBundle\Tests\Functional\Test\ProxyTestCaseTest::testGetHttpClient
Class __PHP_Incomplete_Class has no unserializer

/Users/bdunogier/ezplatform/vendor/friendsofsymfony/http-cache-bundle/vendor/symfony/symfony/src/Symfony/Component/Config/ConfigCache.php:76
/Users/bdunogier/ezplatform/vendor/friendsofsymfony/http-cache-bundle/vendor/symfony/symfony/src/Symfony/Component/HttpKernel/Kernel.php:559
/Users/bdunogier/ezplatform/vendor/friendsofsymfony/http-cache-bundle/vendor/symfony/symfony/src/Symfony/Component/HttpKernel/Kernel.php:132
/Users/bdunogier/ezplatform/vendor/friendsofsymfony/http-cache-bundle/vendor/symfony/symfony/src/Symfony/Bundle/FrameworkBundle/Test/WebTestCase.php:43
/Users/bdunogier/ezplatform/vendor/friendsofsymfony/http-cache-bundle/Test/ProxyTestCase.php:139
/Users/bdunogier/ezplatform/vendor/friendsofsymfony/http-cache-bundle/Tests/Functional/Test/ProxyTestCaseTest.php:20

As far as I can tell, it can't find find \FOS\HttpCache\Test\Proxy\VarnishProxy, even though it is there. I'll keep digging.

dbu · 2016-07-01T13:30:06Z

maybe you need to remove the cache files our little test symfony app generates?

bdunogier · 2016-07-01T14:07:24Z

maybe you need to remove the cache files our little test symfony app generates?

HAH, cache, I knew it !

bdunogier · 2016-07-05T09:06:57Z

One error left:

PHP Warning:  Unexpected character in input:  '�' (ASCII=18) state=0 in /home/travis/build/FriendsOfSymfony/FOSHttpCacheBundle/vendor/symfony/symfony/src/Symfony/Bridge/Twig/Extension/CodeExtension.php on line 140

It seems to be fixed in Sf 2.3.4, but the failing test uses 2.3.0 (--prefer-lowest). My main doubt is that it apparently doesn't happen without this PR, but I hardly see how it can cause this.

Hints, anybody ? @stof maybe ?

lolautruche · 2016-07-05T09:39:43Z

+1 with the solution

Shouldn't there be some semantic config in order to be able to configure session prefix?

bdunogier · 2016-07-18T11:44:02Z

Poke @dbu. As far as I know, the review remarks have been addressed. Let me know if something is missing. If not, I'll squash the commits so that you can merge this :-)

dbu · 2016-07-21T07:15:55Z

UserContext/AnonymousRequestMatcher.php

+    {
+        if ($request->server->has('AUTHORIZATION') ||
+            $request->server->has('HTTP_AUTHORIZATION') ||
+            $request->server->has('PHP_AUTH_USER')


what if people use their custom authentication scheme?

i noticed that we have a configuration user_identifier_headers. maybe we should instead of looking for a session cookie or these headers just inject that list of headers and check for those headers.

the idea with cookies is that as we Vary anyways, the caching proxy has to clean up the cookie to not have random js cookie things in it. so we would not need to really go into the individual cookies.

does symfony itself look at HTTP_AUTHORIZATION and PHP_AUTH_USER? if so, we should change the default value of user_identifier_header in the config.

the idea with cookies is that as we Vary anyways, the caching proxy has to clean up the cookie to not have random js cookie things in it. so we would not need to really go into the individual cookies.

Meaning that a request is anonymous only if it doesn't have any of the headers from user_identifier_headers. The part that checks for custom sessions names should then be removed completely, right ?

Note that HTTP_AUTHORIZATION and PHP_AUTH_USER are server variables, not cookies. The cookie would, afaik, be Authorization in both cases.

The part that checks for custom sessions names should then be removed completely, right ?

yes, exactly. because we also Vary on those headers, if you send a cookie header with google analytics data in it, you break the cache anyways. this has to be handled on the caching proxy.

it looks like header based authentication is always put into the AUTHORIZATION header: https://github.com/symfony/symfony/blob/master/src/Symfony/Component/HttpFoundation/ServerBag.php#L94-L98 so cookie and authorization should be enough

@dbu so to double check here: in default vcl we strip every other cookie other then session. And from what you say this is done somewhere with the Sf proxy as well, so essentially here if there is any cookie, it is session cookie meaning we don't need to loop true them here. Correct?

that is what i think. i just double-checked and for HttpCache, this is in https://github.com/FriendsOfSymfony/FOSHttpCache/blob/master/src/SymfonyCache/UserContextSubscriber.php#L124-L137

for varnish, this is documented here: http://foshttpcache.readthedocs.io/en/latest/varnish-configuration.html#cleaning-the-cookie-header

caching the user context hash without cleaning the cookie header can not work.

dbu · 2016-07-21T07:19:48Z

sorry that i missed this. had a look again, and i think we are almost there. looking over the code one more time, i noticed that we have a inconsistency with what we use for the Vary header. can you look at that please?

i was thinking whether we need any documentation, but its probably too low level.

bdunogier · 2016-07-21T13:11:49Z

i was thinking whether we need any documentation, but its probably too low level.

Honestly glad you say that :-)
But if you think it is worth it, I can cut-my-own-throat and write something down.

i noticed that we have a inconsistency with what we use for the Vary header. can you look at that please?

Sure, I'll have a look, thank you.

bdunogier · 2016-07-22T07:28:05Z

i noticed that we have a inconsistency with what we use for the Vary header. can you look at that please?

@dbu that's about the user_identifier_headers right ? Or did you have something else in mind ?

dbu · 2016-07-22T09:03:36Z

yes, that was about the comment on the code we discuss. just wanted to summarize what remains open to do.

bdunogier · 2016-08-24T13:03:05Z

Changed the AnonymousRequestMatcher:

receives the user_identifier_headers
checks for the given headers (the part specific to cookies should maybe be changed, please let me know)

dbu · 2016-08-24T13:41:28Z

UserContext/AnonymousRequestMatcher.php

+{
+    private $userIdentifierHeaders;
+
+    public function __construct($userIdentifierHeaders = ['Cookie', 'Authorization'])


as this is in the symfony bundle and always created from the service container, i don't think we need to default this. the Configuration.php already defines defaults for user_identifier_headers.

and please typehint array for $userIdentifierHeaders and add a docblock that says that its a list of the header names of headers that make a request not be anonymous.

Ack. Note that UserContextSubscribed does set a default ctor value for them.

Removed the default altogether, and updated the unit test.
Rebased on top of 1.3, and cleaned the commits up.

dbu · 2016-08-24T15:00:42Z

apart from #289 (comment) this looks ready to merge. some build fail because composer has too many options and runs out of memory. not related to this PR

bdunogier · 2016-08-24T16:16:17Z

Changed. Let me know if it's okay, and I'll cleanup the commits a bit.

dbu · 2016-08-24T19:22:51Z

UserContext/AnonymousRequestMatcher.php

+    /**
+     * @param array $userIdentifierHeaders List of request headers that authenticate a non-anonymous request
+     */
+    public function __construct(array $userIdentifierHeaders = [])


my idea was to not specify a default at all, so that calling code is forced to pass an argument and not null. as it is now its is rather dangerous, if anything would go wrong we would simply consider everything anonymous.

dbu · 2016-08-24T19:24:43Z

apart from that 1 comment i am happy with this. please squash the commits as you see fit, and rebase on 1.3 to get the fixed testing setup i just merged.

Fixes the "Unexpected character in input" notice that happens on Travis, when running tests using the phar version.

dbu · 2016-08-25T06:47:04Z

UserContext/AnonymousRequestMatcher.php

+            if (strtolower($header) === 'cookie' && $request->cookies->count()) {
+                return false;
+            }
+            if ($request->headers->has($header)) {


sorry, one more question: what does symfony request do with the cookie header? is that removed from the header bag? what happens if we have a request with only an empty cookie header? won't we then get in here and see the cookie header, making the previous if on the cookies count kind of pointless? should we do

if ($request->headers->has($header)) { if (strtolower($header) === 'cookie' && 0 === $request->cookies->count()) { // ignore empty cookie header continue; } return false; }

Done, with test, @dbu.

the first if with cookie header and nonzero number of cookies is now redundant and should be removed.

Removed the if, adjusted the existing test, and added a new case.

bdunogier · 2016-08-25T07:41:50Z

By the way, once this is merged, do you think we'll be able to tag a new release shortly, @dbu ?

dbu · 2016-08-31T07:50:54Z

yay, thanks bertrand

dbu · 2016-08-31T08:32:08Z

ported to master as well, and tagged 1.3.8

blankse · 2018-01-10T21:50:00Z

I have a problem with this.

I use eZ Platform and a anonymous with cookies (e. g. from piwik) but without the session cookie doesn't use the http cache.

Why would only look after cookie count and not after the specific session cookie?

blankse · 2018-01-11T09:29:19Z

I created a PR for my issue: #421

bdunogier mentioned this pull request Mar 3, 2016

EZP-25533: allow newer versions of http-cache-bundle ezsystems/ezpublish-kernel#1601

Closed

2 tasks

dbu reviewed Mar 29, 2016
View reviewed changes

dbu mentioned this pull request Mar 29, 2016

check sanity of user context hash, if not, prevent setting any cache #274

Merged

bdunogier force-pushed the anonymous_request_hash-1.3 branch from e204ef7 to e4dca8a Compare April 29, 2016 12:15

dbu reviewed Apr 30, 2016
View reviewed changes

andrerom mentioned this pull request Jun 11, 2016

Users get sent to /_fos_user_context_hash upon login (ezpublish application) FriendsOfSymfony/FOSHttpCache#287

Closed

bdunogier force-pushed the anonymous_request_hash-1.3 branch 3 times, most recently from 3720c45 to 5157bde Compare July 4, 2016 10:22

dbu reviewed Jul 21, 2016
View reviewed changes

bdunogier force-pushed the anonymous_request_hash-1.3 branch 2 times, most recently from 17087b9 to ef71732 Compare August 24, 2016 13:00

dbu reviewed Aug 24, 2016
View reviewed changes

bdunogier mentioned this pull request Aug 24, 2016

EZP-25533: removed http-cache-bundle version restriction ezsystems/ezpublish-kernel#1758

Merged

dbu reviewed Aug 24, 2016
View reviewed changes

Bertrand Dunogier and others added 2 commits August 24, 2016 22:02

Do not generate hash for anonymous requests

cad6b5d

Require symfony/symfony:2.3.4

6492160

Fixes the "Unexpected character in input" notice that happens on Travis, when running tests using the phar version.

bdunogier force-pushed the anonymous_request_hash-1.3 branch from 2c97a57 to 6492160 Compare August 24, 2016 20:03

dbu reviewed Aug 25, 2016
View reviewed changes

Ignored empty cookie header in anonymous request matcher

fe38fb5

Bertrand Dunogier added 2 commits August 29, 2016 10:31

fixup! Ignored empty cookie header in anonymous request matcher

d19e9fa

fixup! Ignored empty cookie header in anonymous request matcher

0990981

dbu changed the title ~~Do not generate hash for anonymous requests~~ Do not sanity check hash of anonymous requests Aug 31, 2016

dbu merged commit 5ffcbf5 into FriendsOfSymfony:1.3 Aug 31, 2016

andrerom mentioned this pull request Oct 20, 2016

Remove context header for anonymous users FriendsOfSymfony/FOSHttpCache#182

Closed

Do not sanity check hash of anonymous requests #289

Do not sanity check hash of anonymous requests #289

Uh oh!

Conversation

bdunogier commented Mar 3, 2016 • edited by dbu Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TODO

Uh oh!

bdunogier commented Mar 3, 2016

Uh oh!

ddeboer commented Mar 6, 2016

Uh oh!

bdunogier commented Mar 9, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dbu commented Mar 29, 2016

Uh oh!

bdunogier commented Apr 29, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bdunogier commented Jul 1, 2016

Uh oh!

dbu commented Jul 1, 2016

Uh oh!

dbu commented Jul 1, 2016

Uh oh!

bdunogier commented Jul 1, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dbu commented Jul 1, 2016

Uh oh!

bdunogier commented Jul 1, 2016

Uh oh!

bdunogier commented Jul 1, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dbu commented Jul 1, 2016

Uh oh!

bdunogier commented Jul 1, 2016

Uh oh!

bdunogier commented Jul 5, 2016

Uh oh!

lolautruche commented Jul 5, 2016

Uh oh!

bdunogier commented Jul 18, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dbu commented Jul 21, 2016

Uh oh!

bdunogier commented Jul 21, 2016

Uh oh!

bdunogier commented Jul 22, 2016

Uh oh!

dbu commented Jul 22, 2016

Uh oh!

bdunogier commented Aug 24, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

bdunogier commented Mar 3, 2016 •

edited by dbu

Loading

bdunogier commented Jul 1, 2016 •

edited

Loading

bdunogier commented Jul 1, 2016 •

edited

Loading

bdunogier commented Jul 18, 2016 •

edited

Loading