Since symfony/security-csrf 5.3: Using SessionTokenStorage without a session has no effect and is deprecated

[arderyp]

Symfony version(s) affected

5.4.x

Description

This issue is explained by someone else here: https://stackoverflow.com/questions/70669443/symfony-deprecation-on-sessiontokenstorage-when-generating-a-csrf-token-in-phpun

I am running into it as well. I realize I could get around it by first requesting the page with the form and using the token from the form, or just using the submit() approach, but this would double the amount of crawler requests my test suite makes, which would slow it way down. Is there any other way around this? I have an abstract POST requester tests that simply posts raw data arrays to my various POST controller actions. I don't visit the form page first and submit it through the crawler, because my approach is much, much faster. But since moving from 4.4 to 5.4, I am now seeing this warning and am somewhat concerned.

Any thoughts are greatly appreciated.

How to reproduce

https://stackoverflow.com/questions/70669443/symfony-deprecation-on-sessiontokenstorage-when-generating-a-csrf-token-in-phpun

Possible Solution

either don't deprecate this, or create some workaround for testing environment.

Additional Context

No response

[arderyp]

the answer in the SO post actually seems to work in terms of suppressing the deprecation warning, but it causes the form POST submission to throw a "CSRF Token Invalid" error.

[arderyp]

the other weird bit is that I am getting the token manager after KernelBrowser::loginUser(), which appears to me to be initiating a session: https://github.com/symfony/symfony/blob/5.4/src/Symfony/Bundle/FrameworkBundle/KernelBrowser.php#L145

So, if I'm logging in, thereby creating a session, why would the token manager complain that there is no session?

[arderyp]

Thanks for converting this to a discussion @derrabus. I am not sure if this is an actual bug or not, but the more I poke into it, the more I think it might actually be, or that I might be misunderstanding something.

$this->client->loginUser($authUser, $firewall);
$this->client->getContainer()->get('security.csrf.token_manager')->getToken($tokenId);

Since loginUser() initialized a session and actually set's cookie on it here, should we expect there to be no session in the stack within the TokenManager? Even this does not work, which must initiate a session in the client stack, right?

$this->client->loginUser($authUser, $firewall);
$this->client->request("GET", "/");
$this->client->getContainer()->get('security.csrf.token_manager')->getToken($tokenId);

If this is confusing, or unbelievable, I can try to reproduce in a test repo if that would be helpful.

My testing thusfar seems like one can simply never retrieve the TokenManager in a functional test, as it will never recognize the client session. Might I be misunderstanding something? Or is it intended that the TokenManager should never be retreivable/used in functional tests?

[arderyp]

One option would be to disable csrf in tests, but this feels like a last resort option to me. I'd like my functional.integration tests to be as close to the real app behavior as possible.

[arderyp]

As I continue researching this, if the behavior is expected, it seems to me that functional test implementers have two options:

1. crawl to the form page first, and use the crawler to submit the form (the token will be generated server side and included in the form)
2. disable CSRF in the test environment to submit POST/PATCH/PUT/DELETE requests without crawling to the form page first

The primary assumption being:

• Symfony 6+ does not allow using the CSRF TokenManager/TokenStorage in functional tests

I'd love to hear some symfony's dev's chime in to see if I am understanding this correctly. If this is correct, I think it might be worth updating the documentation and deprecation notes to make this explicitly clear.

[nicodemuz]

I'd love to hear some symfony's dev's chime in to see if I am understanding this correctly. If this is correct, I think it might be worth updating the documentation and deprecation notes to make this explicitly clear.

I'd like to also know what is considered as the best practice when writing automation tests. Should CSRF be disabled in automation tests? Or is there some better way to handle this. @arderyp did you get an answer for your question? @fabpot

[arderyp]

@nicodemuz no I have not, unfortunately

[stof]

If your test tries to submit a POST request without crawling the page to get the form to submit with its CSRF token first, what your test was doing previously is actually achieving a CSRF attack.

[arderyp]

@stof by that logic, nearly all integration test and unit tests are "attacks"

[arderyp]

I've figured it out. Right now, the solution requires replicating and enhancing the loginUser() function on KernelBrowser. I will submit a PR to symfony to make this achievable without needing to do so. Who knows if they will accept it. For now:

use Symfony\Bundle\FrameworkBundle\KernelBrowser;
use Symfony\Bundle\FrameworkBundle\Test\WebTestCase;
use Symfony\Component\BrowserKit\Cookie;
use Symfony\Component\HttpFoundation\Session\SessionInterface;
use Symfony\Component\Security\Core\Authentication\Token\UsernamePasswordToken;
use Symfony\Component\Security\Core\User\UserInterface;

abstract class AbstractWebTestCase extends WebTestCase
{
    private KernelBrowser $client;
    private SessionInterface $session;
    
    public function setUp(): void
    {
        parent::setUp();
        $this->client = static::createClient();
        ...
    }

    ...

    /**
     * This replicates static::createClient()->loginUser()
     * Inspect that method, as there are additional checks there that may be necessary for your use case.
     * The magic here is tracking an internal $session object that can be updated as needed.
     */
    protected function loginUser(UserInterface $user): void
    {	
        $token = new TestBrowserToken($user->getRoles(), $user, $firewallContext);
        $container = static::getContainer();
        $container->get('security.untracked_token_storage')->setToken($token);

	$this->session = $container->get('session.factory')->createSession();
        $this->setLoginSessionValue('_security_'.$firewallContext, serialize($token));

        $domains = array_unique(array_map(function (Cookie $cookie) {
            return $cookie->getName() === $this->session->getName() ? $cookie->getDomain() : '';
        }, $this->client->getCookieJar()->all())) ?: [''];
        
        foreach ($domains as $domain) {
            $cookie = new Cookie($this->session->getName(), $this->session->getId(), null, null, $domain);
            $this->client->getCookieJar()->set($cookie);
        }

        return $this;
    }

    /** @param mixed $value */
    protected function setLoginSessionValue(string $name, $value): self
    {
        if (isset($this->session)) {
            $this->session->set($name, $value);
            $this->session->save();
            return $this;
        }
        throw new \LogicException("loginUser() must be called to initialize session");
    }

    ...
}

use Symfony\Component\Security\Csrf\TokenStorage\SessionTokenStorage;

class MyWebTest extends AbstractWebTestCase
{
    public function testSomething(): void
    {
        $user = ...;
        $this->loginUser($user);
        
        // Technically, you don't need to generate a real token here, and instead could use any test string
        $tokenId = ...;
        $csrfToken = static::getContainer()->get('security.csrf.token_generator')->generateToken();
        $this->setLoginSessionValue(SessionTokenStorage::SESSION_NAMESPACE . "/$tokenId", $csrfToken);

        // Now you can make raw POST requests without crawling to the form page first!
    }
}

[arderyp]

PR: #47001

[marien-probesys]

Hello @arderyp. Following my answer to #45662 (comment), I've found a way to get a session from a client and set the CSRF token in it. This allows two things:

1. not rewriting the whole loginUser method 😁
2. to test controllers which don't need a connected user but still protected by a CSRF token

I've extracted my code in a trait to be used in all my tests.

<?php

namespace App\Tests;

use Symfony\Bundle\FrameworkBundle\KernelBrowser;
use Symfony\Component\BrowserKit\Cookie;
use Symfony\Component\HttpFoundation\Session\Session;
use Symfony\Component\HttpFoundation\Session\Storage\MockFileSessionStorage;
use Symfony\Component\Security\Csrf\TokenGenerator\TokenGeneratorInterface;
use Symfony\Component\Security\Csrf\TokenStorage\SessionTokenStorage;

trait SessionHelper
{
    public function getSession(KernelBrowser $client): Session
    {
        $cookie = $client->getCookieJar()->get('MOCKSESSID');

        // create a new session object
        $container = static::getContainer();
        $session = $container->get('session.factory')->createSession();

        if ($cookie) {
            // get the session id from the session cookie if it exists
            $session->setId($cookie->getValue());
            $session->start();
        } else {
            // or create a new session id and a session cookie
            $session->start();
            $session->save();

            $sessionCookie = new Cookie(
                $session->getName(),
                $session->getId(),
                null,
                null,
                'localhost',
            );
            $client->getCookieJar()->set($sessionCookie);
        }

        return $session;
    }

    public function generateCsrfToken(KernelBrowser $client, string $tokenId): string
    {
        $session = $this->getSession($client);
        $container = static::getContainer();
        $tokenGenerator = $container->get('security.csrf.token_generator');
        $csrfToken = $tokenGenerator->generateToken();
        $session->set(SessionTokenStorage::SESSION_NAMESPACE . "/{$tokenId}", $csrfToken);
        $session->save();
        return $csrfToken;
    }
}

It can be used this way:

<?php

namespace App\Tests\Controller;

use App\Tests\SessionHelper;
use Symfony\Bundle\FrameworkBundle\Test\WebTestCase;

class SessionControllerTest extends WebTestCase
{
    use SessionHelper;

    public function testSomething(): void
    {
        $client = static::createClient();

        $client->request('POST', '/something', [
            '_csrf_token' => $this->generateCsrfToken($client, 'expected token id'),
        ]);

        // assert something
    }
}

I hope it helps!

[dreis2211]

@marien-probesys I've tried your solution on Symfony 5.4 and unfortunately can't get it to work. I'm surely missing something. But just generating the token via security.csrf.token_generator won't set it in the storage inside security.csrf.token_manager, which is why ultimately things are failing for me in CsrfValidationListener::preSubmit as the token is not found. What am I missing?

[arderyp]

@dreis2211 can you post your code? It sounds like you're trying to use the services, which have been deprecated, which is what this issue is about...

[marien-probesys]

@dreis2211 This trait not only generates the token, but it also creates a session based on the cookie MOCKSESSID (see the method getSession()) and stores the token in the session (see the line $session->set([…])).

However, I'm using a solution slightly different, so maybe there's an error in the solution I've posted above? It's basically the same code, I've just extracted a createSession() method. You can take a look at my current implementation here: https://github.com/Probesys/bileto/blob/main/tests/SessionHelper.php and see how I'm using it here: https://github.com/Probesys/bileto/blob/main/tests/Controller/TeamsControllerTest.php#L99-L104 (note that it requires to use the trait, see the line use SessionHelper; at the top of the class).

[dreis2211]

@arderyp & @marien-probesys Thanks for reaching out. Unfortunately, I can't share the code. If time allows, I'll try to craft a reproducer though.

@marien-probesys I tried your approach, but couldn't get it to work either. You're also on Symfony 6.4 already, where things might be different.

As I'm currently in the process of upgrading to Symfony 6.4, I'll revisit this once we're on 6.4 maybe.

[dreis2211]

@marien-probesys I found the missing piece. The test firewall was not called main and thus the firewall context had a mismatch, which is why neither solution worked. Big thanks for the trimmed down approach. Worked like a charm after I fixed the firewall name.