Skip to content

Commit

Permalink
chore: Add Souin as available provider to the existing system
Browse files Browse the repository at this point in the history
chore: Rebase on main

chore: Add the configuration

chore: Update

WIP

Update configuration

add default value for purger configuration declaration

Update ApiPlatformExtension.php

Bump tests

Move files

Fix CI

Fix tests

Update AddTagsListenerTest

Update the tests based on the Cache-Tag header

Add SouinPurgerTests

Fix cs

Add tests

Fix cs

Fix cs

Add more tests

Fix cs

Fix

Remove useless getters

Throw valid exception

Rewording parameters

Fix cs
  • Loading branch information
darkweak committed Nov 18, 2021
1 parent 10827bb commit 7627f07
Show file tree
Hide file tree
Showing 10 changed files with 619 additions and 39 deletions.
21 changes: 10 additions & 11 deletions features/http_cache/tags.feature
Expand Up @@ -17,15 +17,15 @@ Feature: Cache invalidation through HTTP Cache tags
}
"""
Then the response status code should be 201
And the header "Cache-Tags" should not exist
And the header "Cache-Tag" should not exist
And the header "xkey" should not exist
And "/relation_embedders,/related_dummies,/third_levels" IRIs should be purged
And "/relation_embedders /related_dummies /third_levels" IRIs should be purged with xkey

Scenario: Tags must be set for items
When I send a "GET" request to "/relation_embedders/1"
Then the response status code should be 200
And the header "Cache-Tags" should be equal to "/relation_embedders/1,/related_dummies/1,/third_levels/1"
And the header "Cache-Tag" should be equal to "/relation_embedders/1,/related_dummies/1,/third_levels/1"
And the header "xkey" should be equal to "/relation_embedders/1 /related_dummies/1 /third_levels/1"

Scenario: Create some more resources
Expand All @@ -40,13 +40,13 @@ Feature: Cache invalidation through HTTP Cache tags
}
"""
Then the response status code should be 201
And the header "Cache-Tags" should not exist
And the header "Cache-Tag" should not exist
And the header "xkey" should not exist

Scenario: Tags must be set for collections
When I send a "GET" request to "/relation_embedders"
Then the response status code should be 200
And the header "Cache-Tags" should be equal to "/relation_embedders/1,/related_dummies/1,/third_levels/1,/relation_embedders/2,/related_dummies/2,/third_levels/2,/relation_embedders"
And the header "Cache-Tag" should be equal to "/relation_embedders/1,/related_dummies/1,/third_levels/1,/relation_embedders/2,/related_dummies/2,/third_levels/2,/relation_embedders"
And the header "xkey" should be equal to "/relation_embedders/1 /related_dummies/1 /third_levels/1 /relation_embedders/2 /related_dummies/2 /third_levels/2 /relation_embedders"

Scenario: Purge item on update
Expand All @@ -58,7 +58,7 @@ Feature: Cache invalidation through HTTP Cache tags
}
"""
Then the response status code should be 200
And the header "Cache-Tags" should not exist
And the header "Cache-Tag" should not exist
And the header "xkey" should not exist
And "/relation_embedders,/relation_embedders/1,/related_dummies/1" IRIs should be purged
And "/relation_embedders /relation_embedders/1 /related_dummies/1" IRIs should be purged with xkey
Expand All @@ -67,7 +67,7 @@ Feature: Cache invalidation through HTTP Cache tags
When I add "Content-Type" header equal to "application/ld+json"
And I send a "DELETE" request to "/relation_embedders/1"
Then the response status code should be 204
And the header "Cache-Tags" should not exist
And the header "Cache-Tag" should not exist
And the header "xkey" should not exist
And "/relation_embedders,/relation_embedders/1,/related_dummies/1" IRIs should be purged
And "/relation_embedders /relation_embedders/1 /related_dummies/1" IRIs should be purged with xkey
Expand All @@ -89,7 +89,7 @@ Feature: Cache invalidation through HTTP Cache tags

Scenario: Embedded collection must be listed in cache tags
When I send a "GET" request to "/relation2s/1"
Then the header "Cache-Tags" should be equal to "/relation2s/1"
Then the header "Cache-Tag" should be equal to "/relation2s/1"
Then the header "xkey" should be equal to "/relation2s/1"

Scenario: Create a Relation1
Expand Down Expand Up @@ -132,7 +132,7 @@ Feature: Cache invalidation through HTTP Cache tags
When I add "Content-Type" header equal to "application/ld+json"
And I send a "GET" request to "/relation3s"
Then the response status code should be 200
And the header "Cache-Tags" should be equal to "/relation3s/1,/relation2s/1,/relation2s/2,/relation3s"
And the header "Cache-Tag" should be equal to "/relation3s/1,/relation2s/1,/relation2s/2,/relation3s"
And the header "xkey" should be equal to "/relation3s/1 /relation2s/1 /relation2s/2 /relation3s"

Scenario: Update a collection member only
Expand All @@ -144,7 +144,7 @@ Feature: Cache invalidation through HTTP Cache tags
}
"""
Then the response status code should be 200
And the header "Cache-Tags" should not exist
And the header "Cache-Tag" should not exist
And the header "xkey" should not exist
And "/relation3s,/relation3s/1,/relation2s/2,/relation2s,/relation2s/1" IRIs should be purged
And "/relation3s /relation3s/1 /relation2s/2 /relation2s /relation2s/1" IRIs should be purged with xkey
Expand All @@ -153,8 +153,7 @@ Feature: Cache invalidation through HTTP Cache tags
When I add "Content-Type" header equal to "application/ld+json"
And I send a "DELETE" request to "/relation3s/1"
Then the response status code should be 204
And the header "Cache-Tags" should not exist
And the header "Cache-Tag" should not exist
And the header "xkey" should not exist
And "/relation3s,/relation3s/1,/relation2s/2" IRIs should be purged
And "/relation3s /relation3s/1 /relation2s/2" IRIs should be purged with xkey

16 changes: 10 additions & 6 deletions src/HttpCache/EventListener/AddTagsListener.php
Expand Up @@ -22,11 +22,11 @@
use Symfony\Component\HttpKernel\Event\ResponseEvent;

/**
* Sets the list of resources' IRIs included in this response in the "Cache-Tags" and/or "xkey" HTTP headers.
* Sets the list of resources' IRIs included in this response in the configured cache tag HTTP header and/or "xkey" HTTP headers.
*
* The "Cache-Tags" is used because it is supported by CloudFlare.
* By default the "Cache-Tag" HTTP header is used because it is supported by CloudFlare.
*
* @see https://support.cloudflare.com/hc/en-us/articles/206596608-How-to-Purge-Cache-Using-Cache-Tags-Enterprise-only-
* @see https://developers.cloudflare.com/cache/how-to/purge-cache#add-cache-tag-http-response-headers
*
* The "xkey" is used because it is supported by Varnish.
* @see https://docs.varnish-software.com/varnish-cache-plus/vmods/ykey/
Expand All @@ -44,18 +44,22 @@ final class AddTagsListener
private $xkeyEnabled;
private $xkeyGlue;
private $httpTagsEnabled;
private $header;
private $separator;

public function __construct(IriConverterInterface $iriConverter, ResourceMetadataCollectionFactoryInterface $resourceMetadataCollectionFactory = null, bool $xkeyEnabled = false, string $xkeyGlue = ' ', bool $httpTagsEnabled = true)
public function __construct(IriConverterInterface $iriConverter, ResourceMetadataCollectionFactoryInterface $resourceMetadataCollectionFactory = null, bool $xkeyEnabled = false, string $xkeyGlue = ' ', bool $httpTagsEnabled = true, string $header = 'Cache-Tag', string $separator = ',')
{
$this->iriConverter = $iriConverter;
$this->resourceMetadataCollectionFactory = $resourceMetadataCollectionFactory;
$this->xkeyEnabled = $xkeyEnabled;
$this->xkeyGlue = $xkeyGlue;
$this->httpTagsEnabled = $httpTagsEnabled;
$this->header = $header;
$this->separator = $separator;
}

/**
* Adds the "Cache-Tags" and "xkey" headers.
* Adds the configured HTTP cache tag and "xkey" headers.
*/
public function onKernelResponse(ResponseEvent $event): void
{
Expand Down Expand Up @@ -84,7 +88,7 @@ public function onKernelResponse(ResponseEvent $event): void
}

if ($this->httpTagsEnabled) {
$response->headers->set('Cache-Tags', implode(',', $resources));
$response->headers->set($this->header, implode($this->separator, $resources));
}

if ($this->xkeyEnabled) {
Expand Down
127 changes: 127 additions & 0 deletions src/HttpCache/SouinPurger.php
@@ -0,0 +1,127 @@
<?php

/*
* This file is part of the API Platform project.
*
* (c) Kévin Dunglas <dunglas@gmail.com>
*
* For the full copyright and license information, please view the LICENSE
* file that was distributed with this source code.
*/

declare(strict_types=1);

namespace ApiPlatform\HttpCache;

use GuzzleHttp\ClientInterface;
use GuzzleHttp\Exception\GuzzleException;
use Psr\Log\LoggerInterface;
use Symfony\Component\HttpFoundation\Request;

/**
* Purges Souin.
*
* @author Sylvain Combraque <darkweak@protonmail.com>
*
* @experimental
*/
class SouinPurger implements PurgerInterface
{
private const MAX_HEADER_SIZE_PER_BATCH = 1500;
private const SEPARATOR = ', ';
private const SOUIN_COOKIE_NAME = 'souin-authorization-token';

private $logger;

// Clients to send cache invalidation
private $clients;

// Souin relative paths
private $dsn;
private $path;

// Souin token if mandatory
private $token;

/**
* @param ClientInterface[] $clients
*/
public function __construct(
array $clients,
LoggerInterface $logger,
string $dsn,
string $path,
string $token
) {
$this->clients = $clients;
$this->logger = $logger;
$this->dsn = $dsn;
$this->path = $path;
$this->token = $token;
}

private function getParametersFromIris(array $iris): string
{
return implode(self::SEPARATOR, $iris);
}

/**
* @param array|string[] $iris
*
* @return string[]
*/
private function getChunkedRegex(array $iris): array
{
$regex = $this->getParametersFromIris($iris);
$batches = [];
$separatorLength = \strlen(self::SEPARATOR);

while (\strlen($regex) > self::MAX_HEADER_SIZE_PER_BATCH) {
$splitPosition = strrpos(str_split($regex, self::MAX_HEADER_SIZE_PER_BATCH)[0], self::SEPARATOR);
if ($splitPosition) {
[$batches[], $regex] = str_split($regex, $splitPosition);
$regex = substr($regex, $separatorLength);
}
}

$batches[] = $regex;

return $batches;
}

private function banRegex(string $regex): void
{
foreach ($this->clients as $client) {
try {
$client->request(
Request::METHOD_PURGE,
$this->dsn.$this->path,
['headers' => array_merge(
[
'Surrogate-Key' => $regex,
],
$this->token ?
['Cookie' => sprintf('%s=%s', self::SOUIN_COOKIE_NAME, $this->token)] :
[]
)]
);
} catch (GuzzleException $e) {
$this->logger->warning($e->getMessage());
}
}
}

/**
* {@inheritdoc}
*/
public function purge(array $iris)
{
if (!$iris) {
return;
}

foreach ($this->getChunkedRegex($iris) as $chunkedRegex) {
$this->banRegex($chunkedRegex);
}
}
}
37 changes: 28 additions & 9 deletions src/Symfony/Bundle/DependencyInjection/ApiPlatformExtension.php
Expand Up @@ -235,6 +235,12 @@ private function registerCommonConfiguration(ContainerBuilder $container, array
$container->setParameter('api_platform.http_cache.invalidation.xkey.enabled', $config['defaults']['cache_headers']['invalidation']['xkey_enabled'] ?? $this->isConfigEnabled($container, $config['http_cache']['invalidation']['xkey']));
$container->setParameter('api_platform.http_cache.invalidation.xkey.glue', $config['defaults']['cache_headers']['invalidation']['xkey']['glue'] ?? $config['http_cache']['invalidation']['xkey']['glue']);
$container->setParameter('api_platform.http_cache.invalidation.http_tags.enabled', $config['defaults']['cache_headers']['invalidation']['http_tags_enabled'] ?? $this->isConfigEnabled($container, $config['http_cache']['invalidation']['http_tags']));
$container->setParameter('api_platform.http_cache.validation.header', $config['http_cache']['validation']['header']);
$container->setParameter('api_platform.http_cache.validation.separator', $config['http_cache']['validation']['separator']);
$container->setParameter('api_platform.http_cache.invalidation.type', $config['http_cache']['invalidation']['type']);
$container->setParameter('api_platform.http_cache.invalidation.purger.dsn', $config['http_cache']['invalidation']['purger']['dsn']);
$container->setParameter('api_platform.http_cache.invalidation.purger.path', $config['http_cache']['invalidation']['purger']['path']);
$container->setParameter('api_platform.http_cache.invalidation.purger.token', $config['http_cache']['invalidation']['purger']['token']);

$container->setAlias('api_platform.operation_path_resolver.default', $config['default_operation_path_resolver']);
$container->setAlias('api_platform.path_segment_name_generator', $config['path_segment_name_generator']);
Expand Down Expand Up @@ -626,16 +632,29 @@ private function registerHttpCacheConfiguration(ContainerBuilder $container, arr
$definitions[] = $definition;
}

if ($this->isConfigEnabled($container, $config['http_cache']['invalidation']['http_tags'])) {
$container->getDefinition('api_platform.http_cache.purger.varnish')->setArguments([$definitions,
$config['http_cache']['invalidation']['max_header_length'], ]);
$container->setAlias('api_platform.http_cache.purger', 'api_platform.http_cache.purger.varnish');
}
switch ($config['http_cache']['invalidation']['type']) {
case 'souin':
$container->getDefinition('api_platform.http_cache.purger.souin')->setArguments([
$definitions,
$config['http_cache']['invalidation']['purger']['dsn'],
$config['http_cache']['invalidation']['purger']['path'],
$config['http_cache']['invalidation']['purger']['token'],
]);
$container->setAlias('api_platform.http_cache.purger', 'api_platform.http_cache.purger.souin');
break;
default:
if ($this->isConfigEnabled($container, $config['http_cache']['invalidation']['http_tags'])) {
$container->getDefinition('api_platform.http_cache.purger.varnish')->setArguments([$definitions,
$config['http_cache']['invalidation']['max_header_length'], ]);
$container->setAlias('api_platform.http_cache.purger', 'api_platform.http_cache.purger.varnish');
}

if ($this->isConfigEnabled($container, $config['http_cache']['invalidation']['xkey'])) {
$container->getDefinition('api_platform.http_cache.purger.varnish.xkey')->setArguments([$definitions,
$config['http_cache']['invalidation']['max_header_length'], ]);
$container->setAlias('api_platform.http_cache.purger.xkey', 'api_platform.http_cache.purger.varnish.xkey');
if ($this->isConfigEnabled($container, $config['http_cache']['invalidation']['xkey'])) {
$container->getDefinition('api_platform.http_cache.purger.varnish.xkey')->setArguments([$definitions,
$config['http_cache']['invalidation']['max_header_length'], ]);
$container->setAlias('api_platform.http_cache.purger.xkey', 'api_platform.http_cache.purger.varnish.xkey');
}
break;
}
}

Expand Down
38 changes: 37 additions & 1 deletion src/Symfony/Bundle/DependencyInjection/Configuration.php
Expand Up @@ -424,7 +424,43 @@ private function addHttpCacheSection(ArrayNodeDefinition $rootNode): void
->end()
->arrayNode('http_tags')
->canBeDisabled()
->info('Enable support for Cache-Tags invalidation.')
->info('Enable support for Cache-Tag invalidation.')
->end()
->arrayNode('purger')
->addDefaultsIfNotSet()
->info('Define the purger configuration.')
->children()
->scalarNode('dsn')
->defaultValue('')
->info('Purger dsn')
->end()
->scalarNode('path')
->defaultValue('')
->info('Purger API path')
->end()
->scalarNode('token')
->defaultValue('')
->info('JWT token to use if the provider API if secured')
->end()
->end()
->end()
->scalarNode('type')
->defaultValue('')
->info('Cache provider to use (e.g. souin, varnish, whatever)')
->end()
->end()
->end()
->arrayNode('validation')
->addDefaultsIfNotSet()
->info('Enable the tags-based cache validation system.')
->children()
->scalarNode('header')
->defaultValue('Cache-Tag')
->info('HTTP header to use to return the associated tags')
->end()
->scalarNode('separator')
->defaultValue(',')
->info('The separator for the tags')
->end()
->end()
->end()
Expand Down
4 changes: 4 additions & 0 deletions src/Symfony/Bundle/Resources/config/http_cache_tags.xml
Expand Up @@ -15,7 +15,11 @@
<argument>%api_platform.http_cache.invalidation.xkey.enabled%</argument>
<argument>%api_platform.http_cache.invalidation.xkey.glue%</argument>
<argument>%api_platform.http_cache.invalidation.http_tags.enabled%</argument>
<argument>%api_platform.http_cache.validation.header%</argument>
<argument>%api_platform.http_cache.validation.separator%</argument>
<tag name="kernel.event_listener" event="kernel.response" method="onKernelResponse" priority="-2" />
</service>

<service id="api_platform.http_cache.purger.souin" class="ApiPlatform\HttpCache\SouinPurger" public="false" />
</services>
</container>

0 comments on commit 7627f07

Please sign in to comment.