Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TASK] Migrate cHash handling #2725

Closed
DanielSiepmann opened this issue Sep 14, 2020 · 3 comments · Fixed by #2972
Closed

[TASK] Migrate cHash handling #2725

DanielSiepmann opened this issue Sep 14, 2020 · 3 comments · Fixed by #2972
Assignees

Comments

@DanielSiepmann
Copy link
Contributor

cHash handling was disabled with some code as it wasn't compatible, see #2211.
The mentioned "issue" or missing feature inside of TYPO3 is fixed since v10 LTS: https://review.typo3.org/c/Packages/TYPO3.CMS/+/63062.

Therefore this workaround should be removed, and a proper example configuration should be provided instead.

Right now we apply the following patch to EXT:solr:

diff --git a/Classes/Domain/Search/Uri/SearchUriBuilder.php b/Classes/Domain/Search/Uri/SearchUriBuilder.php
index 81eb4321..7fc5246a 100644
--- a/Classes/Domain/Search/Uri/SearchUriBuilder.php
+++ b/Classes/Domain/Search/Uri/SearchUriBuilder.php
@@ -310,36 +310,22 @@ class SearchUriBuilder
      */
     protected function buildLinkWithInMemoryCache($pageUid, array $arguments)
     {
-        $values = [];
-        $structure = $arguments;
-        $this->getSubstitution($structure, $values);
-        $hash = md5($pageUid . json_encode($structure));
+        $hash = md5($pageUid . json_encode($arguments));
         if (isset(self::$preCompiledLinks[$hash])) {
             self::$hitCount++;
             $uriCacheTemplate = self::$preCompiledLinks[$hash];
         } else {
             self::$missCount++;
-            $this->uriBuilder->setTargetPageUid($pageUid);
-            $uriCacheTemplate = $this->uriBuilder->setArguments($structure)->setUseCacheHash(false)->build();
-
-            // even if we call build with disabled cHash in TYPO3 9 a cHash will be generated when site management is active
-            // to prevent wrong cHashes we remove the cHash here from the cached uri template.
-            // @todo: This can be removed when https://forge.typo3.org/issues/87120 is resolved and we can ship a proper configuration
-            $urlHelper = GeneralUtility::makeInstance(UrlHelper::class, $uriCacheTemplate);
-            $urlHelper->removeQueryParameter('cHash');
-            $uriCacheTemplate = $urlHelper->getUrl();
+            $uriCacheTemplate = $this->uriBuilder
+                ->setTargetPageUid($pageUid)
+                ->setArguments($arguments)
+                ->setUseCacheHash(true)
+                ->build();
 
             self::$preCompiledLinks[$hash] = $uriCacheTemplate;
         }
 
-        $keys = array_map(function($value) {
-            return urlencode($value);
-        }, array_keys($values));
-        $values = array_map(function($value) {
-            return urlencode($value);
-        }, $values);
-        $uri = str_replace($keys, $values, $uriCacheTemplate);
-        return $uri;
+        return $uriCacheTemplate;
     }
 
     /**

Also getSubstitution() could be removed.

Possible example configuration:

\TYPO3\CMS\Core\Utility\ArrayUtility::mergeRecursiveWithOverrule($GLOBALS['TYPO3_CONF_VARS'], [
    'FE' => [
        'cacheHash' => [
            'excludedParameters' => [
                'solr' => '^tx_solr',
            ],
        ],
    ],
]);
@dkd-kaehm
Copy link
Collaborator

@3l73 @dkd-friedrich
Current master state on 5c27dd7
assigns the cHash on all EXT:Solr links/facets/components.
There is the time to go the way proposed in this issue.

@dkd-kaehm dkd-kaehm added this to the 11.1 - next major release milestone Jun 7, 2021
dkd-friedrich added a commit to dkd-friedrich/ext-solr that referenced this issue Jun 28, 2021
cHash handling ws adapted/disabled to fix generated links in TYPO3 9,
as this adaptions lead to invalid links, e.g. facet links, in TYPO3 10
and Site configuration support is available since EXT:solr 10 there is
no need for this adaptions.

This commit removes the implemented removal of the cHash parameter,
now the UriBuilder is used to build the link template.

Resolves: TYPO3-Solr#2725
@dkd-friedrich
Copy link
Member

I've prepared a PR (#2972) to fix this issue, changes are based on the patch kindly provided by @DanielSiepmann. I just added a reset() to the UriBuilder to avoid invalid settings and skipped the call of setUseCacheHash() since this has no effect anymore and will just trigger a notice.

I will do some more tests in the next days and check the integration tests if necessary.

dkd-friedrich added a commit to dkd-friedrich/ext-solr that referenced this issue Jul 7, 2021
cHash handling ws adapted/disabled to fix generated links in TYPO3 9,
as this adaptions lead to invalid links, e.g. facet links, in TYPO3 10
and Site configuration support is available since EXT:solr 10 there is
no need for this adaptions.

This commit removes the implemented removal of the cHash parameter,
now the UriBuilder is used to build the link template.

Resolves: TYPO3-Solr#2725
dkd-friedrich added a commit to dkd-friedrich/ext-solr that referenced this issue Jul 7, 2021
cHash handling ws adapted/disabled to fix generated links in TYPO3 9,
as this adaptions lead to invalid links, e.g. facet links, in TYPO3 10
and Site configuration support is available since EXT:solr 10 there is
no need for this adaptions.

This commit removes the implemented removal of the cHash parameter,
now the UriBuilder is used to build the link template.

Resolves: TYPO3-Solr#2725
dkd-friedrich added a commit to dkd-friedrich/ext-solr that referenced this issue Jul 7, 2021
cHash handling ws adapted/disabled to fix generated links in TYPO3 9,
as this adaptions lead to invalid links, e.g. facet links, in TYPO3 10
and Site configuration support is available since EXT:solr 10 there is
no need for this adaptions.

This commit removes the implemented removal of the cHash parameter,
now the UriBuilder is used to build the link template.

Resolves: TYPO3-Solr#2725
dkd-kaehm added a commit to dkd-friedrich/ext-solr that referenced this issue Jul 8, 2021
EXT:solrs components like range facets can not be properly handled by cacheHash stack, because the amount of possible range-combinations is infinite, therefore they must be excluded from cacheHash calculation.

This change makes it possible to exclude all EXT:solr parameters from cacheHash.
To prevent misconfigurations, the new extension configuration setting "pluginNamespaces" was introduced, which is used in flex form and in 
TYPO3_CONF_VARS/FE/cacheHash/excludedParameters. 
This setting makes it impossible to chose invalid/unhandled EXT:solr plugin namespace on flex form(Plugin -> Options -> Plugin Namespace)

Migration:

Needed only if other as default(tx_solr) plugin namespace is used in instance.

Add the used namespace[s] to 
`$GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['solr']['pluginNamespaces']` 
or via backend "Settings" -> "Extension Configuration" -> "solr" -> "A list of white listed plugin namespaces" 

Fixes: TYPO3-Solr#2725
dkd-kaehm added a commit to dkd-friedrich/ext-solr that referenced this issue Jul 9, 2021
EXT:solrs components like range facets can not be properly handled by cacheHash stack, because the amount of possible range-combinations is infinite, therefore they must be excluded from cacheHash calculation.

This change makes it possible to exclude all EXT:solr parameters from cacheHash.
To prevent misconfigurations, the new extension configuration setting "pluginNamespaces" was introduced, which is used in flex form and in 
TYPO3_CONF_VARS/FE/cacheHash/excludedParameters. 
This setting makes it impossible to chose invalid/unhandled EXT:solr plugin namespace on flex form(Plugin -> Options -> Plugin Namespace)

## Migrations:

### Plugin namespaces:

Needed only if other as default(tx_solr) plugin namespace is used in instance.

Add the used namespace[s] to 
`$GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['solr']['pluginNamespaces']` 
or via backend "Settings" -> "Extension Configuration" -> "solr" -> "A list of white listed plugin namespaces" 

### Global q parameter

Needed only if global "q" parameter without plugin namespace is used and wants to be included in cacheHash calculation.
Set the setting `$GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['solr']['pluginNamespaces'] = '1'` 
or enable it via backend "Settings" -> "Extension Configuration" -> "solr" -> "Include/Exclude global q parameter in/from cacheHash" 



Fixes: TYPO3-Solr#2725
dkd-friedrich pushed a commit to dkd-friedrich/ext-solr that referenced this issue Jul 14, 2021
EXT:solrs components like range facets can not be properly handled by
cHash stack, because the amount of possible range-combinations is
infinite, therefore they must be excluded from cacheHash calculation.

This change makes it possible to exclude all EXT:solr parameters from
cacheHash.
To prevent misconfigurations, the new extension configuration setting
"pluginNamespaces" was introduced, which is used in FlexForm and in
TYPO3_CONF_VARS/FE/cacheHash/excludedParameters.
This setting makes it impossible to chose invalid/unhandled EXT:solr
plugin namespace on FlexForm (Plugin -> Options -> Plugin Namespace)

## Migrations:

### Plugin namespaces:

Needed only if other as default (tx_solr) plugin namespace is used in
instance.

Add the used namespace[s] to
`$GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['solr']['pluginNamespaces']`
or via backend "Settings" -> "Extension Configuration" -> "solr" -> "A
list of white listed plugin namespaces"

### Global q parameter

Needed only if global "q" parameter without plugin namespace is used and
wants to be included in cacheHash calculation.
Set the setting
`$GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['solr']['pluginNamespaces'] =
'1'`
or enable it via backend "Settings" -> "Extension Configuration" ->
"solr" -> "Include/Exclude global q parameter in/from cacheHash"

Fixes: TYPO3-Solr#2725
@dkd-friedrich
Copy link
Member

Rafael and I have taken a closer look at the problem and have now chosen a slightly different path.

Since no cHash calculation is possible for the links of the range facets generated via JavaScript and a more extensive caching was possible in the old variant, we decided to exclude the tx_solr parameters. Configuration options for namespaces and parameters is added and a default cHash configuration provided.

dkd-kaehm added a commit that referenced this issue Jul 14, 2021
EXT:solrs components like range facets can not be properly handled by
cHash stack, because the amount of possible range-combinations is
infinite, therefore they must be excluded from cacheHash calculation.

This change makes it possible to exclude all EXT:solr parameters from
cacheHash.
To prevent misconfigurations, the new extension configuration setting
"pluginNamespaces" was introduced, which is used in FlexForm and in
TYPO3_CONF_VARS/FE/cacheHash/excludedParameters.
This setting makes it impossible to chose invalid/unhandled EXT:solr
plugin namespace on FlexForm (Plugin -> Options -> Plugin Namespace)

## Migrations:

### Plugin namespaces:

Needed only if other as default (tx_solr) plugin namespace is used in
instance.

Add the used namespace[s] to
`$GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['solr']['pluginNamespaces']`
or via backend "Settings" -> "Extension Configuration" -> "solr" -> "A
list of white listed plugin namespaces"

### Global q parameter

Needed only if global "q" parameter without plugin namespace is used and
wants to be included in cacheHash calculation.
Set the setting
`$GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['solr']['pluginNamespaces'] =
'1'`
or enable it via backend "Settings" -> "Extension Configuration" ->
"solr" -> "Include/Exclude global q parameter in/from cacheHash"

Fixes: #2725
dkd-kaehm added a commit that referenced this issue Oct 8, 2021
EXT:solrs components like range facets can not be properly handled by
cHash stack, because the amount of possible range-combinations is
infinite, therefore they must be excluded from cacheHash calculation.

This change makes it possible to exclude all EXT:solr parameters from
cacheHash.
To prevent misconfigurations, the new extension configuration setting
"pluginNamespaces" was introduced, which is used in FlexForm and in
TYPO3_CONF_VARS/FE/cacheHash/excludedParameters.
This setting makes it impossible to chose invalid/unhandled EXT:solr
plugin namespace on FlexForm (Plugin -> Options -> Plugin Namespace)

## Migrations:

### Plugin namespaces:

Needed only if other as default (tx_solr) plugin namespace is used in
instance.

Add the used namespace[s] to
`$GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['solr']['pluginNamespaces']`
or via backend "Settings" -> "Extension Configuration" -> "solr" -> "A
list of white listed plugin namespaces"

### Global q parameter

Needed only if global "q" parameter without plugin namespace is used and
wants to be included in cacheHash calculation.
Set the setting
`$GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['solr']['pluginNamespaces'] =
'1'`
or enable it via backend "Settings" -> "Extension Configuration" ->
"solr" -> "Include/Exclude global q parameter in/from cacheHash"

Fixes: #2725
dkd-kaehm added a commit that referenced this issue Oct 8, 2021
EXT:solrs components like range facets can not be properly handled by
cHash stack, because the amount of possible range-combinations is
infinite, therefore they must be excluded from cacheHash calculation.

This change makes it possible to exclude all EXT:solr parameters from
cacheHash.
To prevent misconfigurations, the new extension configuration setting
"pluginNamespaces" was introduced, which is used in FlexForm and in
TYPO3_CONF_VARS/FE/cacheHash/excludedParameters.
This setting makes it impossible to chose invalid/unhandled EXT:solr
plugin namespace on FlexForm (Plugin -> Options -> Plugin Namespace)

## Migrations:

### Plugin namespaces:

Needed only if other as default (tx_solr) plugin namespace is used in
instance.

Add the used namespace[s] to
`$GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['solr']['pluginNamespaces']`
or via backend "Settings" -> "Extension Configuration" -> "solr" -> "A
list of white listed plugin namespaces"

### Global q parameter

Needed only if global "q" parameter without plugin namespace is used and
wants to be included in cacheHash calculation.
Set the setting
`$GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['solr']['pluginNamespaces'] =
'1'`
or enable it via backend "Settings" -> "Extension Configuration" ->
"solr" -> "Include/Exclude global q parameter in/from cacheHash"

Fixes: #2725
dkd-kaehm added a commit that referenced this issue Oct 8, 2021
EXT:solrs components like range facets can not be properly handled by
cHash stack, because the amount of possible range-combinations is
infinite, therefore they must be excluded from cacheHash calculation.

This change makes it possible to exclude all EXT:solr parameters from
cacheHash.
To prevent misconfigurations, the new extension configuration setting
"pluginNamespaces" was introduced, which is used in FlexForm and in
TYPO3_CONF_VARS/FE/cacheHash/excludedParameters.
This setting makes it impossible to chose invalid/unhandled EXT:solr
plugin namespace on FlexForm (Plugin -> Options -> Plugin Namespace)

## Migrations:

### Plugin namespaces:

Needed only if other as default (tx_solr) plugin namespace is used in
instance.

Add the used namespace[s] to
`$GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['solr']['pluginNamespaces']`
or via backend "Settings" -> "Extension Configuration" -> "solr" -> "A
list of white listed plugin namespaces"

### Global q parameter

Needed only if global "q" parameter without plugin namespace is used and
wants to be included in cacheHash calculation.
Set the setting
`$GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['solr']['pluginNamespaces'] =
'1'`
or enable it via backend "Settings" -> "Extension Configuration" ->
"solr" -> "Include/Exclude global q parameter in/from cacheHash"

Fixes: #2725
dkd-kaehm added a commit that referenced this issue Oct 8, 2021
EXT:solrs components like range facets can not be properly handled by
cHash stack, because the amount of possible range-combinations is
infinite, therefore they must be excluded from cacheHash calculation.

This change makes it possible to exclude all EXT:solr parameters from
cacheHash.
To prevent misconfigurations, the new extension configuration setting
"pluginNamespaces" was introduced, which is used in FlexForm and in
TYPO3_CONF_VARS/FE/cacheHash/excludedParameters.
This setting makes it impossible to chose invalid/unhandled EXT:solr
plugin namespace on FlexForm (Plugin -> Options -> Plugin Namespace)

## Migrations:

### Plugin namespaces:

Needed only if other as default (tx_solr) plugin namespace is used in
instance.

Add the used namespace[s] to
`$GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['solr']['pluginNamespaces']`
or via backend "Settings" -> "Extension Configuration" -> "solr" -> "A
list of white listed plugin namespaces"

### Global q parameter

Needed only if global "q" parameter without plugin namespace is used and
wants to be included in cacheHash calculation.
Set the setting
`$GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['solr']['pluginNamespaces'] =
'1'`
or enable it via backend "Settings" -> "Extension Configuration" ->
"solr" -> "Include/Exclude global q parameter in/from cacheHash"

Fixes: #2725
dkd-kaehm added a commit that referenced this issue Oct 8, 2021
EXT:solrs components like range facets can not be properly handled by
cHash stack, because the amount of possible range-combinations is
infinite, therefore they must be excluded from cacheHash calculation.

This change makes it possible to exclude all EXT:solr parameters from
cacheHash.
To prevent misconfigurations, the new extension configuration setting
"pluginNamespaces" was introduced, which is used in FlexForm and in
TYPO3_CONF_VARS/FE/cacheHash/excludedParameters.
This setting makes it impossible to chose invalid/unhandled EXT:solr
plugin namespace on FlexForm (Plugin -> Options -> Plugin Namespace)

## Migrations:

### Plugin namespaces:

Needed only if other as default (tx_solr) plugin namespace is used in
instance.

Add the used namespace[s] to
`$GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['solr']['pluginNamespaces']`
or via backend "Settings" -> "Extension Configuration" -> "solr" -> "A
list of white listed plugin namespaces"

### Global q parameter

Needed only if global "q" parameter without plugin namespace is used and
wants to be included in cacheHash calculation.
Set the setting
`$GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['solr']['pluginNamespaces'] =
'1'`
or enable it via backend "Settings" -> "Extension Configuration" ->
"solr" -> "Include/Exclude global q parameter in/from cacheHash"

Fixes: #2725
dkd-kaehm added a commit that referenced this issue Oct 8, 2021
EXT:solrs components like range facets can not be properly handled by
cHash stack, because the amount of possible range-combinations is
infinite, therefore they must be excluded from cacheHash calculation.

This change makes it possible to exclude all EXT:solr parameters from
cacheHash.
To prevent misconfigurations, the new extension configuration setting
"pluginNamespaces" was introduced, which is used in FlexForm and in
TYPO3_CONF_VARS/FE/cacheHash/excludedParameters.
This setting makes it impossible to chose invalid/unhandled EXT:solr
plugin namespace on FlexForm (Plugin -> Options -> Plugin Namespace)

## Migrations:

### Plugin namespaces:

Needed only if other as default (tx_solr) plugin namespace is used in
instance.

Add the used namespace[s] to
`$GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['solr']['pluginNamespaces']`
or via backend "Settings" -> "Extension Configuration" -> "solr" -> "A
list of white listed plugin namespaces"

### Global q parameter

Needed only if global "q" parameter without plugin namespace is used and
wants to be included in cacheHash calculation.
Set the setting
`$GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['solr']['pluginNamespaces'] =
'1'`
or enable it via backend "Settings" -> "Extension Configuration" ->
"solr" -> "Include/Exclude global q parameter in/from cacheHash"

Fixes: #2725
dkd-kaehm added a commit that referenced this issue Oct 8, 2021
EXT:solrs components like range facets can not be properly handled by
cHash stack, because the amount of possible range-combinations is
infinite, therefore they must be excluded from cacheHash calculation.

This change makes it possible to exclude all EXT:solr parameters from
cacheHash.
To prevent misconfigurations, the new extension configuration setting
"pluginNamespaces" was introduced, which is used in FlexForm and in
TYPO3_CONF_VARS/FE/cacheHash/excludedParameters.
This setting makes it impossible to chose invalid/unhandled EXT:solr
plugin namespace on FlexForm (Plugin -> Options -> Plugin Namespace)

## Migrations:

### Plugin namespaces:

Needed only if other as default (tx_solr) plugin namespace is used in
instance.

Add the used namespace[s] to
`$GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['solr']['pluginNamespaces']`
or via backend "Settings" -> "Extension Configuration" -> "solr" -> "A
list of white listed plugin namespaces"

### Global q parameter

Needed only if global "q" parameter without plugin namespace is used and
wants to be included in cacheHash calculation.
Set the setting
`$GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['solr']['pluginNamespaces'] =
'1'`
or enable it via backend "Settings" -> "Extension Configuration" ->
"solr" -> "Include/Exclude global q parameter in/from cacheHash"

Fixes: #2725
dkd-kaehm added a commit to dkd-kaehm/ext-solr that referenced this issue Dec 2, 2021
EXT:solrs components like range facets can not be properly handled by
cHash stack, because the amount of possible range-combinations is
infinite, therefore they must be excluded from cacheHash calculation.

This change makes it possible to exclude all EXT:solr parameters from
cacheHash.
To prevent misconfigurations, the new extension configuration setting
"pluginNamespaces" was introduced, which is used in FlexForm and in
TYPO3_CONF_VARS/FE/cacheHash/excludedParameters.
This setting makes it impossible to chose invalid/unhandled EXT:solr
plugin namespace on FlexForm (Plugin -> Options -> Plugin Namespace)

## Migrations:

### Plugin namespaces:

Needed only if other as default (tx_solr) plugin namespace is used in
instance.

Add the used namespace[s] to
`$GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['solr']['pluginNamespaces']`
or via backend "Settings" -> "Extension Configuration" -> "solr" -> "A
list of white listed plugin namespaces"

### Global q parameter

Needed only if global "q" parameter without plugin namespace is used and
wants to be included in cacheHash calculation.
Set the setting
`$GLOBALS['TYPO3_CONF_VARS']['EXTENSIONS']['solr']['pluginNamespaces'] =
'1'`
or enable it via backend "Settings" -> "Extension Configuration" ->
"solr" -> "Include/Exclude global q parameter in/from cacheHash"

Fixes: TYPO3-Solr#2725
dkd-friedrich added a commit to dkd-friedrich/ext-solr that referenced this issue Oct 13, 2023
Adds missing documentation for the following extension configuration
settings:
- pluginNamespaces
- includeGlobalQParameterInCacheHash
- useConfigurationMonitorTables
- enableRouteEnhancer

Relates: TYPO3-Solr#2725
Relates: TYPO3-Solr#3810
dkd-kaehm pushed a commit that referenced this issue Oct 13, 2023
Adds missing documentation for the following extension configuration
settings:
- pluginNamespaces
- includeGlobalQParameterInCacheHash
- useConfigurationMonitorTables
- enableRouteEnhancer

Relates: #2725
Relates: #3810
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants