From 539922124f75c6c99a7c6b70239442ce7ca55381 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Christian=20M=C3=BCller?= Date: Wed, 22 Nov 2023 12:09:52 +0100 Subject: [PATCH 01/21] BUGFIX: Update requirements.txt Update requirements to build documentation based on security suggestions. --- Neos.Flow/Documentation/requirements.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Neos.Flow/Documentation/requirements.txt b/Neos.Flow/Documentation/requirements.txt index 12e992402f..55342970ba 100644 --- a/Neos.Flow/Documentation/requirements.txt +++ b/Neos.Flow/Documentation/requirements.txt @@ -58,5 +58,5 @@ sphinxcontrib-qthelp==1.0.6 # via sphinx sphinxcontrib-serializinghtml==1.1.9 # via sphinx -urllib3==2.0.6 +urllib3==2.0.7 # via requests From aac8b84564d92323c4991ca4c4ca70abb46e824b Mon Sep 17 00:00:00 2001 From: Jenkins Date: Wed, 22 Nov 2023 11:18:00 +0000 Subject: [PATCH 02/21] TASK: Update references [skip ci] --- .../TheDefinitiveGuide/PartV/AnnotationReference.rst | 2 +- .../Documentation/TheDefinitiveGuide/PartV/CommandReference.rst | 2 +- .../PartV/FluidAdaptorViewHelperReference.rst | 2 +- .../Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst | 2 +- .../TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst | 2 +- .../TheDefinitiveGuide/PartV/TypeConverterReference.rst | 2 +- .../TheDefinitiveGuide/PartV/ValidatorReference.rst | 2 +- 7 files changed, 7 insertions(+), 7 deletions(-) diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/AnnotationReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/AnnotationReference.rst index f08176a510..216c9c642f 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/AnnotationReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/AnnotationReference.rst @@ -3,7 +3,7 @@ Flow Annotation Reference ========================= -This reference was automatically generated from code on 2023-11-21 +This reference was automatically generated from code on 2023-11-22 .. _`Flow Annotation Reference: After`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/CommandReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/CommandReference.rst index c37c402e25..adf53ed733 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/CommandReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/CommandReference.rst @@ -19,7 +19,7 @@ commands that may be available, use:: ./flow help -The following reference was automatically generated from code on 2023-11-21 +The following reference was automatically generated from code on 2023-11-22 .. _`Flow Command Reference: NEOS.FLOW`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/FluidAdaptorViewHelperReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/FluidAdaptorViewHelperReference.rst index 942e89a928..3c7c7db344 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/FluidAdaptorViewHelperReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/FluidAdaptorViewHelperReference.rst @@ -3,7 +3,7 @@ FluidAdaptor ViewHelper Reference ================================= -This reference was automatically generated from code on 2023-11-21 +This reference was automatically generated from code on 2023-11-22 .. _`FluidAdaptor ViewHelper Reference: f:debug`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst index 707f4e9a7e..39a4e750d1 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst @@ -3,7 +3,7 @@ Flow Signals Reference ====================== -This reference was automatically generated from code on 2023-11-21 +This reference was automatically generated from code on 2023-11-22 .. _`Flow Signals Reference: AbstractAdvice (``Neos\Flow\Aop\Advice\AbstractAdvice``)`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst index bc0662222b..cbf24f9733 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst @@ -3,7 +3,7 @@ TYPO3 Fluid ViewHelper Reference ================================ -This reference was automatically generated from code on 2023-11-21 +This reference was automatically generated from code on 2023-11-22 .. _`TYPO3 Fluid ViewHelper Reference: f:alias`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TypeConverterReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TypeConverterReference.rst index 69e179149d..d5899dddcd 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TypeConverterReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TypeConverterReference.rst @@ -3,7 +3,7 @@ Flow TypeConverter Reference ============================ -This reference was automatically generated from code on 2023-11-21 +This reference was automatically generated from code on 2023-11-22 .. _`Flow TypeConverter Reference: ArrayConverter`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ValidatorReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ValidatorReference.rst index a75012be57..cb0214b480 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ValidatorReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ValidatorReference.rst @@ -3,7 +3,7 @@ Flow Validator Reference ======================== -This reference was automatically generated from code on 2023-11-21 +This reference was automatically generated from code on 2023-11-22 .. _`Flow Validator Reference: AggregateBoundaryValidator`: From cd7ad13267abf4f8d3dec215a96b0694b705b8f8 Mon Sep 17 00:00:00 2001 From: Jenkins Date: Wed, 22 Nov 2023 11:18:37 +0000 Subject: [PATCH 03/21] TASK: Update references [skip ci] --- .../TheDefinitiveGuide/PartV/AnnotationReference.rst | 2 +- .../Documentation/TheDefinitiveGuide/PartV/CommandReference.rst | 2 +- .../PartV/FluidAdaptorViewHelperReference.rst | 2 +- .../Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst | 2 +- .../TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst | 2 +- .../TheDefinitiveGuide/PartV/TypeConverterReference.rst | 2 +- .../TheDefinitiveGuide/PartV/ValidatorReference.rst | 2 +- 7 files changed, 7 insertions(+), 7 deletions(-) diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/AnnotationReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/AnnotationReference.rst index 72fb0e2f66..216c9c642f 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/AnnotationReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/AnnotationReference.rst @@ -3,7 +3,7 @@ Flow Annotation Reference ========================= -This reference was automatically generated from code on 2023-11-07 +This reference was automatically generated from code on 2023-11-22 .. _`Flow Annotation Reference: After`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/CommandReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/CommandReference.rst index 2d64c3dac8..adf53ed733 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/CommandReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/CommandReference.rst @@ -19,7 +19,7 @@ commands that may be available, use:: ./flow help -The following reference was automatically generated from code on 2023-11-07 +The following reference was automatically generated from code on 2023-11-22 .. _`Flow Command Reference: NEOS.FLOW`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/FluidAdaptorViewHelperReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/FluidAdaptorViewHelperReference.rst index e1d3fe98ab..3c7c7db344 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/FluidAdaptorViewHelperReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/FluidAdaptorViewHelperReference.rst @@ -3,7 +3,7 @@ FluidAdaptor ViewHelper Reference ================================= -This reference was automatically generated from code on 2023-11-07 +This reference was automatically generated from code on 2023-11-22 .. _`FluidAdaptor ViewHelper Reference: f:debug`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst index b06eb6bcf8..39a4e750d1 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst @@ -3,7 +3,7 @@ Flow Signals Reference ====================== -This reference was automatically generated from code on 2023-11-07 +This reference was automatically generated from code on 2023-11-22 .. _`Flow Signals Reference: AbstractAdvice (``Neos\Flow\Aop\Advice\AbstractAdvice``)`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst index 13a2adb8e1..cbf24f9733 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst @@ -3,7 +3,7 @@ TYPO3 Fluid ViewHelper Reference ================================ -This reference was automatically generated from code on 2023-11-07 +This reference was automatically generated from code on 2023-11-22 .. _`TYPO3 Fluid ViewHelper Reference: f:alias`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TypeConverterReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TypeConverterReference.rst index 7f5d3c5879..d5899dddcd 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TypeConverterReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TypeConverterReference.rst @@ -3,7 +3,7 @@ Flow TypeConverter Reference ============================ -This reference was automatically generated from code on 2023-11-07 +This reference was automatically generated from code on 2023-11-22 .. _`Flow TypeConverter Reference: ArrayConverter`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ValidatorReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ValidatorReference.rst index 6a8118699e..cb0214b480 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ValidatorReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ValidatorReference.rst @@ -3,7 +3,7 @@ Flow Validator Reference ======================== -This reference was automatically generated from code on 2023-11-07 +This reference was automatically generated from code on 2023-11-22 .. _`Flow Validator Reference: AggregateBoundaryValidator`: From 2d9fe58ecd57fdbe881664ee9820de2bb0cc5f41 Mon Sep 17 00:00:00 2001 From: Jenkins Date: Wed, 22 Nov 2023 12:14:08 +0000 Subject: [PATCH 04/21] TASK: Update references [skip ci] --- .../TheDefinitiveGuide/PartV/AnnotationReference.rst | 2 +- .../Documentation/TheDefinitiveGuide/PartV/CommandReference.rst | 2 +- .../PartV/FluidAdaptorViewHelperReference.rst | 2 +- .../Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst | 2 +- .../TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst | 2 +- .../TheDefinitiveGuide/PartV/TypeConverterReference.rst | 2 +- .../TheDefinitiveGuide/PartV/ValidatorReference.rst | 2 +- 7 files changed, 7 insertions(+), 7 deletions(-) diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/AnnotationReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/AnnotationReference.rst index 72fb0e2f66..216c9c642f 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/AnnotationReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/AnnotationReference.rst @@ -3,7 +3,7 @@ Flow Annotation Reference ========================= -This reference was automatically generated from code on 2023-11-07 +This reference was automatically generated from code on 2023-11-22 .. _`Flow Annotation Reference: After`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/CommandReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/CommandReference.rst index 2d64c3dac8..adf53ed733 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/CommandReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/CommandReference.rst @@ -19,7 +19,7 @@ commands that may be available, use:: ./flow help -The following reference was automatically generated from code on 2023-11-07 +The following reference was automatically generated from code on 2023-11-22 .. _`Flow Command Reference: NEOS.FLOW`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/FluidAdaptorViewHelperReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/FluidAdaptorViewHelperReference.rst index e1d3fe98ab..3c7c7db344 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/FluidAdaptorViewHelperReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/FluidAdaptorViewHelperReference.rst @@ -3,7 +3,7 @@ FluidAdaptor ViewHelper Reference ================================= -This reference was automatically generated from code on 2023-11-07 +This reference was automatically generated from code on 2023-11-22 .. _`FluidAdaptor ViewHelper Reference: f:debug`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst index b06eb6bcf8..39a4e750d1 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst @@ -3,7 +3,7 @@ Flow Signals Reference ====================== -This reference was automatically generated from code on 2023-11-07 +This reference was automatically generated from code on 2023-11-22 .. _`Flow Signals Reference: AbstractAdvice (``Neos\Flow\Aop\Advice\AbstractAdvice``)`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst index 13a2adb8e1..cbf24f9733 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst @@ -3,7 +3,7 @@ TYPO3 Fluid ViewHelper Reference ================================ -This reference was automatically generated from code on 2023-11-07 +This reference was automatically generated from code on 2023-11-22 .. _`TYPO3 Fluid ViewHelper Reference: f:alias`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TypeConverterReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TypeConverterReference.rst index 7f5d3c5879..d5899dddcd 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TypeConverterReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TypeConverterReference.rst @@ -3,7 +3,7 @@ Flow TypeConverter Reference ============================ -This reference was automatically generated from code on 2023-11-07 +This reference was automatically generated from code on 2023-11-22 .. _`Flow TypeConverter Reference: ArrayConverter`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ValidatorReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ValidatorReference.rst index 6a8118699e..cb0214b480 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ValidatorReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ValidatorReference.rst @@ -3,7 +3,7 @@ Flow Validator Reference ======================== -This reference was automatically generated from code on 2023-11-07 +This reference was automatically generated from code on 2023-11-22 .. _`Flow Validator Reference: AggregateBoundaryValidator`: From 185275c8df843f2335c605488b7d8c2e72e6b656 Mon Sep 17 00:00:00 2001 From: Jenkins Date: Wed, 22 Nov 2023 12:54:51 +0000 Subject: [PATCH 05/21] TASK: Add changelog for 7.3.17 [skip ci] See https://jenkins.neos.io/job/flow-release/430/ --- .../PartV/ChangeLogs/7317.rst | 159 ++++++++++++++++++ 1 file changed, 159 insertions(+) create mode 100644 Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ChangeLogs/7317.rst diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ChangeLogs/7317.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ChangeLogs/7317.rst new file mode 100644 index 0000000000..4d3ee55515 --- /dev/null +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ChangeLogs/7317.rst @@ -0,0 +1,159 @@ +`7.3.17 (2023-11-22) `_ +================================================================================================ + +Overview of merged pull requests +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +`BUGFIX: Set InvalidHashException status code to 400 `_ +---------------------------------------------------------------------------------------------------------------------- + +``InvalidHashException`` now declares ``400`` as it's status code (not the inherited ``500`` it has now), as that is clearly a case of a "bad request". + +* See: `#3159 `_ + +**Upgrade instructions** + +This might need adjustment, if you rely on the ``InvalidHashException`` throwing a status code of ``500`` somewhere. + + +* Packages: ``Flow`` + +`BUGFIX: Return the expected result for `is_dir('resource://sha1')` `_ +------------------------------------------------------------------------------------------------------------------------------------- + +* Fixes: `#3225 `_ + + +* Packages: ``Flow`` + +`BUGFIX: Use method to set validated instances container `_ +-------------------------------------------------------------------------------------------------------------------------- + +* Fixes: `#3205 `_ + + +* Packages: ``Flow`` + +`BUGFIX: Require collection packages as `self.version` again `_ +------------------------------------------------------------------------------------------------------------------------------ + +* See: `#3035 `_ for the original change + + +* Packages: ``Flow`` ``Eel`` ``FluidAdaptor`` ``Kickstarter`` + +`BUGFIX: Only set distinct on count clause if explicitely set to improve performance `_ +------------------------------------------------------------------------------------------------------------------------------------------------------ + +F.e. Postgres has performance issues with large datasets and the DISTINCT clause. In a test this change reduced the query time of a count query for ~900.000 entities by >80%. + +In a custom project this affected their Neos Media.UI in which the following results were found: + +* Count all assets | 580ms -> 260ms +* Query 20 assets | 690ms -> 350ms +* Query 100 assets | 990ms -> 650ms +* Module load | 1900ms -> 1400ms + +**Review instructions** + +Everything should work the same, as https://github.com/neos/flow-development-collection/pull/415 already sets the distinct flag where (possibly) necessary. + + +* Packages: ``Flow`` + +`BUGFIX: Sanitize uploaded svg files from suspicious content `_ +------------------------------------------------------------------------------------------------------------------------------ + +Adding an internal methods ``isSanitizingRequired`` and ``sanitizeImportedFileContent`` to the resourceManager. The import is adjusted to first determine the mediaType of an imported resource to decide wether sanitizing is needed which for now happens only for SVG files. If no sanitizing is needed the code will perform as before by passing streams or filenames around. + +If suspicious content was removed from a warning is logged that mentions the remove data and line. The sanitizing is done using "enshrined/svg-sanitize" that is used by other cms aswell. + +The initial implementation will only sanitize SVG files as those can contain malicious scripts. In future this should be expanded to a feature that allows registering of custom sanitizing functions. + +The sanitizing logic itself ist basically the same as what is done by typo3 here: https://github.com/TYPO3/typo3/blob/`357b07064cf2c7f1735cfb8f73ac4a7248ab040e `_/typo3/sysext/core/Classes/Resource/Security/SvgSanitizer.php + +This addresses the issue described here: https://nvd.nist.gov/vuln/detail/CVE-2023-37611 + +**Review Instructions** + +The change adds quite a bit of complexity to the importResource method to avoid loading the file content into ram whenever possible. As this method accepts filenames and resources this leads to quite some nested checking. I consider this kindoff necessary as one does not want to read a full video file into php ram to check wether it may be an svg. + +Better suggestions are welcome. + + +* Packages: ``Utility.MediaTypes`` + +`TASK: Update default .htaccess for _Resources `_ +---------------------------------------------------------------------------------------------------------------- + +PHP 5 is a thing of the past, but for PHP 8 the module is name just ``mod_php.c``, so that needs to be added. + +**Upgrade instructions** + +Depending in the way you deploy and whether you have that file even in version control, the change might need to be applied manually to your setup. + +**Review instructions** + + +* Packages: ``Flow`` + +`TASK: Routing Documentation Adjustment `_ +---------------------------------------------------------------------------------------------------------- + +Correction of an erroneous path in routing documentation. + +* Packages: ``Flow`` + +`TASK: PEG Parser declares properties `_ +------------------------------------------------------------------------------------------------------- + +Prevents deprecation warnings for dynamic properties. + +* Packages: ``Flow`` ``Eel`` + +`TASK: Clean up stored throwable dumps `_ +-------------------------------------------------------------------------------------------------------- + +Whenever a new dump is written, check the existing dumps and remove those that are older than allowed or exceed the maximum count. + +By default nothing is cleaned up. + +* Resolves: `#3158 `_ + +**Review instructions** + +Should remove old dump files as configured… + + +* Packages: ``Flow`` + +`TASK: Fix overlooked dependency… `_ +----------------------------------------------------------------------------------------------------- + +* See: `#3035 `_ for the original change + + +* Packages: ``Flow`` + +`TASK: Fix cache RedisBackend unittest `_ +-------------------------------------------------------------------------------------------------------- + +A test failed due to a missing return value from a method not being mocked (correctly), + + +* Packages: ``Cache`` + +`TASK: Fix documentation builds `_ +------------------------------------------------------------------------------------------------- + +… by pinning updated dependencies. + +**Review instructions** + +Best is to see if the builds succeed on RTD again with this merged… + + +* Packages: ``Flow`` + +`Detailed log `_ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From b472625291fd00657e8dcedb3f1052f5595d2017 Mon Sep 17 00:00:00 2001 From: Jenkins Date: Wed, 22 Nov 2023 12:56:07 +0000 Subject: [PATCH 06/21] TASK: Update references [skip ci] --- .../TheDefinitiveGuide/PartV/AnnotationReference.rst | 2 +- .../Documentation/TheDefinitiveGuide/PartV/CommandReference.rst | 2 +- .../PartV/FluidAdaptorViewHelperReference.rst | 2 +- .../Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst | 2 +- .../TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst | 2 +- .../TheDefinitiveGuide/PartV/TypeConverterReference.rst | 2 +- .../TheDefinitiveGuide/PartV/ValidatorReference.rst | 2 +- 7 files changed, 7 insertions(+), 7 deletions(-) diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/AnnotationReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/AnnotationReference.rst index 045e027aa9..216c9c642f 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/AnnotationReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/AnnotationReference.rst @@ -3,7 +3,7 @@ Flow Annotation Reference ========================= -This reference was automatically generated from code on 2023-11-17 +This reference was automatically generated from code on 2023-11-22 .. _`Flow Annotation Reference: After`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/CommandReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/CommandReference.rst index 9cc46f1b6d..b4e8dec2fb 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/CommandReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/CommandReference.rst @@ -19,7 +19,7 @@ commands that may be available, use:: ./flow help -The following reference was automatically generated from code on 2023-11-17 +The following reference was automatically generated from code on 2023-11-22 .. _`Flow Command Reference: NEOS.FLOW`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/FluidAdaptorViewHelperReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/FluidAdaptorViewHelperReference.rst index bef1107e32..3c7c7db344 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/FluidAdaptorViewHelperReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/FluidAdaptorViewHelperReference.rst @@ -3,7 +3,7 @@ FluidAdaptor ViewHelper Reference ================================= -This reference was automatically generated from code on 2023-11-17 +This reference was automatically generated from code on 2023-11-22 .. _`FluidAdaptor ViewHelper Reference: f:debug`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst index deff238690..89214e49b7 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst @@ -3,7 +3,7 @@ Flow Signals Reference ====================== -This reference was automatically generated from code on 2023-11-17 +This reference was automatically generated from code on 2023-11-22 .. _`Flow Signals Reference: AbstractAdvice (``Neos\Flow\Aop\Advice\AbstractAdvice``)`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst index 1eeb0877cd..cbf24f9733 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst @@ -3,7 +3,7 @@ TYPO3 Fluid ViewHelper Reference ================================ -This reference was automatically generated from code on 2023-11-17 +This reference was automatically generated from code on 2023-11-22 .. _`TYPO3 Fluid ViewHelper Reference: f:alias`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TypeConverterReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TypeConverterReference.rst index 5a3efdf3fe..d5899dddcd 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TypeConverterReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TypeConverterReference.rst @@ -3,7 +3,7 @@ Flow TypeConverter Reference ============================ -This reference was automatically generated from code on 2023-11-17 +This reference was automatically generated from code on 2023-11-22 .. _`Flow TypeConverter Reference: ArrayConverter`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ValidatorReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ValidatorReference.rst index 88b57140f7..cb0214b480 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ValidatorReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ValidatorReference.rst @@ -3,7 +3,7 @@ Flow Validator Reference ======================== -This reference was automatically generated from code on 2023-11-17 +This reference was automatically generated from code on 2023-11-22 .. _`Flow Validator Reference: AggregateBoundaryValidator`: From 29c3d6291b7e88b95e6542ab65d422411c85caea Mon Sep 17 00:00:00 2001 From: Jenkins Date: Wed, 22 Nov 2023 12:57:40 +0000 Subject: [PATCH 07/21] TASK: Add changelog for 8.0.14 [skip ci] See https://jenkins.neos.io/job/flow-release/431/ --- .../PartV/ChangeLogs/8014.rst | 192 ++++++++++++++++++ 1 file changed, 192 insertions(+) create mode 100644 Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ChangeLogs/8014.rst diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ChangeLogs/8014.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ChangeLogs/8014.rst new file mode 100644 index 0000000000..0a4afb3e3b --- /dev/null +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ChangeLogs/8014.rst @@ -0,0 +1,192 @@ +`8.0.14 (2023-11-22) `_ +================================================================================================ + +Overview of merged pull requests +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +`BUGFIX: No useless read in `FileBackend` (improve performance) `_ +--------------------------------------------------------------------------------------------------------------------------------- + +``FileBackend::findIdentifiersByTags`` now early returns if ``$tags`` is empty. Otherwise it would read every cache entry completely unnecessarily from the filesystem. + +
+ Profile before + +!`Screenshot_1 `_ + +
+ +
+ Profile after + +Screenshot_2 + +Screenshot_3 + +
+ + +* Packages: ``Flow`` ``Cache`` + +`BUGFIX: Set InvalidHashException status code to 400 `_ +---------------------------------------------------------------------------------------------------------------------- + +``InvalidHashException`` now declares ``400`` as it's status code (not the inherited ``500`` it has now), as that is clearly a case of a "bad request". + +* See: `#3159 `_ + +**Upgrade instructions** + +This might need adjustment, if you rely on the ``InvalidHashException`` throwing a status code of ``500`` somewhere. + + +* Packages: ``Flow`` + +`BUGFIX: Return the expected result for `is_dir('resource://sha1')` `_ +------------------------------------------------------------------------------------------------------------------------------------- + +* Fixes: `#3225 `_ + + +* Packages: ``Flow`` + +`BUGFIX: Use method to set validated instances container `_ +-------------------------------------------------------------------------------------------------------------------------- + +* Fixes: `#3205 `_ + + +* Packages: ``Flow`` + +`BUGFIX: Require collection packages as `self.version` again `_ +------------------------------------------------------------------------------------------------------------------------------ + +* See: `#3035 `_ for the original change + + +* Packages: ``Flow`` ``Eel`` ``FluidAdaptor`` ``Kickstarter`` + +`BUGFIX: Only set distinct on count clause if explicitely set to improve performance `_ +------------------------------------------------------------------------------------------------------------------------------------------------------ + +F.e. Postgres has performance issues with large datasets and the DISTINCT clause. In a test this change reduced the query time of a count query for ~900.000 entities by >80%. + +In a custom project this affected their Neos Media.UI in which the following results were found: + +* Count all assets | 580ms -> 260ms +* Query 20 assets | 690ms -> 350ms +* Query 100 assets | 990ms -> 650ms +* Module load | 1900ms -> 1400ms + +**Review instructions** + +Everything should work the same, as https://github.com/neos/flow-development-collection/pull/415 already sets the distinct flag where (possibly) necessary. + + +* Packages: ``Flow`` + +`BUGFIX: Sanitize uploaded svg files from suspicious content `_ +------------------------------------------------------------------------------------------------------------------------------ + +Adding an internal methods ``isSanitizingRequired`` and ``sanitizeImportedFileContent`` to the resourceManager. The import is adjusted to first determine the mediaType of an imported resource to decide wether sanitizing is needed which for now happens only for SVG files. If no sanitizing is needed the code will perform as before by passing streams or filenames around. + +If suspicious content was removed from a warning is logged that mentions the remove data and line. The sanitizing is done using "enshrined/svg-sanitize" that is used by other cms aswell. + +The initial implementation will only sanitize SVG files as those can contain malicious scripts. In future this should be expanded to a feature that allows registering of custom sanitizing functions. + +The sanitizing logic itself ist basically the same as what is done by typo3 here: https://github.com/TYPO3/typo3/blob/`357b07064cf2c7f1735cfb8f73ac4a7248ab040e `_/typo3/sysext/core/Classes/Resource/Security/SvgSanitizer.php + +This addresses the issue described here: https://nvd.nist.gov/vuln/detail/CVE-2023-37611 + +**Review Instructions** + +The change adds quite a bit of complexity to the importResource method to avoid loading the file content into ram whenever possible. As this method accepts filenames and resources this leads to quite some nested checking. I consider this kindoff necessary as one does not want to read a full video file into php ram to check wether it may be an svg. + +Better suggestions are welcome. + + +* Packages: ``Utility.MediaTypes`` + +`TASK: Update default .htaccess for _Resources `_ +---------------------------------------------------------------------------------------------------------------- + +PHP 5 is a thing of the past, but for PHP 8 the module is name just ``mod_php.c``, so that needs to be added. + +**Upgrade instructions** + +Depending in the way you deploy and whether you have that file even in version control, the change might need to be applied manually to your setup. + + +* Packages: ``Flow`` + +`TASK: Routing Documentation Adjustment `_ +---------------------------------------------------------------------------------------------------------- + +Correction of an erroneous path in routing documentation. + +* Packages: ``Flow`` + +`TASK: Migrate to PHPStan for Flow 8 `_ +------------------------------------------------------------------------------------------------------ + +This is a backport of https://github.com/neos/flow-development-collection/pull/3216 + +Adds PHPStan level 1 to the whole Flow code base and CI. +Psalm was removed. + + +* Packages: ``Flow`` ``.github`` ``Cache`` + +`TASK: PEG Parser declares properties `_ +------------------------------------------------------------------------------------------------------- + +Prevents deprecation warnings for dynamic properties. + +* Packages: ``Flow`` ``Eel`` + +`TASK: Clean up stored throwable dumps `_ +-------------------------------------------------------------------------------------------------------- + +Whenever a new dump is written, check the existing dumps and remove those that are older than allowed or exceed the maximum count. + +By default nothing is cleaned up. + +* Resolves: `#3158 `_ + +**Review instructions** + +Should remove old dump files as configured… + + +* Packages: ``Flow`` + +`TASK: Fix overlooked dependency… `_ +----------------------------------------------------------------------------------------------------- + +* See: `#3035 `_ for the original change + + +* Packages: ``Flow`` + +`TASK: Fix cache RedisBackend unittest `_ +-------------------------------------------------------------------------------------------------------- + +A test failed due to a missing return value from a method not being mocked (correctly), + + +* Packages: ``Cache`` + +`TASK: Fix documentation builds `_ +------------------------------------------------------------------------------------------------- + +… by pinning updated dependencies. + +**Review instructions** + +Best is to see if the builds succeed on RTD again with this merged… + + +* Packages: ``Flow`` + +`Detailed log `_ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From 338db404534c43dafaa561a12a9514bb7ffe302b Mon Sep 17 00:00:00 2001 From: Jenkins Date: Wed, 22 Nov 2023 12:57:52 +0000 Subject: [PATCH 08/21] TASK: Update composer manifest for ~8.0.0 See https://jenkins.neos.io/job/flow-release/431/ --- Neos.Eel/composer.json | 2 +- Neos.Flow/composer.json | 26 +++++++++++++------------- Neos.FluidAdaptor/composer.json | 2 +- Neos.Kickstarter/composer.json | 4 ++-- 4 files changed, 17 insertions(+), 17 deletions(-) diff --git a/Neos.Eel/composer.json b/Neos.Eel/composer.json index 6171ed0c38..0544dce133 100644 --- a/Neos.Eel/composer.json +++ b/Neos.Eel/composer.json @@ -5,7 +5,7 @@ "description": "The Embedded Expression Language (Eel) is a building block for creating Domain Specific Languages", "require": { "php": "^8.0", - "neos/flow": "self.version", + "neos/flow": "~8.0.0", "neos/cache": "self.version", "neos/utility-unicode": "self.version", "neos/utility-objecthandling": "self.version" diff --git a/Neos.Flow/composer.json b/Neos.Flow/composer.json index b4027a18e9..5f3e4bba02 100644 --- a/Neos.Flow/composer.json +++ b/Neos.Flow/composer.json @@ -14,19 +14,19 @@ "ext-reflection": "*", "ext-xml": "*", - "neos/cache": "self.version", - "neos/eel": "self.version", - "neos/error-messages": "self.version", - "neos/utility-arrays": "self.version", - "neos/utility-files": "self.version", - "neos/utility-mediatypes": "self.version", - "neos/utility-objecthandling": "self.version", - "neos/utility-opcodecache": "self.version", - "neos/utility-schema": "self.version", - "neos/utility-unicode": "self.version", - "neos/flow-log": "self.version", - "neos/http-factories": "self.version", - "neos/utility-pdo": "self.version", + "neos/cache": "~8.0.0", + "neos/eel": "~8.0.0", + "neos/error-messages": "~8.0.0", + "neos/utility-arrays": "~8.0.0", + "neos/utility-files": "~8.0.0", + "neos/utility-mediatypes": "~8.0.0", + "neos/utility-objecthandling": "~8.0.0", + "neos/utility-opcodecache": "~8.0.0", + "neos/utility-schema": "~8.0.0", + "neos/utility-unicode": "~8.0.0", + "neos/flow-log": "~8.0.0", + "neos/http-factories": "~8.0.0", + "neos/utility-pdo": "~8.0.0", "neos/composer-plugin": "^2.0", diff --git a/Neos.FluidAdaptor/composer.json b/Neos.FluidAdaptor/composer.json index f06eccf929..93457608c7 100644 --- a/Neos.FluidAdaptor/composer.json +++ b/Neos.FluidAdaptor/composer.json @@ -7,7 +7,7 @@ ], "require": { "php": "^8.0", - "neos/flow": "self.version", + "neos/flow": "~8.0.0", "neos/cache": "self.version", "neos/utility-files": "self.version", "neos/utility-objecthandling": "self.version", diff --git a/Neos.Kickstarter/composer.json b/Neos.Kickstarter/composer.json index 4e9f413c83..5964734d6b 100644 --- a/Neos.Kickstarter/composer.json +++ b/Neos.Kickstarter/composer.json @@ -5,8 +5,8 @@ "license": "MIT", "require": { "php": "^8.0", - "neos/flow": "self.version", - "neos/fluid-adaptor": "self.version", + "neos/flow": "~8.0.0", + "neos/fluid-adaptor": "~8.0.0", "neos/utility-arrays": "self.version" }, "autoload": { From 7c4a40e4e1110e561dcca82a9250b5e7468bec72 Mon Sep 17 00:00:00 2001 From: Jenkins Date: Wed, 22 Nov 2023 13:00:18 +0000 Subject: [PATCH 09/21] TASK: Add changelog for 8.1.10 [skip ci] See https://jenkins.neos.io/job/flow-release/432/ --- .../PartV/ChangeLogs/8110.rst | 192 ++++++++++++++++++ 1 file changed, 192 insertions(+) create mode 100644 Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ChangeLogs/8110.rst diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ChangeLogs/8110.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ChangeLogs/8110.rst new file mode 100644 index 0000000000..5f93d35207 --- /dev/null +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ChangeLogs/8110.rst @@ -0,0 +1,192 @@ +`8.1.10 (2023-11-22) `_ +================================================================================================ + +Overview of merged pull requests +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +`BUGFIX: No useless read in `FileBackend` (improve performance) `_ +--------------------------------------------------------------------------------------------------------------------------------- + +``FileBackend::findIdentifiersByTags`` now early returns if ``$tags`` is empty. Otherwise it would read every cache entry completely unnecessarily from the filesystem. + +
+ Profile before + +!`Screenshot_1 `_ + +
+ +
+ Profile after + +Screenshot_2 + +Screenshot_3 + +
+ + +* Packages: ``Flow`` ``Cache`` + +`BUGFIX: Set InvalidHashException status code to 400 `_ +---------------------------------------------------------------------------------------------------------------------- + +``InvalidHashException`` now declares ``400`` as it's status code (not the inherited ``500`` it has now), as that is clearly a case of a "bad request". + +* See: `#3159 `_ + +**Upgrade instructions** + +This might need adjustment, if you rely on the ``InvalidHashException`` throwing a status code of ``500`` somewhere. + + +* Packages: ``Flow`` + +`BUGFIX: Return the expected result for `is_dir('resource://sha1')` `_ +------------------------------------------------------------------------------------------------------------------------------------- + +* Fixes: `#3225 `_ + + +* Packages: ``Flow`` + +`BUGFIX: Use method to set validated instances container `_ +-------------------------------------------------------------------------------------------------------------------------- + +* Fixes: `#3205 `_ + + +* Packages: ``Flow`` + +`BUGFIX: Require collection packages as `self.version` again `_ +------------------------------------------------------------------------------------------------------------------------------ + +* See: `#3035 `_ for the original change + + +* Packages: ``Flow`` ``Eel`` ``FluidAdaptor`` ``Kickstarter`` + +`BUGFIX: Only set distinct on count clause if explicitely set to improve performance `_ +------------------------------------------------------------------------------------------------------------------------------------------------------ + +F.e. Postgres has performance issues with large datasets and the DISTINCT clause. In a test this change reduced the query time of a count query for ~900.000 entities by >80%. + +In a custom project this affected their Neos Media.UI in which the following results were found: + +* Count all assets | 580ms -> 260ms +* Query 20 assets | 690ms -> 350ms +* Query 100 assets | 990ms -> 650ms +* Module load | 1900ms -> 1400ms + +**Review instructions** + +Everything should work the same, as https://github.com/neos/flow-development-collection/pull/415 already sets the distinct flag where (possibly) necessary. + + +* Packages: ``Flow`` + +`BUGFIX: Sanitize uploaded svg files from suspicious content `_ +------------------------------------------------------------------------------------------------------------------------------ + +Adding an internal methods ``isSanitizingRequired`` and ``sanitizeImportedFileContent`` to the resourceManager. The import is adjusted to first determine the mediaType of an imported resource to decide wether sanitizing is needed which for now happens only for SVG files. If no sanitizing is needed the code will perform as before by passing streams or filenames around. + +If suspicious content was removed from a warning is logged that mentions the remove data and line. The sanitizing is done using "enshrined/svg-sanitize" that is used by other cms aswell. + +The initial implementation will only sanitize SVG files as those can contain malicious scripts. In future this should be expanded to a feature that allows registering of custom sanitizing functions. + +The sanitizing logic itself ist basically the same as what is done by typo3 here: https://github.com/TYPO3/typo3/blob/`357b07064cf2c7f1735cfb8f73ac4a7248ab040e `_/typo3/sysext/core/Classes/Resource/Security/SvgSanitizer.php + +This addresses the issue described here: https://nvd.nist.gov/vuln/detail/CVE-2023-37611 + +**Review Instructions** + +The change adds quite a bit of complexity to the importResource method to avoid loading the file content into ram whenever possible. As this method accepts filenames and resources this leads to quite some nested checking. I consider this kindoff necessary as one does not want to read a full video file into php ram to check wether it may be an svg. + +Better suggestions are welcome. + + +* Packages: ``Utility.MediaTypes`` + +`TASK: Update default .htaccess for _Resources `_ +---------------------------------------------------------------------------------------------------------------- + +PHP 5 is a thing of the past, but for PHP 8 the module is name just ``mod_php.c``, so that needs to be added. + +**Upgrade instructions** + +Depending in the way you deploy and whether you have that file even in version control, the change might need to be applied manually to your setup. + + +* Packages: ``Flow`` + +`TASK: Routing Documentation Adjustment `_ +---------------------------------------------------------------------------------------------------------- + +Correction of an erroneous path in routing documentation. + +* Packages: ``Flow`` + +`TASK: Migrate to PHPStan for Flow 8 `_ +------------------------------------------------------------------------------------------------------ + +This is a backport of https://github.com/neos/flow-development-collection/pull/3216 + +Adds PHPStan level 1 to the whole Flow code base and CI. +Psalm was removed. + + +* Packages: ``Flow`` ``.github`` ``Cache`` + +`TASK: PEG Parser declares properties `_ +------------------------------------------------------------------------------------------------------- + +Prevents deprecation warnings for dynamic properties. + +* Packages: ``Flow`` ``Eel`` + +`TASK: Clean up stored throwable dumps `_ +-------------------------------------------------------------------------------------------------------- + +Whenever a new dump is written, check the existing dumps and remove those that are older than allowed or exceed the maximum count. + +By default nothing is cleaned up. + +* Resolves: `#3158 `_ + +**Review instructions** + +Should remove old dump files as configured… + + +* Packages: ``Flow`` + +`TASK: Fix overlooked dependency… `_ +----------------------------------------------------------------------------------------------------- + +* See: `#3035 `_ for the original change + + +* Packages: ``Flow`` + +`TASK: Fix cache RedisBackend unittest `_ +-------------------------------------------------------------------------------------------------------- + +A test failed due to a missing return value from a method not being mocked (correctly), + + +* Packages: ``Cache`` + +`TASK: Fix documentation builds `_ +------------------------------------------------------------------------------------------------- + +… by pinning updated dependencies. + +**Review instructions** + +Best is to see if the builds succeed on RTD again with this merged… + + +* Packages: ``Flow`` + +`Detailed log `_ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From ce07c0663ba07376770d81df955fb39a4c4bc6a4 Mon Sep 17 00:00:00 2001 From: Jenkins Date: Wed, 22 Nov 2023 13:00:30 +0000 Subject: [PATCH 10/21] TASK: Update composer manifest for ~8.1.0 See https://jenkins.neos.io/job/flow-release/432/ --- Neos.Eel/composer.json | 2 +- Neos.Flow/composer.json | 26 +++++++++++++------------- Neos.FluidAdaptor/composer.json | 2 +- Neos.Kickstarter/composer.json | 4 ++-- 4 files changed, 17 insertions(+), 17 deletions(-) diff --git a/Neos.Eel/composer.json b/Neos.Eel/composer.json index 6171ed0c38..195230d48a 100644 --- a/Neos.Eel/composer.json +++ b/Neos.Eel/composer.json @@ -5,7 +5,7 @@ "description": "The Embedded Expression Language (Eel) is a building block for creating Domain Specific Languages", "require": { "php": "^8.0", - "neos/flow": "self.version", + "neos/flow": "~8.1.0", "neos/cache": "self.version", "neos/utility-unicode": "self.version", "neos/utility-objecthandling": "self.version" diff --git a/Neos.Flow/composer.json b/Neos.Flow/composer.json index 6e478b5b56..0c9fe0642c 100644 --- a/Neos.Flow/composer.json +++ b/Neos.Flow/composer.json @@ -14,19 +14,19 @@ "ext-reflection": "*", "ext-xml": "*", - "neos/cache": "self.version", - "neos/eel": "self.version", - "neos/error-messages": "self.version", - "neos/utility-arrays": "self.version", - "neos/utility-files": "self.version", - "neos/utility-mediatypes": "self.version", - "neos/utility-objecthandling": "self.version", - "neos/utility-opcodecache": "self.version", - "neos/utility-schema": "self.version", - "neos/utility-unicode": "self.version", - "neos/flow-log": "self.version", - "neos/http-factories": "self.version", - "neos/utility-pdo": "self.version", + "neos/cache": "~8.1.0", + "neos/eel": "~8.1.0", + "neos/error-messages": "~8.1.0", + "neos/utility-arrays": "~8.1.0", + "neos/utility-files": "~8.1.0", + "neos/utility-mediatypes": "~8.1.0", + "neos/utility-objecthandling": "~8.1.0", + "neos/utility-opcodecache": "~8.1.0", + "neos/utility-schema": "~8.1.0", + "neos/utility-unicode": "~8.1.0", + "neos/flow-log": "~8.1.0", + "neos/http-factories": "~8.1.0", + "neos/utility-pdo": "~8.1.0", "neos/composer-plugin": "^2.0", diff --git a/Neos.FluidAdaptor/composer.json b/Neos.FluidAdaptor/composer.json index f06eccf929..48ec0349ae 100644 --- a/Neos.FluidAdaptor/composer.json +++ b/Neos.FluidAdaptor/composer.json @@ -7,7 +7,7 @@ ], "require": { "php": "^8.0", - "neos/flow": "self.version", + "neos/flow": "~8.1.0", "neos/cache": "self.version", "neos/utility-files": "self.version", "neos/utility-objecthandling": "self.version", diff --git a/Neos.Kickstarter/composer.json b/Neos.Kickstarter/composer.json index 4e9f413c83..30694bc2b1 100644 --- a/Neos.Kickstarter/composer.json +++ b/Neos.Kickstarter/composer.json @@ -5,8 +5,8 @@ "license": "MIT", "require": { "php": "^8.0", - "neos/flow": "self.version", - "neos/fluid-adaptor": "self.version", + "neos/flow": "~8.1.0", + "neos/fluid-adaptor": "~8.1.0", "neos/utility-arrays": "self.version" }, "autoload": { From 487c797941ad60db129485154f5fd6fa337e9170 Mon Sep 17 00:00:00 2001 From: Jenkins Date: Wed, 22 Nov 2023 13:07:18 +0000 Subject: [PATCH 11/21] TASK: Add changelog for 8.2.8 [skip ci] See https://jenkins.neos.io/job/flow-release/433/ --- .../PartV/ChangeLogs/828.rst | 192 ++++++++++++++++++ 1 file changed, 192 insertions(+) create mode 100644 Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ChangeLogs/828.rst diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ChangeLogs/828.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ChangeLogs/828.rst new file mode 100644 index 0000000000..75623209d1 --- /dev/null +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ChangeLogs/828.rst @@ -0,0 +1,192 @@ +`8.2.8 (2023-11-22) `_ +============================================================================================== + +Overview of merged pull requests +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +`BUGFIX: No useless read in `FileBackend` (improve performance) `_ +--------------------------------------------------------------------------------------------------------------------------------- + +``FileBackend::findIdentifiersByTags`` now early returns if ``$tags`` is empty. Otherwise it would read every cache entry completely unnecessarily from the filesystem. + +
+ Profile before + +!`Screenshot_1 `_ + +
+ +
+ Profile after + +Screenshot_2 + +Screenshot_3 + +
+ + +* Packages: ``Flow`` ``Cache`` + +`BUGFIX: Set InvalidHashException status code to 400 `_ +---------------------------------------------------------------------------------------------------------------------- + +``InvalidHashException`` now declares ``400`` as it's status code (not the inherited ``500`` it has now), as that is clearly a case of a "bad request". + +* See: `#3159 `_ + +**Upgrade instructions** + +This might need adjustment, if you rely on the ``InvalidHashException`` throwing a status code of ``500`` somewhere. + + +* Packages: ``Flow`` + +`BUGFIX: Return the expected result for `is_dir('resource://sha1')` `_ +------------------------------------------------------------------------------------------------------------------------------------- + +* Fixes: `#3225 `_ + + +* Packages: ``Flow`` + +`BUGFIX: Use method to set validated instances container `_ +-------------------------------------------------------------------------------------------------------------------------- + +* Fixes: `#3205 `_ + + +* Packages: ``Flow`` + +`BUGFIX: Require collection packages as `self.version` again `_ +------------------------------------------------------------------------------------------------------------------------------ + +* See: `#3035 `_ for the original change + + +* Packages: ``Flow`` ``Eel`` ``FluidAdaptor`` ``Kickstarter`` + +`BUGFIX: Only set distinct on count clause if explicitely set to improve performance `_ +------------------------------------------------------------------------------------------------------------------------------------------------------ + +F.e. Postgres has performance issues with large datasets and the DISTINCT clause. In a test this change reduced the query time of a count query for ~900.000 entities by >80%. + +In a custom project this affected their Neos Media.UI in which the following results were found: + +* Count all assets | 580ms -> 260ms +* Query 20 assets | 690ms -> 350ms +* Query 100 assets | 990ms -> 650ms +* Module load | 1900ms -> 1400ms + +**Review instructions** + +Everything should work the same, as https://github.com/neos/flow-development-collection/pull/415 already sets the distinct flag where (possibly) necessary. + + +* Packages: ``Flow`` + +`BUGFIX: Sanitize uploaded svg files from suspicious content `_ +------------------------------------------------------------------------------------------------------------------------------ + +Adding an internal methods ``isSanitizingRequired`` and ``sanitizeImportedFileContent`` to the resourceManager. The import is adjusted to first determine the mediaType of an imported resource to decide wether sanitizing is needed which for now happens only for SVG files. If no sanitizing is needed the code will perform as before by passing streams or filenames around. + +If suspicious content was removed from a warning is logged that mentions the remove data and line. The sanitizing is done using "enshrined/svg-sanitize" that is used by other cms aswell. + +The initial implementation will only sanitize SVG files as those can contain malicious scripts. In future this should be expanded to a feature that allows registering of custom sanitizing functions. + +The sanitizing logic itself ist basically the same as what is done by typo3 here: https://github.com/TYPO3/typo3/blob/`357b07064cf2c7f1735cfb8f73ac4a7248ab040e `_/typo3/sysext/core/Classes/Resource/Security/SvgSanitizer.php + +This addresses the issue described here: https://nvd.nist.gov/vuln/detail/CVE-2023-37611 + +**Review Instructions** + +The change adds quite a bit of complexity to the importResource method to avoid loading the file content into ram whenever possible. As this method accepts filenames and resources this leads to quite some nested checking. I consider this kindoff necessary as one does not want to read a full video file into php ram to check wether it may be an svg. + +Better suggestions are welcome. + + +* Packages: ``Utility.MediaTypes`` + +`TASK: Update default .htaccess for _Resources `_ +---------------------------------------------------------------------------------------------------------------- + +PHP 5 is a thing of the past, but for PHP 8 the module is name just ``mod_php.c``, so that needs to be added. + +**Upgrade instructions** + +Depending in the way you deploy and whether you have that file even in version control, the change might need to be applied manually to your setup. + + +* Packages: ``Flow`` + +`TASK: Routing Documentation Adjustment `_ +---------------------------------------------------------------------------------------------------------- + +Correction of an erroneous path in routing documentation. + +* Packages: ``Flow`` + +`TASK: Migrate to PHPStan for Flow 8 `_ +------------------------------------------------------------------------------------------------------ + +This is a backport of https://github.com/neos/flow-development-collection/pull/3216 + +Adds PHPStan level 1 to the whole Flow code base and CI. +Psalm was removed. + + +* Packages: ``Flow`` ``.github`` ``Cache`` + +`TASK: PEG Parser declares properties `_ +------------------------------------------------------------------------------------------------------- + +Prevents deprecation warnings for dynamic properties. + +* Packages: ``Flow`` ``Eel`` + +`TASK: Clean up stored throwable dumps `_ +-------------------------------------------------------------------------------------------------------- + +Whenever a new dump is written, check the existing dumps and remove those that are older than allowed or exceed the maximum count. + +By default nothing is cleaned up. + +* Resolves: `#3158 `_ + +**Review instructions** + +Should remove old dump files as configured… + + +* Packages: ``Flow`` + +`TASK: Fix overlooked dependency… `_ +----------------------------------------------------------------------------------------------------- + +* See: `#3035 `_ for the original change + + +* Packages: ``Flow`` + +`TASK: Fix cache RedisBackend unittest `_ +-------------------------------------------------------------------------------------------------------- + +A test failed due to a missing return value from a method not being mocked (correctly), + + +* Packages: ``Cache`` + +`TASK: Fix documentation builds `_ +------------------------------------------------------------------------------------------------- + +… by pinning updated dependencies. + +**Review instructions** + +Best is to see if the builds succeed on RTD again with this merged… + + +* Packages: ``Flow`` + +`Detailed log `_ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From 00b62bb77658bd76f5a4399356a0dc82a23f1350 Mon Sep 17 00:00:00 2001 From: Jenkins Date: Wed, 22 Nov 2023 13:07:29 +0000 Subject: [PATCH 12/21] TASK: Update composer manifest for ~8.2.0 See https://jenkins.neos.io/job/flow-release/433/ --- Neos.Eel/composer.json | 2 +- Neos.Flow/composer.json | 26 +++++++++++++------------- Neos.FluidAdaptor/composer.json | 2 +- Neos.Kickstarter/composer.json | 4 ++-- 4 files changed, 17 insertions(+), 17 deletions(-) diff --git a/Neos.Eel/composer.json b/Neos.Eel/composer.json index 6171ed0c38..467cb57508 100644 --- a/Neos.Eel/composer.json +++ b/Neos.Eel/composer.json @@ -5,7 +5,7 @@ "description": "The Embedded Expression Language (Eel) is a building block for creating Domain Specific Languages", "require": { "php": "^8.0", - "neos/flow": "self.version", + "neos/flow": "~8.2.0", "neos/cache": "self.version", "neos/utility-unicode": "self.version", "neos/utility-objecthandling": "self.version" diff --git a/Neos.Flow/composer.json b/Neos.Flow/composer.json index 21c0eebb17..9eebed3a7f 100644 --- a/Neos.Flow/composer.json +++ b/Neos.Flow/composer.json @@ -14,19 +14,19 @@ "ext-reflection": "*", "ext-xml": "*", - "neos/cache": "self.version", - "neos/eel": "self.version", - "neos/error-messages": "self.version", - "neos/utility-arrays": "self.version", - "neos/utility-files": "self.version", - "neos/utility-mediatypes": "self.version", - "neos/utility-objecthandling": "self.version", - "neos/utility-opcodecache": "self.version", - "neos/utility-schema": "self.version", - "neos/utility-unicode": "self.version", - "neos/flow-log": "self.version", - "neos/http-factories": "self.version", - "neos/utility-pdo": "self.version", + "neos/cache": "~8.2.0", + "neos/eel": "~8.2.0", + "neos/error-messages": "~8.2.0", + "neos/utility-arrays": "~8.2.0", + "neos/utility-files": "~8.2.0", + "neos/utility-mediatypes": "~8.2.0", + "neos/utility-objecthandling": "~8.2.0", + "neos/utility-opcodecache": "~8.2.0", + "neos/utility-schema": "~8.2.0", + "neos/utility-unicode": "~8.2.0", + "neos/flow-log": "~8.2.0", + "neos/http-factories": "~8.2.0", + "neos/utility-pdo": "~8.2.0", "neos/composer-plugin": "^2.0", diff --git a/Neos.FluidAdaptor/composer.json b/Neos.FluidAdaptor/composer.json index f06eccf929..91483a6c86 100644 --- a/Neos.FluidAdaptor/composer.json +++ b/Neos.FluidAdaptor/composer.json @@ -7,7 +7,7 @@ ], "require": { "php": "^8.0", - "neos/flow": "self.version", + "neos/flow": "~8.2.0", "neos/cache": "self.version", "neos/utility-files": "self.version", "neos/utility-objecthandling": "self.version", diff --git a/Neos.Kickstarter/composer.json b/Neos.Kickstarter/composer.json index 4e9f413c83..da870036f4 100644 --- a/Neos.Kickstarter/composer.json +++ b/Neos.Kickstarter/composer.json @@ -5,8 +5,8 @@ "license": "MIT", "require": { "php": "^8.0", - "neos/flow": "self.version", - "neos/fluid-adaptor": "self.version", + "neos/flow": "~8.2.0", + "neos/fluid-adaptor": "~8.2.0", "neos/utility-arrays": "self.version" }, "autoload": { From 028b8043d8a28e9137f54648d6f03148c59e7d75 Mon Sep 17 00:00:00 2001 From: Jenkins Date: Wed, 29 Nov 2023 08:55:26 +0000 Subject: [PATCH 13/21] TASK: Update references [skip ci] --- .../TheDefinitiveGuide/PartV/AnnotationReference.rst | 2 +- .../Documentation/TheDefinitiveGuide/PartV/CommandReference.rst | 2 +- .../PartV/FluidAdaptorViewHelperReference.rst | 2 +- .../Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst | 2 +- .../TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst | 2 +- .../TheDefinitiveGuide/PartV/TypeConverterReference.rst | 2 +- .../TheDefinitiveGuide/PartV/ValidatorReference.rst | 2 +- 7 files changed, 7 insertions(+), 7 deletions(-) diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/AnnotationReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/AnnotationReference.rst index 216c9c642f..5cfc7d5197 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/AnnotationReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/AnnotationReference.rst @@ -3,7 +3,7 @@ Flow Annotation Reference ========================= -This reference was automatically generated from code on 2023-11-22 +This reference was automatically generated from code on 2023-11-29 .. _`Flow Annotation Reference: After`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/CommandReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/CommandReference.rst index b4e8dec2fb..cc31789c98 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/CommandReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/CommandReference.rst @@ -19,7 +19,7 @@ commands that may be available, use:: ./flow help -The following reference was automatically generated from code on 2023-11-22 +The following reference was automatically generated from code on 2023-11-29 .. _`Flow Command Reference: NEOS.FLOW`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/FluidAdaptorViewHelperReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/FluidAdaptorViewHelperReference.rst index 3c7c7db344..2e3d93e38e 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/FluidAdaptorViewHelperReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/FluidAdaptorViewHelperReference.rst @@ -3,7 +3,7 @@ FluidAdaptor ViewHelper Reference ================================= -This reference was automatically generated from code on 2023-11-22 +This reference was automatically generated from code on 2023-11-29 .. _`FluidAdaptor ViewHelper Reference: f:debug`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst index 89214e49b7..0ac6d1e284 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst @@ -3,7 +3,7 @@ Flow Signals Reference ====================== -This reference was automatically generated from code on 2023-11-22 +This reference was automatically generated from code on 2023-11-29 .. _`Flow Signals Reference: AbstractAdvice (``Neos\Flow\Aop\Advice\AbstractAdvice``)`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst index cbf24f9733..b4c3cddc57 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst @@ -3,7 +3,7 @@ TYPO3 Fluid ViewHelper Reference ================================ -This reference was automatically generated from code on 2023-11-22 +This reference was automatically generated from code on 2023-11-29 .. _`TYPO3 Fluid ViewHelper Reference: f:alias`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TypeConverterReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TypeConverterReference.rst index d5899dddcd..81a2ecffea 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TypeConverterReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TypeConverterReference.rst @@ -3,7 +3,7 @@ Flow TypeConverter Reference ============================ -This reference was automatically generated from code on 2023-11-22 +This reference was automatically generated from code on 2023-11-29 .. _`Flow TypeConverter Reference: ArrayConverter`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ValidatorReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ValidatorReference.rst index cb0214b480..7fb78d4c97 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ValidatorReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ValidatorReference.rst @@ -3,7 +3,7 @@ Flow Validator Reference ======================== -This reference was automatically generated from code on 2023-11-22 +This reference was automatically generated from code on 2023-11-29 .. _`Flow Validator Reference: AggregateBoundaryValidator`: From 0031c4b145cd0ab9822c5c8464d7930c690179de Mon Sep 17 00:00:00 2001 From: Karsten Dambekalns Date: Wed, 29 Nov 2023 14:41:15 +0100 Subject: [PATCH 14/21] TASK: Add PHP 8.3 to build workflow matrix --- .github/workflows/build.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 4c62d0d77e..b44dc37bb9 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -16,7 +16,7 @@ jobs: strategy: fail-fast: false matrix: - php-versions: ['8.0', '8.1', '8.2'] + php-versions: ['8.0', '8.1', '8.2', '8.3'] # see https://mariadb.com/kb/en/mariadb-server-release-dates/ # this should be a current release, e.g. the LTS version mariadb-versions: ['10.6'] From 3e9652778fbcca9be24b93ed3ebf10f81073cb72 Mon Sep 17 00:00:00 2001 From: Karsten Dambekalns Date: Wed, 29 Nov 2023 17:25:10 +0100 Subject: [PATCH 15/21] TASK: Fix serialized string in DependencyInjectionTest --- .../Functional/ObjectManagement/DependencyInjectionTest.php | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Neos.Flow/Tests/Functional/ObjectManagement/DependencyInjectionTest.php b/Neos.Flow/Tests/Functional/ObjectManagement/DependencyInjectionTest.php index cdc173f232..1290db41c7 100644 --- a/Neos.Flow/Tests/Functional/ObjectManagement/DependencyInjectionTest.php +++ b/Neos.Flow/Tests/Functional/ObjectManagement/DependencyInjectionTest.php @@ -93,7 +93,7 @@ public function propertiesAreReinjectedIfTheObjectIsUnserialized() $singletonA = $this->objectManager->get(Fixtures\SingletonClassA::class); - $prototypeA = unserialize('O:' . strlen($className) . ':"' . $className . '":0:{};'); + $prototypeA = unserialize('O:' . strlen($className) . ':"' . $className . '":0:{}'); self::assertSame($singletonA, $prototypeA->getSingletonA()); } From d6e37fbc4db8e6cf9892b5f44f2fee8755fae222 Mon Sep 17 00:00:00 2001 From: Karsten Dambekalns Date: Wed, 29 Nov 2023 17:26:04 +0100 Subject: [PATCH 16/21] TASK: Fix serialized string in FrameworkTest --- Neos.Flow/Tests/Functional/Aop/FrameworkTest.php | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Neos.Flow/Tests/Functional/Aop/FrameworkTest.php b/Neos.Flow/Tests/Functional/Aop/FrameworkTest.php index 4f9019aa04..130d7e277e 100644 --- a/Neos.Flow/Tests/Functional/Aop/FrameworkTest.php +++ b/Neos.Flow/Tests/Functional/Aop/FrameworkTest.php @@ -95,7 +95,7 @@ public function withinPointcutsAlsoAcceptClassNames() public function adviceInformationIsAlsoBuiltWhenTheTargetClassIsUnserialized() { $className = Fixtures\TargetClass01::class; - $targetClass = unserialize('O:' . strlen($className) . ':"' . $className . '":0:{};'); + $targetClass = unserialize('O:' . strlen($className) . ':"' . $className . '":0:{}'); self::assertSame('Hello, me', $targetClass->greet('Flow')); } From b6bed6e0033752d1b5778965857a4824466d6fbf Mon Sep 17 00:00:00 2001 From: Jenkins Date: Thu, 30 Nov 2023 09:21:56 +0000 Subject: [PATCH 17/21] TASK: Update references [skip ci] --- .../TheDefinitiveGuide/PartV/AnnotationReference.rst | 2 +- .../Documentation/TheDefinitiveGuide/PartV/CommandReference.rst | 2 +- .../PartV/FluidAdaptorViewHelperReference.rst | 2 +- .../Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst | 2 +- .../TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst | 2 +- .../TheDefinitiveGuide/PartV/TypeConverterReference.rst | 2 +- .../TheDefinitiveGuide/PartV/ValidatorReference.rst | 2 +- 7 files changed, 7 insertions(+), 7 deletions(-) diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/AnnotationReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/AnnotationReference.rst index 216c9c642f..a02aa7160b 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/AnnotationReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/AnnotationReference.rst @@ -3,7 +3,7 @@ Flow Annotation Reference ========================= -This reference was automatically generated from code on 2023-11-22 +This reference was automatically generated from code on 2023-11-30 .. _`Flow Annotation Reference: After`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/CommandReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/CommandReference.rst index adf53ed733..3c60d71d6e 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/CommandReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/CommandReference.rst @@ -19,7 +19,7 @@ commands that may be available, use:: ./flow help -The following reference was automatically generated from code on 2023-11-22 +The following reference was automatically generated from code on 2023-11-30 .. _`Flow Command Reference: NEOS.FLOW`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/FluidAdaptorViewHelperReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/FluidAdaptorViewHelperReference.rst index 3c7c7db344..217f774455 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/FluidAdaptorViewHelperReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/FluidAdaptorViewHelperReference.rst @@ -3,7 +3,7 @@ FluidAdaptor ViewHelper Reference ================================= -This reference was automatically generated from code on 2023-11-22 +This reference was automatically generated from code on 2023-11-30 .. _`FluidAdaptor ViewHelper Reference: f:debug`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst index 39a4e750d1..a3ac68eeb1 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst @@ -3,7 +3,7 @@ Flow Signals Reference ====================== -This reference was automatically generated from code on 2023-11-22 +This reference was automatically generated from code on 2023-11-30 .. _`Flow Signals Reference: AbstractAdvice (``Neos\Flow\Aop\Advice\AbstractAdvice``)`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst index cbf24f9733..40b3e90fb2 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst @@ -3,7 +3,7 @@ TYPO3 Fluid ViewHelper Reference ================================ -This reference was automatically generated from code on 2023-11-22 +This reference was automatically generated from code on 2023-11-30 .. _`TYPO3 Fluid ViewHelper Reference: f:alias`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TypeConverterReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TypeConverterReference.rst index d5899dddcd..cdfe673a5e 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TypeConverterReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TypeConverterReference.rst @@ -3,7 +3,7 @@ Flow TypeConverter Reference ============================ -This reference was automatically generated from code on 2023-11-22 +This reference was automatically generated from code on 2023-11-30 .. _`Flow TypeConverter Reference: ArrayConverter`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ValidatorReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ValidatorReference.rst index cb0214b480..905185a770 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ValidatorReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ValidatorReference.rst @@ -3,7 +3,7 @@ Flow Validator Reference ======================== -This reference was automatically generated from code on 2023-11-22 +This reference was automatically generated from code on 2023-11-30 .. _`Flow Validator Reference: AggregateBoundaryValidator`: From db8f06ea53cc64773581dbb43017c5cf107eea0b Mon Sep 17 00:00:00 2001 From: Denny Lubitz Date: Tue, 5 Dec 2023 17:07:58 +0100 Subject: [PATCH 18/21] Revert "BUGFIX: Sanitize uploaded svg files for suspicious contents" This reverts commit a1642ef31f19a974f34a302c98c13c77b6422ba1. --- .../ResourceManagement/ResourceManager.php | 71 ++----------------- Neos.Flow/composer.json | 3 +- .../Classes/MediaTypes.php | 15 ---- .../Tests/Unit/MediaTypesTest.php | 16 ----- composer.json | 1 - 5 files changed, 6 insertions(+), 100 deletions(-) diff --git a/Neos.Flow/Classes/ResourceManagement/ResourceManager.php b/Neos.Flow/Classes/ResourceManagement/ResourceManager.php index ead47c4390..47b10ec27e 100644 --- a/Neos.Flow/Classes/ResourceManagement/ResourceManager.php +++ b/Neos.Flow/Classes/ResourceManagement/ResourceManager.php @@ -11,12 +11,10 @@ * source code. */ -use enshrined\svgSanitize\Sanitizer; use Neos\Flow\Annotations as Flow; use Neos\Flow\Log\Utility\LogEnvironment; use Neos\Flow\ObjectManagement\ObjectManagerInterface; use Neos\Flow\Persistence\PersistenceManagerInterface; -use Neos\Utility\MediaTypes; use Neos\Utility\ObjectAccess; use Neos\Flow\ResourceManagement\Storage\StorageInterface; use Neos\Flow\ResourceManagement\Storage\WritableStorageInterface; @@ -162,30 +160,14 @@ public function importResource($source, $collectionName = ResourceManager::DEFAU $collection = $this->collections[$collectionName]; try { - if (is_resource($source)) { - $mediaType = MediaTypes::getMediaTypeFromResource($source); - if ($this->isSanitizingRequired($mediaType)) { - $content = stream_get_contents($source); - $resource = $this->importResourceFromContent($content, '', $collectionName, $forcedPersistenceObjectIdentifier); - } else { - $resource = $collection->importResource($source); - } - } else { - $resource = fopen($source, 'rb'); - $mediaType = MediaTypes::getMediaTypeFromResource($resource); - fclose($resource); - if ($this->isSanitizingRequired($mediaType)) { - $content = file_get_contents($source); - $resource = $this->importResourceFromContent($content, '', $collectionName, $forcedPersistenceObjectIdentifier); - } else { - $resource = $collection->importResource($source); - } - $pathInfo = UnicodeFunctions::pathinfo($source); - $resource->setFilename($pathInfo['basename']); - } + $resource = $collection->importResource($source); if ($forcedPersistenceObjectIdentifier !== null) { ObjectAccess::setProperty($resource, 'Persistence_Object_Identifier', $forcedPersistenceObjectIdentifier, true); } + if (!is_resource($source)) { + $pathInfo = UnicodeFunctions::pathinfo($source); + $resource->setFilename($pathInfo['basename']); + } } catch (Exception $exception) { throw new Exception(sprintf('Importing a file into the resource collection "%s" failed: %s', $collectionName, $exception->getMessage()), 1375197120, $exception); } @@ -224,11 +206,6 @@ public function importResourceFromContent($content, $filename, $collectionName = throw new Exception(sprintf('Tried to import a file into the resource collection "%s" but no such collection exists. Please check your settings and the code which triggered the import.', $collectionName), 1380878131); } - $mediaType = MediaTypes::getMediaTypeFromFileContent($content); - if ($this->isSanitizingRequired($mediaType)) { - $content = $this->sanitizeImportedFileContent($mediaType, $content, $filename); - } - /* @var CollectionInterface $collection */ $collection = $this->collections[$collectionName]; @@ -631,44 +608,6 @@ protected function initializeCollections() } } - /** - * Decide weather the given media-type has to be sanitized - * for now this only checks svg file to solve the issue here https://nvd.nist.gov/vuln/detail/CVE-2023-37611 - * - * @todo create a feature from this and allow to register code for sanitizing file content before importing - */ - protected function isSanitizingRequired(string $mediaType): bool - { - return $mediaType === 'image/svg+xml'; - } - - /** - * Sanitize file content and remove content that is suspicious - * for now this only checks svg file to solve the issue here https://nvd.nist.gov/vuln/detail/CVE-2023-37611 - * - * @todo create a feature from this and allow to register code for sanitizing file content before importing - */ - protected function sanitizeImportedFileContent(string $mediaType, string $content, $filename = ''): string - { - if ($mediaType === 'image/svg+xml') { - // @todo: Simplify again when https://github.com/darylldoyle/svg-sanitizer/pull/90 is merged and released. - $previousXmlErrorHandling = libxml_use_internal_errors(true); - $sanitizer = new Sanitizer(); - $sanitizedContent = $sanitizer->sanitize($content); - libxml_clear_errors(); - libxml_use_internal_errors($previousXmlErrorHandling); - $issues = $sanitizer->getXmlIssues(); - if ($issues && count($issues) > 0) { - if ($sanitizedContent === false) { - throw new Exception('Sanitizing of suspicious file "' . $filename . '" failed during import.', 1695395560); - } - $content = $sanitizedContent; - $this->logger->warning(sprintf('Imported file "%s" contained suspicious content and was sanitized.', $filename), $issues); - } - } - return $content; - } - /** * Prepare an uploaded file to be imported as resource object. Will check the validity of the file, * move it outside of upload folder if open_basedir is enabled and check the filename. diff --git a/Neos.Flow/composer.json b/Neos.Flow/composer.json index 2e28c2f73f..a83b2b3b19 100644 --- a/Neos.Flow/composer.json +++ b/Neos.Flow/composer.json @@ -51,8 +51,7 @@ "composer/composer": "^2.2.8", - "egulias/email-validator": "^2.1.17 || ^3.0", - "enshrined/svg-sanitize": "^0.16.0" + "egulias/email-validator": "^2.1.17 || ^3.0" }, "require-dev": { "vimeo/psalm": "~4.30.0", diff --git a/Neos.Utility.MediaTypes/Classes/MediaTypes.php b/Neos.Utility.MediaTypes/Classes/MediaTypes.php index 8240e549fe..249d21ced8 100644 --- a/Neos.Utility.MediaTypes/Classes/MediaTypes.php +++ b/Neos.Utility.MediaTypes/Classes/MediaTypes.php @@ -1830,21 +1830,6 @@ public static function getMediaTypeFromFileContent(string $fileContent): string return isset(self::$mediaTypeToFileExtension[$mediaType]) ? $mediaType : 'application/octet-stream'; } - /** - * Returns a Media Type based on the given resource - * - * @param resource $resource The resource to determine the media type from - * @return string The IANA Internet Media Type - */ - public static function getMediaTypeFromResource($resource): string - { - if (!is_resource($resource)) { - throw new \TypeError('Argument "resource" has to be a resource'); - } - $mediaType = self::trimMediaType(mime_content_type($resource)); - return isset(self::$mediaTypeToFileExtension[$mediaType]) ? $mediaType : 'application/octet-stream'; - } - /** * Returns the primary filename extension based on the given Media Type. * diff --git a/Neos.Utility.MediaTypes/Tests/Unit/MediaTypesTest.php b/Neos.Utility.MediaTypes/Tests/Unit/MediaTypesTest.php index d5b1ad93c8..89b1f9d73c 100644 --- a/Neos.Utility.MediaTypes/Tests/Unit/MediaTypesTest.php +++ b/Neos.Utility.MediaTypes/Tests/Unit/MediaTypesTest.php @@ -68,22 +68,6 @@ public function getMediaTypeFromFileContent(string $filename, string $expectedMe self::assertSame($expectedMediaType, MediaTypes::getMediaTypeFromFileContent($fileContent)); } - /** - * @test - * @dataProvider filesAndMediaTypes - */ - public function getMediaTypeFromResource(string $filename, string $expectedMediaType) - { - $filePath = __DIR__ . '/Fixtures/' . $filename; - $resource = is_file($filePath) ? fopen($filePath, 'rb') : fopen('data://text/plain,', 'rb'); - if ($resource !== false) { - self::assertSame($expectedMediaType, MediaTypes::getMediaTypeFromResource($resource)); - fclose($resource); - } else { - $this->fail('fixture ' . $filePath . ' could not be read'); - } - } - /** * Data Provider */ diff --git a/composer.json b/composer.json index 3413f3a310..855973c080 100644 --- a/composer.json +++ b/composer.json @@ -32,7 +32,6 @@ "neos/composer-plugin": "^2.0", "composer/composer": "^2.2.8", "egulias/email-validator": "^2.1.17 || ^3.0", - "enshrined/svg-sanitize": "^0.16.0", "typo3fluid/fluid": "~2.7.0", "guzzlehttp/psr7": "^1.8.4", "ext-mbstring": "*" From 41db681f75b2a98798730a0a218d85bd4c206538 Mon Sep 17 00:00:00 2001 From: Karsten Dambekalns Date: Tue, 5 Dec 2023 19:39:01 +0100 Subject: [PATCH 19/21] BUGFIX: Assume content exists, if stream size is unknown If a PSR7 stream is returned from an `ActionController` action, no content arrives at the client, if the stream has an unknown size. Why is that? Because the check in our `ActionResponse` in `hasContent()` is implemented like this: $this->content->getSize() > 0 If a stream returns `null` here, because the size is unknown, we should assume content exists... --- Neos.Flow/Classes/Mvc/ActionResponse.php | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/Neos.Flow/Classes/Mvc/ActionResponse.php b/Neos.Flow/Classes/Mvc/ActionResponse.php index ded3df3dfc..f52ec1a0ac 100644 --- a/Neos.Flow/Classes/Mvc/ActionResponse.php +++ b/Neos.Flow/Classes/Mvc/ActionResponse.php @@ -345,6 +345,7 @@ public function buildHttpResponse(): ResponseInterface */ private function hasContent(): bool { - return $this->content->getSize() > 0; + $contentSize = $this->content->getSize(); + return $contentSize === null || $contentSize > 0; } } From 7da588d5b2dac78c3132a3c0a5150b6cb8b9e297 Mon Sep 17 00:00:00 2001 From: Karsten Dambekalns Date: Wed, 6 Dec 2023 10:30:05 +0100 Subject: [PATCH 20/21] TASK: Check size before setting Content-Length --- Neos.Flow/Classes/Http/Helper/ResponseInformationHelper.php | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Neos.Flow/Classes/Http/Helper/ResponseInformationHelper.php b/Neos.Flow/Classes/Http/Helper/ResponseInformationHelper.php index 9c7fdc0a57..1858a2b88e 100644 --- a/Neos.Flow/Classes/Http/Helper/ResponseInformationHelper.php +++ b/Neos.Flow/Classes/Http/Helper/ResponseInformationHelper.php @@ -228,7 +228,7 @@ public static function makeStandardsCompliant(ResponseInterface $response, Reque } } - if (!$response->hasHeader('Content-Length')) { + if (!$response->hasHeader('Content-Length') && $response->getBody()->getSize() !== null) { $response = $response->withHeader('Content-Length', $response->getBody()->getSize()); } From c30144369bb40719cf817d6e4cbc4860cf96cb17 Mon Sep 17 00:00:00 2001 From: Jenkins Date: Wed, 6 Dec 2023 10:11:59 +0000 Subject: [PATCH 21/21] TASK: Update references [skip ci] --- .../TheDefinitiveGuide/PartV/AnnotationReference.rst | 2 +- .../Documentation/TheDefinitiveGuide/PartV/CommandReference.rst | 2 +- .../PartV/FluidAdaptorViewHelperReference.rst | 2 +- .../Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst | 2 +- .../TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst | 2 +- .../TheDefinitiveGuide/PartV/TypeConverterReference.rst | 2 +- .../TheDefinitiveGuide/PartV/ValidatorReference.rst | 2 +- 7 files changed, 7 insertions(+), 7 deletions(-) diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/AnnotationReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/AnnotationReference.rst index 5cfc7d5197..5f7c21abd5 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/AnnotationReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/AnnotationReference.rst @@ -3,7 +3,7 @@ Flow Annotation Reference ========================= -This reference was automatically generated from code on 2023-11-29 +This reference was automatically generated from code on 2023-12-06 .. _`Flow Annotation Reference: After`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/CommandReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/CommandReference.rst index cc31789c98..a4740d37de 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/CommandReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/CommandReference.rst @@ -19,7 +19,7 @@ commands that may be available, use:: ./flow help -The following reference was automatically generated from code on 2023-11-29 +The following reference was automatically generated from code on 2023-12-06 .. _`Flow Command Reference: NEOS.FLOW`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/FluidAdaptorViewHelperReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/FluidAdaptorViewHelperReference.rst index 2e3d93e38e..2a2d2520d5 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/FluidAdaptorViewHelperReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/FluidAdaptorViewHelperReference.rst @@ -3,7 +3,7 @@ FluidAdaptor ViewHelper Reference ================================= -This reference was automatically generated from code on 2023-11-29 +This reference was automatically generated from code on 2023-12-06 .. _`FluidAdaptor ViewHelper Reference: f:debug`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst index 0ac6d1e284..59fd978a61 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/SignalsReference.rst @@ -3,7 +3,7 @@ Flow Signals Reference ====================== -This reference was automatically generated from code on 2023-11-29 +This reference was automatically generated from code on 2023-12-06 .. _`Flow Signals Reference: AbstractAdvice (``Neos\Flow\Aop\Advice\AbstractAdvice``)`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst index b4c3cddc57..0e262744b7 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TYPO3FluidViewHelperReference.rst @@ -3,7 +3,7 @@ TYPO3 Fluid ViewHelper Reference ================================ -This reference was automatically generated from code on 2023-11-29 +This reference was automatically generated from code on 2023-12-06 .. _`TYPO3 Fluid ViewHelper Reference: f:alias`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TypeConverterReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TypeConverterReference.rst index 81a2ecffea..e5092fe781 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TypeConverterReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/TypeConverterReference.rst @@ -3,7 +3,7 @@ Flow TypeConverter Reference ============================ -This reference was automatically generated from code on 2023-11-29 +This reference was automatically generated from code on 2023-12-06 .. _`Flow TypeConverter Reference: ArrayConverter`: diff --git a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ValidatorReference.rst b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ValidatorReference.rst index 7fb78d4c97..333d3f90d4 100644 --- a/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ValidatorReference.rst +++ b/Neos.Flow/Documentation/TheDefinitiveGuide/PartV/ValidatorReference.rst @@ -3,7 +3,7 @@ Flow Validator Reference ======================== -This reference was automatically generated from code on 2023-11-29 +This reference was automatically generated from code on 2023-12-06 .. _`Flow Validator Reference: AggregateBoundaryValidator`: