[!!!][FEATURE] Improve ValidatorTask

The ext:linkvalidator ValidatorTask has been improved. Therefore
the task was switched from marker based templates to FluidEmail.

The mail content was extended for a proper list of broken links
along with all necessary information such as record id, page id,
and so on. This allows editors now to easily access the records
with broken links without having to visit the backend module first.

Because of heavy refactoring and introduction of a new PSR-14
event, extension authors are now able to fully customize the mail
content and the configuration (such as to, from, subject) dynamically
(e.g. adding another recipient only on production context).

Both the task and its field provider got added strict types. The
task however currently still needs some type casts for backwards
compatibilty as the tasks state is stored as serialized string in
the database.

The whole link analyzing process was moved into the dedicated class
`LinkAnalyzerResult`.

A new field `languages` is introduced which allows to restrict
the report to the defined languages which is handy if different
tasks should be used (e.g. editors responsible for only one
localization).

Resolves: #29342
Releases: master
Change-Id: Ibe070038ef66f876de38e47d6f82f4f090af96ce
Reviewed-on: https://review.typo3.org/c/Packages/TYPO3.CMS/+/65549
Tested-by: Sybille Peters <sypets@gmx.de>
Tested-by: TYPO3com <noreply@typo3.com>
Tested-by: Daniel Goerz <daniel.goerz@posteo.de>
Reviewed-by: Sybille Peters <sypets@gmx.de>
Reviewed-by: Daniel Goerz <daniel.goerz@posteo.de>

typo3/sysext/core/Documentation/Changelog/master/Breaking-29342-ImproveValidatorTask.rst

@@ -0,0 +1,69 @@
+.. include:: ../../Includes.txt
+
+========================================
+Breaking: #29342 - Improve ValidatorTask
+========================================
+
+See :issue:`29342`
+
+Description
+===========
+
+In TYPO3 version 10 `ext:linkvalidator` has been improved a lot. The
+:php:`\TYPO3\CMS\Linkvalidator\Task\ValidatorTask` - a scheduler task for reporting
+broken links via email has been refactored now.
+
+The old marker templates have been replaced by `FluidEmail`. A Fluid templates is now
+used for generating the report email. The marker template has been removed completely
+along with corresponding functionality.
+
+The following property of the :php:`ValidatorTask` class has been removed:
+
+* :php:`$emailTemplateFile`
+
+The following hooks have been removed and won't be executed anymore:
+
+* :php:`$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['linkvalidator']['reportEmailMarkers']`
+* :php:`$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['linkvalidator']['buildMailMarkers']`
+
+The following properties of the :php:`ValidatorTask` class have changed their type:
+
+* :php:`$page` is now :php`int`
+* :php:`$depth` is now :php`int`
+* :php:`$emailOnBrokenLinkOnly` is now :php:`bool`
+* :php:`$configuration` is now :php:`string`
+
+
+Impact
+======
+
+It is no longer possible to set a custom marker based template file with
+`emailTemplateFile`. Instead, the new field `emailTemplateName` can be used to
+specify a Fluid template file, see Migration section below.
+
+
+Affected Installations
+======================
+
+All installations which use:
+
+* the task by providing a custom template file.
+* one of the hooks mentioned above.
+
+
+Migration
+=========
+
+If you currently don't use a custom template or one of the hooks mentioned above,
+you don't need to migrate anything.
+
+Otherwiese you have to provide your custom templates using the new field
+`emailTemplateName` in the task configuration and adding your custom template
+path to :php:`$GLOBALS['TYPO3_CONF_VARS']['MAIL']['templateRootPaths']`.
+
+Furthermore use the new PSR-14 event
+:php:`\TYPO3\CMS\Linkvalidator\Event\ModifyValidatorTaskEmailEvent` to adjust the
+:php:`\TYPO3\CMS\Linkvalidator\Result\LinkAnalyzerResult` along with the `FluidEmail`
+object.
+
+.. index:: Backend, CLI, NotScanned, ext:linkvalidator

typo3/sysext/core/Documentation/Changelog/master/Feature-29342-ImproveValidatorTask.rst

@@ -0,0 +1,134 @@
+.. include:: ../../Includes.txt
+
+=======================================
+Feature: #29342 - Improve ValidatorTask
+=======================================
+
+See :issue:`29342`
+
+Description
+===========
+
+The :php:`\TYPO3\CMS\Linkvalidator\Task\ValidatorTask` - scheduler task for
+reporting broken links via email which still used marker templates, has been
+improved. This is achieved by switching to `FluidEmail`, extending the task
+configuration along with the mails content for more detailed reports, code
+refactoring and introduction of strict types for both
+:php:`\TYPO3\CMS\Linkvalidator\Task\ValidatorTask` and
+:php:`\TYPO3\CMS\Linkvalidator\Task\ValidatorTaskAdditionalFieldProvider`.
+
+The task configuration has got the following new fields:
+
+* `languages`: Comma separated list of language uids
+* `emailTemplateName`: Name of the fluid templates
+
+With `languages` it's now possible to limit the report to specified system
+languages. This is useful if multiple tasks for different groups of recipients
+should be registered.
+
+With `emailTemplateName` it's possible to use different custom templates for each
+task. Therefore the template path must be set in
+:php:`$GLOBALS['TYPO3_CONF_VARS']['MAIL']['templateRootPaths']`. Additionally
+the used `SystemEmail` layout can also be changed by setting your custom layout
+path in :php:`$GLOBALS['TYPO3_CONF_VARS']['MAIL']['layoutRootPaths']`. If no
+`emailTemplateName` is set or the specified input is invalid, the task
+automatically adds the default template name on task execution.
+
+Furthermore the following new PSR-14 event has been introduced:
+
+* :php:`\TYPO3\CMS\Linkvalidator\Event\ModifyValidatorTaskEmailEvent`
+
+This event can be used to manipulate the :php:`\TYPO3\CMS\Linkvalidator\Result\LinkAnalyzerResult`,
+which contains all information from the linkvalidator API. Also the `FluidEmail`
+object can be adjusted here. This allows to e.g. pass additional information to
+the view by using :php:`$fluidEmail->assign()` or dynamically adding mail information
+such as the recievers list. The added values in the event take precedence over the
+`modTSconfig` configuration. Therefore the event contains the merged `modTSconfig`
+to access further information about the actuall configuration of the task when
+assigning new values to `FluidEmail`.
+
+Note: As it's now also possible to set the recipient addresses dynamically using
+the event, the `email` field in the task configuration can remain empty but will
+be added, if defined, on top of already defined recipients from the event. All
+other values such as `subject`, `from` or `replyTo` will only be set according to
+`modTSconfig` if not already defined through the event.
+
+An example implementation of the PSR-14 event:
+
+.. code-block:: php
+
+   <?php
+   declare(strict_types=1);
+   namespace Vendor\Extension\EventListener;
+
+   use TYPO3\CMS\Linkvalidator\Event\ModifyValidatorTaskEmailEvent;
+
+   class ModifyValidatorTaskEmail
+   {
+      public function modify(ModifyValidatorTaskEmailEvent $event): void
+      {
+         $linkAnalyzerResult = $event->getLinkAnalyzerResult();
+         $fluidEmail = $event->getFluidEmail();
+         $modTSconfig = $event->getModTSconfig();
+
+         if ($modTSconfig['mail.']['fromname] === 'John Smith') {
+            $fluidEmail->assign('myAdditionalVariable', 'foobar');
+         }
+
+         $fluidEmail->subject(
+            $linkAnalyzerResult->getTotalBrokenLinksCount() . ' new broken links'
+         );
+
+         $fluidEmail->to(new Address('custom@mail.com'));
+      }
+   }
+
+.. code-block:: yaml
+
+   Vendor\Extension\EventListener\ModifyValidatorTaskEmail:
+     tags:
+      - name: event.listener
+        identifier: 'modify-validation-task-email'
+        event: TYPO3\CMS\Linkvalidator\Event\ModifyValidatorTaskEmailEvent
+        method: 'modify'
+
+The :php:`\TYPO3\CMS\Linkvalidator\Result\LinkAnalyzerResult` contains following
+information by default:
+
+* :php:`$oldBrokenLinkCounts`: Amount of broken links from the last run, separated by type (e.g. all, internal)
+* :php:`$newBrokenLinkCounts`: Amount of broken links from this run, separated by type (e.g. all, internal)
+* :php:`$brokenLinks`: List of broken links with the raw database data
+* :php:`$differentToLastResult`: Whether the broken links count changed
+
+The :php:`brokenLinks` property gets further processed internally to provide additional
+information for the email. Following additional information is provided by default:
+
+* :php:`full_record`: The full record, the broken link was found in (e.g. pages or tt_content)
+* :php:`record_title`: Value of :php:`full_record` title field
+* :php:`record_type`: The title of the record type (e.g. "Page" or "Page Content")
+* :php:`language_code`: The language code of the broken link
+* :php:`real_pid`: The real page id of the record the broken link was found in
+* :php:`page_record`: The whole page row of records parent page
+
+More can be added using the PSR-14 event.
+
+Additionally to the already existing content the email now includes a list of all
+broken links fetched according to the task configuration. This list consists of
+following columns:
+
+* `Record`: The :php:`record_uid` and :php:`record_title`
+* `Language`: The :php:`language_code` and language id
+* `Page`: The :php:`real_pid` and :php:`page_record.title` of the parent page
+* `Record Type`: The :php:`record_type`
+* `Link Target`: The :php:`target`
+* `Link Type`: Type of the broken link (Either `internal`, `external` or `file`)
+
+
+Impact
+======
+
+The main improvement is the more detailed report which is delivered by `FluidEmail`,
+using the default `SystemEmail` layout. Along with the new PSR-14 event, extension authors
+are now able to fully customize the content of the report as needed.
+
+.. index:: Backend, CLI, NotScanned, ext:linkvalidator

typo3/sysext/core/Resources/Private/Layouts/SystemEmail.html

@@ -228,6 +228,18 @@
             font-weight:normal;
             text-decoration:underline;
         }
+        .bodyContainer table.statistics th,
+        .bodyContainer table.statistics td {
+            padding: 0 5px;
+        }
+        .bodyContainer table.statistics th:first-child,
+        .bodyContainer table.statistics td:first-child {
+            padding-left: 0;
+        }
+        .bodyContainer table.statistics th:last-child,
+        .bodyContainer table.statistics td:last-child {
+            padding-right: 0;
+        }
         #templateFooter{
             background-color:#f2f2f2;
             background-image:none;

typo3/sysext/linkvalidator/Classes/Event/ModifyValidatorTaskEmailEvent.php

@@ -0,0 +1,64 @@
+<?php
+
+declare(strict_types=1);
+
+/*
+ * This file is part of the TYPO3 CMS project.
+ *
+ * It is free software; you can redistribute it and/or modify it under
+ * the terms of the GNU General Public License, either version 2
+ * of the License, or any later version.
+ *
+ * For the full copyright and license information, please read the
+ * LICENSE.txt file that was distributed with this source code.
+ *
+ * The TYPO3 project - inspiring people to share!
+ */
+
+namespace TYPO3\CMS\Linkvalidator\Event;
+
+use TYPO3\CMS\Core\Mail\FluidEmail;
+use TYPO3\CMS\Linkvalidator\Result\LinkAnalyzerResult;
+
+/**
+ * Allows to process and modify the LinkAnalyzer result and FluidEmail object
+ */
+final class ModifyValidatorTaskEmailEvent
+{
+    /**
+     * @var LinkAnalyzerResult
+     */
+    private $linkAnalyzerResult;
+
+    /**
+     * @var FluidEmail
+     */
+    private $fluidEmail;
+
+    /**
+     * @var array
+     */
+    private $modTSconfig;
+
+    public function __construct(LinkAnalyzerResult $linkAnalyzerResult, FluidEmail $fluidEmail, array $modTSconfig)
+    {
+        $this->linkAnalyzerResult = $linkAnalyzerResult;
+        $this->fluidEmail = $fluidEmail;
+        $this->modTSconfig = $modTSconfig;
+    }
+
+    public function getLinkAnalyzerResult(): LinkAnalyzerResult
+    {
+        return $this->linkAnalyzerResult;
+    }
+
+    public function getFluidEmail(): FluidEmail
+    {
+        return $this->fluidEmail;
+    }
+
+    public function getModTSconfig(): array
+    {
+        return $this->modTSconfig;
+    }
+}

typo3/sysext/linkvalidator/Classes/Repository/BrokenLinkRepository.php

@@ -195,16 +195,52 @@ public function removeAllBrokenLinksOfRecordsOnPageIds(array $pageIds, array $li
      * @param string[] $linkTypes Link types to validate
      * @param string[] $searchFields table => [fields1, field2, ...], ... : fields in which linkvalidator should
      *   search for broken links
+     * @param int[] $languages Allowed languages
      * @return array
      */
-    public function getAllBrokenLinksForPages(array $pageIds, array $linkTypes, array $searchFields = []): array
-    {
+    public function getAllBrokenLinksForPages(
+        array $pageIds,
+        array $linkTypes,
+        array $searchFields = [],
+        array $languages = []
+    ): array {
         $queryBuilder = GeneralUtility::makeInstance(ConnectionPool::class)
             ->getQueryBuilderForTable(self::TABLE);
         if (!$GLOBALS['BE_USER']->isAdmin()) {
             $queryBuilder->getRestrictions()
                 ->add(GeneralUtility::makeInstance(EditableRestriction::class, $searchFields, $queryBuilder));
         }
+
+        $constraints = [
+            $queryBuilder->expr()->orX(
+                $queryBuilder->expr()->andX(
+                    $queryBuilder->expr()->in(
+                        'record_uid',
+                        $queryBuilder->createNamedParameter($pageIds, Connection::PARAM_INT_ARRAY)
+                    ),
+                    $queryBuilder->expr()->eq('table_name', $queryBuilder->createNamedParameter('pages'))
+                ),
+                $queryBuilder->expr()->andX(
+                    $queryBuilder->expr()->in(
+                        'record_pid',
+                        $queryBuilder->createNamedParameter($pageIds, Connection::PARAM_INT_ARRAY)
+                    ),
+                    $queryBuilder->expr()->neq('table_name', $queryBuilder->createNamedParameter('pages'))
+                )
+            ),
+            $queryBuilder->expr()->in(
+                'link_type',
+                $queryBuilder->createNamedParameter($linkTypes, Connection::PARAM_STR_ARRAY)
+            )
+        ];
+
+        if ($languages !== []) {
+            $constraints[] = $queryBuilder->expr()->in(
+                'language',
+                $queryBuilder->createNamedParameter($languages, Connection::PARAM_INT_ARRAY)
+            );
+        }
+
         $records = $queryBuilder
             ->select(self::TABLE . '.*')
             ->from(self::TABLE)
@@ -214,28 +250,7 @@ public function getAllBrokenLinksForPages(array $pageIds, array $linkTypes, arra
                 'pages',
                 $queryBuilder->expr()->eq('tx_linkvalidator_link.record_pid', $queryBuilder->quoteIdentifier('pages.uid'))
             )
-            ->where(
-                $queryBuilder->expr()->orX(
-                    $queryBuilder->expr()->andX(
-                        $queryBuilder->expr()->in(
-                            'record_uid',
-                            $queryBuilder->createNamedParameter($pageIds, Connection::PARAM_INT_ARRAY)
-                        ),
-                        $queryBuilder->expr()->eq('table_name', $queryBuilder->createNamedParameter('pages'))
-                    ),
-                    $queryBuilder->expr()->andX(
-                        $queryBuilder->expr()->in(
-                            'record_pid',
-                            $queryBuilder->createNamedParameter($pageIds, Connection::PARAM_INT_ARRAY)
-                        ),
-                        $queryBuilder->expr()->neq('table_name', $queryBuilder->createNamedParameter('pages'))
-                    )
-                ),
-                $queryBuilder->expr()->in(
-                    'link_type',
-                    $queryBuilder->createNamedParameter($linkTypes, Connection::PARAM_STR_ARRAY)
-                )
-            )
+            ->where(...$constraints)
             ->orderBy('tx_linkvalidator_link.record_uid')
             ->addOrderBy('tx_linkvalidator_link.uid')
             ->execute()