[DOCS] Document known issues with ext:linkvalidator

Linkvalidator extension has a couple of known issues,
especially with external link checking.

Those issues are now documented, so integrators can
acknowledge them and implement counter-measures.

Resolves: #101597
Releases: main, 12.4, 11.5
Change-Id: I00dc01243426fc56c21a4bd11815c3009e3dd3ad
Reviewed-on: https://review.typo3.org/c/Packages/TYPO3.CMS/+/80365
Reviewed-by: Stefan B�rk <stefan@buerk.tech>
Tested-by: core-ci <typo3@b13.com>
Tested-by: Stefan B�rk <stefan@buerk.tech>

typo3/sysext/linkvalidator/Documentation/Configuration/Index.rst

@@ -120,11 +120,14 @@ linktypes
 
          file: Check links to files located in your local TYPO3 installation.
 
-         external: Check links to external files.
+         external: Check links to external URLs.
 
          This list may be extended by other extensions providing a linktype
          checker.
 
+         ..  warning::
+             External links can lead to some :ref:`known issues<usagePitfallsExternalLinks>`.
+
    Default
          db,file,external
 

typo3/sysext/linkvalidator/Documentation/Index.rst

@@ -48,6 +48,7 @@ send status mails when broken links are detected.
    Administration/Index
    Configuration/Index
    LargeSites/Index
+   KnownProblems/Index
 
 .. Meta Menu
 

typo3/sysext/linkvalidator/Documentation/Introduction/Index.rst

@@ -32,7 +32,7 @@ It includes the following features:
 
 - The LinkValidator can check all kinds of links. This includes internal
   links to pages and content elements, file links to files in the local
-  file system and external links to files somewhere else in the web.
+  file system and external links to resources somewhere else in the web.
 
 - The LinkValidator checks a number of fields by default, for example
   :sql:`header` and :sql:`bodytext` fields of content elements.

typo3/sysext/linkvalidator/Documentation/KnownProblems/Index.rst

@@ -0,0 +1,49 @@
+.. include:: /Includes.rst.txt
+
+.. _known-problems:
+
+==============
+Known problems
+==============
+
+
+.. _usagePitfallsExternalLinks:
+
+Problems with external links
+============================
+
+The most relevant known problems currently concern "external broken links".
+
+Be polite when crawling external sites: They should not be checked too often.
+The extension currently provides no functionality to limit such requests.
+
+There are (at least) 2 possible counter-measures:
+
+#. Turn off external link checking entirely
+   by removing `external` from Page TSconfig :ref:`linktypes`
+
+#. :ref:`Override the ExternalLinktype class <linktype-implementation>`
+   (in your own extension), to check only specific URLs or exclude specific
+   URLs or handle only specific error types as errors.
+
+
+.. _usagePitfallsFalsePositives:
+
+Falsely reported broken links
+=============================
+
+Linkvalidator may find false negatives when checking links. This can be
+annoying for editors since there is no way to declare them "ok" manually,
+they will be shown to editors over and over again.
+
+There are a couple of scenarios where linkvalidator may show a link as
+broken leading to a misleading result:
+
+*   Automatic server-side external link checking may fail. There are many possible
+    reasons for false negatives in this area like server connectivity issues or
+    funny deny rules on the target system.
+*   Sometimes links may not be broken as such, but are considered "broken" due to
+    HTTP status like 4xx.
+*   The broken link information may be "stale", the link has been checked a while ago
+    and did lead to a negative result due to temporary a system outage or similar, is
+    now working again, but has not been rechecked yet.