v4.0.0: Significantly improved crawl reliability, performance, and result triage

v4.0.0

Version 4.0.0 is a major release that moves the package from CodeQ to NEOSidekick and significantly improves crawl reliability, performance, and result triage.

Breaking changes / Upgrade notes

• The Composer package is now neosidekick/linkchecker.
• The Neos package key changed from CodeQ.LinkChecker to NEOSidekick.LinkChecker.
• PHP namespaces changed from CodeQ\LinkChecker to NEOSidekick\LinkChecker.
• Settings now live below NEOSidekick.LinkChecker.
• The persisted result table is renamed from codeq_linkchecker_domain_model_resultitem to neosidekick_linkchecker_domain_model_resultitem.
• Backend-triggered crawls now use Flowpack.JobQueue.Common, with a Doctrine queue configured by default.

A Flow code migration and Doctrine migrations are included. After upgrading, run:

./flow flow:core:migrate Your.SitePackage --force
./flow doctrine:migrate
./flow flowpack.jobqueue.common:queue:setup NEOSidekick.LinkChecker.Crawl

Backend crawls require a worker, for example:

./flow flowpack.jobqueue.common:job:work NEOSidekick.LinkChecker.Crawl --verbose

NEW FEATURES

• New grouped backend triage UI for reviewing link issues by broken target, source page, domain, status, and impact.
• Link health summary and impact filters to prioritize issues affecting many pages.
• Bulk actions to mark whole groups as fixed or ignore them.
• JobQueue-backed backend crawls with queue status, duplicate-job prevention, and failed job visibility.
• checklinks:audit command for inspecting stored result health, duplicates, ignored rows, and active issues.
• Optional incremental node crawling via --only-changed.
• Optional between-run cache for healthy external links.
• HEAD-first external link checks with GET fallback for faster checking.
• Per-host rate limiting, response byte limits, redirect following, retry/backoff handling, and configurable user agent.
• OK / warning / broken classification to reduce false positives.
• Configurable ignore rules for suppressing known findings.
• CC recipients for email notifications.

Changed

• Backend crawl logic was moved out of the controller into dedicated runner and queue services.
• Link checker results now use stable fingerprints to prevent duplicates.
• Existing duplicate findings are deduplicated by migration.
• Notifications now count only truly broken links, excluding warnings such as auth walls, bot blocks, rate limits, and invalid phone-number hints.
• External redirects are followed by default so working redirected links are no longer reported as broken.
• Transient server errors are revalidated before being persisted.
• Long-running crawls now persist more frequently and clean up crawler/context state to improve stability.
• Result labels, node labels, backend edit links, source links, and status badges were improved in the backend module.
• English and German translations were expanded and corrected.
• README documentation and backend screenshot were updated.

Fixed

• Duplicate link checker results.
• Long-running crawl persistence issues.
• Backend crawl reset status handling.
• Translation mistakes and incomplete HTTP status labels.
• Several result accuracy issues that caused false positives.
• Code style issues from the previous release line.

Removed

• Removed the direct dependency on symfony/polyfill-php80.

Full changelog: v3.1.0...v4.0.0

Version 3.1.0: Neos 8 and PHP 8 support, performance improvements and bugfixes

What's Changed

• BUGFIX: Handle undefined env variable gracefully by @rolandschuetz in #13
• CHORE: Update dependencies by @gradinarufelix in #18
• Improve memory usage
• Fix #21
• Fix bug where a site without a primary domain would result in a type error, by omitting these sites as they have no domain
• Improve verbosity of output in CLI

Full Changelog: v3.0.0...v3.1.0

Version 3.0.0 Improved UI and API

The setting options and commands changed. The backend UI was greatly improved, the email notifications are replaced, retry if connection timed out was added and domains are now automatically detected based on the site's domain.

Contributions:
Thanks to @mhsdesign for his development and to Code Q for sponsoring and open sourcing the package. Concept and UX design by @rolandschuetz

v2.0.0 Support for PHP 8.1

update the library spatie/crawler to v6 since its dependency in the v4 tightenco/collect doesnt work with php8.1 in the required version.

requires us to use at least php 7.4 so this is a breaking change, as we don't support php 7.1 - 7.3 anymore

v1.0.1

Full Changelog: v1.0.0...v1.0.1

v1.0.0 Stable Release

What's Changed

• FEATURE: Add link checker module by @simonschaufi in #1

Full Changelog: https://github.com/code-q-web-factory/CodeQ.LinkChecker/commits/v1.0.0