v1.3.0

Fixed

• Crawler now re-validates response filters against the GET response, not just the HEAD. Servers that return different headers on HEAD and GET (e.g. router URLs whose HEAD reports text/html but whose GET redirects to a PDF) are now correctly rejected.

Added

• BinaryContentFilter: a new filter that rejects responses whose body starts with a known binary signature (PDF, images, archives, ISO BMFF, RIFF, WOFF, etc.) or contains a NUL byte in the leading bytes. Catches non-HTML content that slips past header-based checks and prevents the HTML parser from receiving binary data. Enabled by default in CrawlConfiguration.filters.