v2.36.0 — counting facets

v2.36.0

Counting facets.

Bots crawl search and listing pages through every facet combination, such as ?f[0]=type:article&f[1]=tag:a&f[2]=tag:b&f[3]=year:2026. Each combination is an uncacheable faceted search, and people rarely apply more than two or three. Until now no rule could cap the number of facets without a bot getting past it just by writing the URL differently. query_count closes that.

PHP 8.1–8.5 supported. One new rule variable, opt-in. Nothing changes for a config that doesn't use it.

Added

• query_count.<name> and query_count (#440, #441). query_count.<name> is how many values the client sent for one query parameter, as a number, so the existing numeric operators do the rest:

  plugins:
    - plugin: "Kanopi\\Firewall\\Plugins\\Url"
      response: challenge
      enable: true
      metadata:
        name: facet-crawling
      config:
        - type: AND
          rules:
            - "path@starts_with:/search"
            - "query_count.f@greater_than:3"

  It's counted from the raw query string, so every way of sending a value counts the same, in any order and with other parameters in between:

  Query	query_count.f
  f[0]=…&f[1]=…&f[2]=…&f[3]=…	4
  f[]= ×4, sparse keys f[0]/f[7]/…, named keys f[x]/…	4
  f=a&f=b&f=c&f=d (PHP keeps only one)	4
  f%5B0%5D=… ×4 (encoded brackets)	4
  no f	0

  query_count with no name counts every parameter, for a blunt cap. Names are case-sensitive, as the application reads them. Available in URL and Reputation rules and as a rate-limit key component.

Why not before

• query.f resolves to nothing when f arrives as a list. That's deliberate, so contains can't match across values sent separately.
• PHP's parsed query keeps only the last of f=a&f=b.
• The closest rule possible before this, query.f.3@exists, missed sparse keys, named keys and repeated names. A regex on the whole query string missed repeated names, and anything placed between two facets.

Use challenge, not block

A block rule also bans the address, so a person who really did apply four facets would be shut out for the ban period. The new Stop facet crawling recipe uses response: challenge. If you use block, set metadata.record: false to refuse the request without banning. Pair either with an address-keyed rate limit on the search path, for crawlers that stay under the cap but request thousands of combinations.

presets/drupal.yml has a commented-out entry for Search API's f facets. It's off by default, because the right threshold and path depend on the site.

Full changelog: v2.35.1...v2.36.0