v1.26.0: Papalymo Totolymo

[Xe]

Sorry this took so long. A lot of work was done behind the scenes to do mass testing of Anubis on many versions of Google Chrome. After about 10 attempts failed, the current one seems to work, leading to the confidence of being able to cut this release.

This should support Chrome 69 and newer.

• Add option to disable the honeypot.
• Add (data)/clients/google-user-triggered-fetchers.yaml snippet that allows Google-owned user-triggered fetchers (Google Translate's website translation proxy, Google Read Aloud, Google Messages link previews) by their published IP ranges, fixing the infinite challenge loop for visitors using Google Translate (#444)
• Update QwantBot remote addresses range with latest value
• Migrate check-spelling workflow config to cspell
• Add Anubis Kubernetes Operator to the docs (#1675).
• Bump Playwright browser tooling to 1.61.1 and playwright-go to v0.6100.0.
• Add FreeBSD/Windows binaries to the yeetfile.
• Add systemd system extensions to the yeetfile.
• Set an explicit esbuild --target=chrome66 so modern syntax (e.g. optional chaining) is transpiled down. This lowers the minimum supported browser from Chrome 80 to Chrome 66.
• Patch GHSA-6wcg-mqvh-fcvg by containing subrequest logic to Anubis instances in subrequest mode.
• Implement robot9001 style delays on the honeypot feature so that the first hit takes 1 millisecond, the second takes 2, etc.
• Move metrics server configuration to the policy file.
• Expose pprof endpoints on the metrics listener to enable profiling Anubis in production.
• fix: prevent nil pointer panic in challenge validation when threshold rules match during PassChallenge (http: panic serving 127.0.0.1:52184: runtime error: invalid memory address or nil pointer dereference #1463)
• Instruct reverse proxies to not cache error pages.
• Fixed mixed tab/space indentation in Caddy documentation code block
• Improve error messages and fix broken REDIRECT_DOMAINS link in docs (#1193)
• Add Bulgarian locale (#1394)
• Fixed case-sensitivity mismatch in geoipchecker.go
• Use Go's native version stamping instead of a handrolled variant.
• Fix CEL internal errors when iterating headers/query map wrappers by implementing map iterators for HTTPHeaders and URLValues (#1465).
• Enable metrics serving via TLS, including mutual TLS (mTLS).
• Enable HTTP basic auth for the metrics server.
• Fix a bug in the dataset poisoning maze that could allow denial of service #1580.
• Add config option to add ASN to logs/metrics.
• Log weight when issuing challenge.
• Block x.ai's crawler for code review training.
• Gate pprof endpoints behind metrics.debug in the policy file.
• Limit naive honeypot r9k delay to one second.
• Fix an obscure case where adding query values to a subrequest match could cause an invalid rule match when using path based matching for protected resources.
• Anubis now requires Go 1.26 to build.
• Fix an edge case where load average expression values could nil pointer dereference when Anubis just started up.
• Fix an obscure case where Anubis in subrequest mode could allow redirects to invalid domains with strange instructions.
• Fix path_regex and CEL path rules not matching when using Traefik forwardAuth middleware. Anubis now checks X-Forwarded-Uri (Traefik) in addition to X-Original-URI (nginx) when resolving the request path in subrequest mode (#1628).
• Validate bounds in the CEL randInt helper so non-positive or platform-overflowing arguments surface a typed CEL error instead of an evaluator panic.
• Fix a race in the bbolt store where the asynchronous cleanup scheduled by an expired read could delete a value that had just been refreshed; the delete now only fires when the key still carries the same expired generation it observed.
• Marginally increase the performances of requests processing
• Marginally improve the performances of PoW validation
• Marginally improve the performances of challenges generation/display
• Significantly improve the performances of the gzip middleware
• Significantly improve the performances of the PoW validation
• Add trimpath option to artifact builds
• Add COOKIE_HTTP_ONLY option to set the HttpOnly flag on Anubis cookies
• Improve the performances of rules validation
• Only compute the JA4H fingerprint when a policy references the X-Http-Fingerprint-JA4H header, taking it off the hot path for configurations that don't use it (#834).
• Migrate the target reverse proxy off the deprecated httputil.ReverseProxy.Director to Rewrite for Go 1.26 compatibility, preserving the inbound Host and X-Forwarded-*/Forwarded headers.

What's Changed

• docs(admin/policy): document ReadWritePaths for logging to files by @Xe in docs(admin/policy): document ReadWritePaths for logging to files #1469
• docs: fix mixed tab/space indentation in Caddy config example by @pzhlkj6612 in docs: fix mixed tab/space indentation in Caddy config example #1506
• ci: purge govulncheck, it's less signal than i hoped by @Xe in ci: purge govulncheck, it's less signal than i hoped #1515
• feat(http): set "Cache-Control: no-store" on error responses by @gucci-on-fleek in feat(http): set "Cache-Control: no-store" on error responses #1474
• docs(known-instances): Add missing one. by @p0008874 in docs(known-instances): Add missing one. #1500
• chore(l10n): update French translation by @HarmfulBreeze in chore(l10n): update French translation #1496
• docs: remove developer/code-quality by @gucci-on-fleek in docs: remove developer/code-quality #1475
• build(deps): bump the github-actions group across 1 directory with 11 updates by @dependabot[bot] in build(deps): bump the github-actions group across 1 directory with 11 updates #1516
• build(deps): bump the npm group across 1 directory with 6 updates by @dependabot[bot] in build(deps): bump the npm group across 1 directory with 6 updates #1512
• chore: gofix by @JasonLovesDoggo in chore: gofix #1466
• fix: nil deref when solve request lands on a different DIFFICULTY than the original req by @JasonLovesDoggo in fix: nil deref when solve request lands on a different DIFFICULTY than the original req #1467
• feat(docs): Add instructions on using Anubis with envoy-gateway by @fhoekstra in feat(docs): Add instructions on using Anubis with envoy-gateway #1460
• feat(docs): Update HAProxy Advanced Variant documentation by @rozmansi in feat(docs): Update HAProxy Advanced Variant documentation #1521
• feat(data): add Citoid to good bots list by @mvolz in feat(data): add Citoid to good bots list #1524
• docs(faq): document that disabling JIT makes Anubis slow by @Xe in docs(faq): document that disabling JIT makes Anubis slow #1526
• fix(locales/ja): change the position of the バージョン (version) by @fu-sen in fix(locales/ja): change the position of the バージョン (version) #1527
• fix: add cel iterator by @JasonLovesDoggo in fix: add cel iterator #1465
• fix: Address issued raised in Minor error message nits #1193 by @dwhitemv25 in fix: Address issued raised in #1193 #1519
• feat(anubis): enable pprof endpoints on the metrics server by @Xe in feat(anubis): enable pprof endpoints on the metrics server #1530
• feat(i18n): add Bulgarian language translation by @ramromcode in feat(i18n): add Bulgarian language translation #1394
• chore: update vi.json by @honhatduy in chore: update vi.json #1459
• fix(locales/de): revise and update German translations by @michi-onl in fix(locales/de): revise and update German translations #1457
• build(deps): bump actions-hub/kubectl from 1.35.2 to 1.35.3 in the github-actions group by @dependabot[bot] in build(deps): bump actions-hub/kubectl from 1.35.2 to 1.35.3 in the github-actions group #1532
• chore(sponsors): add embeDD GmbH by @Xe in chore(sponsors): add embeDD GmbH #1539
• build(deps): bump the npm group with 5 updates by @dependabot[bot] in build(deps): bump the npm group with 5 updates #1531
• docs(faq): document minimum versions of browsers Anubis supports by @Xe in docs(faq): document minimum versions of browsers Anubis supports #1540
• build(deps): bump the npm group across 1 directory with 5 updates by @dependabot[bot] in build(deps): bump the npm group across 1 directory with 5 updates #1559
• build(deps): bump the github-actions group across 1 directory with 7 updates by @dependabot[bot] in build(deps): bump the github-actions group across 1 directory with 7 updates #1560
• feat: move metrics server config to the policy file by @Xe in feat: move metrics server config to the policy file #1572
• feat(data): add Meta's web indexer used for AI purposes by @bnjbvr in feat(data): add Meta's web indexer used for AI purposes #1573
• feat(metrics): enable TLS/mTLS serving support by @Xe in feat(metrics): enable TLS/mTLS serving support #1576
• feat(metrics): basic auth support by @Xe in feat(metrics): basic auth support #1579
• fix(honeypot): remove DoS vector by @Xe in fix(honeypot): remove DoS vector #1581
• fix: Thoth geoip compare by @lenny87 in fix: Thoth geoip compare #1564
• fix(metrics): bind to the right network/bindhost by @Xe in fix(metrics): bind to the right network/bindhost #1606
• feat: add ASN data from Thoth to logs/metrics by @lillian-b in feat: add ASN data from Thoth to logs/metrics #1608
• feat: Log weight when issuing challenge by @tdgroot in feat: Log weight when issuing challenge #1611
• fix: patch GHSA-6wcg-mqvh-fcvg by @Xe in fix: patch GHSA-6wcg-mqvh-fcvg #1616
• build(deps-dev): bump the npm group across 1 directory with 6 updates by @dependabot[bot] in build(deps-dev): bump the npm group across 1 directory with 6 updates #1621
• build(deps): bump github.com/go-jose/go-jose/v3 from 3.0.4 to 3.0.5 by @dependabot[bot] in build(deps): bump github.com/go-jose/go-jose/v3 from 3.0.4 to 3.0.5 #1629
• fix(policy): correctly wire subrequest mode through CEL/path checkers by @Xe in fix(policy): correctly wire subrequest mode through CEL/path checkers #1630
• fix(honeypot/naive): apply robot9001 style delays by @Xe in fix(honeypot/naive): apply robot9001 style delays #1632
• build(deps): bump the npm group across 1 directory with 6 updates by @dependabot[bot] in build(deps): bump the npm group across 1 directory with 6 updates #1646
• perf(lib): iterate s.policy.Bots by index to drop per-call heap copy by @jvoisin in perf(lib): iterate s.policy.Bots by index to drop per-call heap copy #1639
• fix: small security fixes by @Xe in fix: small security fixes #1651
• perf(internal/gzip): pool *gzip.Writer per middleware instance by @jvoisin in perf(internal/gzip): pool *gzip.Writer per middleware instance #1654
• perf: enable uuid randomness pool and minor cleanups by @jvoisin in perf: enable uuid randomness pool and minor cleanups #1652
• perf(challenge/proofofwork): stream sha256 into stack buffer in Validate by @jvoisin in perf(challenge/proofofwork): stream sha256 into stack buffer in Validate #1653
• chore: use Go stdlib version stamping by @Xe in chore: use Go stdlib version stamping #1665
• chore: ban x.ai by @Xe in chore: ban x.ai #1673
• feat: add HttpOnly cookie option by @tdgroot in feat: add HttpOnly cookie option #1679
• fix: add trimpath option to artifact builds by @tdgroot in fix: add trimpath option to artifact builds #1682
• ci: remove spelling checks by @Xe in ci: remove spelling checks #1688
• perf(policy): only compute ja4h fingerprint when referenced by @jvoisin in perf(policy): only compute ja4h fingerprint when referenced #1689
• perf: don't look into the store for OpenGraph things if it is disabled by @jvoisin in perf: don't look into the store for OpenGraph things if it is disabled #1690
• refactor(honeypot/naive): compute network hash once in incrementNetwork by @jvoisin in refactor(honeypot/naive): compute network hash once in incrementNetwork #1693
• perf: iterate s.policy.Bots by index to drop per-call heap copy by @jvoisin in perf: iterate s.policy.Bots by index to drop per-call heap copy #1691
• perf(internal/gzip): share writer pool across middleware reconstructions by @jvoisin in perf(internal/gzip): share writer pool across middleware reconstructions #1694
• perf(internal): cut allocations in computeXFFHeader by @jvoisin in perf(internal): cut allocations in computeXFFHeader #1692
• build(deps-dev): bump the npm group across 1 directory with 4 updates by @dependabot[bot] in build(deps-dev): bump the npm group across 1 directory with 4 updates #1677
• docs: use stable container in traefik deployment example by @RealOrangeOne in docs: use stable container in traefik deployment example #1676
• chore: bump go version to 1.26 by @Xe in chore: bump go version to 1.26 #1695
• chore: tag v1.26.0-pre1 by @Xe in chore: tag v1.26.0-pre1 #1705
• feat(qwantbot): update remote addresses by @mhardy-qwant in feat(qwantbot): update remote addresses #1719
• ci: add cspell to replace check-spelling by @Xe in ci: add cspell to replace check-spelling #1722
• chore: set up golangci-lint by @Xe in chore: set up golangci-lint #1720
• chore: add golangci words to the spelling dictionary by @Xe in chore: add golangci words to the spelling dictionary #1724
• build(deps): bump the npm group across 1 directory with 8 updates by @dependabot[bot] in build(deps): bump the npm group across 1 directory with 8 updates #1723
• docs: add anubis-kubernetes-operator as another option by @eznix86 in docs: add anubis-kubernetes-operator as another option #1675
• chore: update playwright dependencies by @JasonLovesDoggo in chore: update playwright dependencies #1279
• build(deps): bump the gomod group across 1 directory with 16 updates by @dependabot[bot] in build(deps): bump the gomod group across 1 directory with 16 updates #1727
• build(deps): bump the github-actions group across 1 directory with 14 updates by @dependabot[bot] in build(deps): bump the github-actions group across 1 directory with 14 updates #1725
• build(deps): bump the gomod group with 2 updates by @dependabot[bot] in build(deps): bump the gomod group with 2 updates #1730
• build(deps-dev): bump prettier from 3.8.5 to 3.9.1 in the npm group by @dependabot[bot] in build(deps-dev): bump prettier from 3.8.5 to 3.9.1 in the npm group #1729
• chore(data): fix google translate bug Anubis breaks google translate #444 by @Xe in chore(data): fix google translate bug #444 #1734
• fix(web): lower chrome support floor to Chrome 66 by @Xe in fix(web): lower chrome support floor to Chrome 66 #1738
• feat(yeetfile): build freebsd and windows binaries by @Xe in feat(yeetfile): build freebsd and windows binaries #1735
• chore: add chrome testing command by @Xe in chore: add chrome testing command #1740
• fix(crawlers): update MojeekBot remote_addresses subnet to match official spec by @bochard07 in fix(crawlers): update MojeekBot remote_addresses subnet to match official spec #1746
• feat(config): add option to disable the honeypot by @Xe in feat(config): add option to disable the honeypot #1759

New Contributors

• @pzhlkj6612 made their first contribution in docs: fix mixed tab/space indentation in Caddy config example #1506
• @HarmfulBreeze made their first contribution in chore(l10n): update French translation #1496
• @fhoekstra made their first contribution in feat(docs): Add instructions on using Anubis with envoy-gateway #1460
• @rozmansi made their first contribution in feat(docs): Update HAProxy Advanced Variant documentation #1521
• @mvolz made their first contribution in feat(data): add Citoid to good bots list #1524
• @fu-sen made their first contribution in fix(locales/ja): change the position of the バージョン (version) #1527
• @dwhitemv25 made their first contribution in fix: Address issued raised in #1193 #1519
• @ramromcode made their first contribution in feat(i18n): add Bulgarian language translation #1394
• @honhatduy made their first contribution in chore: update vi.json #1459
• @bnjbvr made their first contribution in feat(data): add Meta's web indexer used for AI purposes #1573
• @lenny87 made their first contribution in fix: Thoth geoip compare #1564
• @jvoisin made their first contribution in perf(lib): iterate s.policy.Bots by index to drop per-call heap copy #1639
• @RealOrangeOne made their first contribution in docs: use stable container in traefik deployment example #1676
• @mhardy-qwant made their first contribution in feat(qwantbot): update remote addresses #1719
• @eznix86 made their first contribution in docs: add anubis-kubernetes-operator as another option #1675
• @bochard07 made their first contribution in fix(crawlers): update MojeekBot remote_addresses subnet to match official spec #1746

Full Changelog: v1.25.0...v1.26.0

---

This discussion was created from the release v1.26.0: Papalymo Totolymo.