Automated Docs Update - #3373
Merged
Merged
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## dev #3373 +/- ##
=====================================
- Coverage 90% 90% -0%
=====================================
Files 454 454
Lines 46687 46687
=====================================
- Hits 41966 41955 -11
- Misses 4721 4732 +11 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
liquidsec
approved these changes
Aug 12, 2026
liquidsec
added a commit
that referenced
this pull request
Aug 24, 2026
* trufflehog: stream HTTP body via stdin, include source url on findings * add js-audit preset * bump trufflehog _module_threads to 2 * Update trufflehog * [create-pull-request] automated change * Bump pre-commit from 4.6.0 to 4.6.1 Bumps [pre-commit](https://github.com/pre-commit/pre-commit) from 4.6.0 to 4.6.1. - [Release notes](https://github.com/pre-commit/pre-commit/releases) - [Changelog](https://github.com/pre-commit/pre-commit/blob/main/CHANGELOG.md) - [Commits](pre-commit/pre-commit@v4.6.0...v4.6.1) --- updated-dependencies: - dependency-name: pre-commit dependency-version: 4.6.1 dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Bump actions/setup-python from 6 to 7 in the github-actions group Bumps the github-actions group with 1 update: [actions/setup-python](https://github.com/actions/setup-python). Updates `actions/setup-python` from 6 to 7 - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v6...v7) --- updated-dependencies: - dependency-name: actions/setup-python dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions ... Signed-off-by: dependabot[bot] <support@github.com> * Bump fastapi from 0.139.2 to 0.140.0 Bumps [fastapi](https://github.com/fastapi/fastapi) from 0.139.2 to 0.140.0. - [Release notes](https://github.com/fastapi/fastapi/releases) - [Commits](fastapi/fastapi@0.139.2...0.140.0) --- updated-dependencies: - dependency-name: fastapi dependency-version: 0.140.0 dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Bump pytest-env from 1.6.0 to 1.7.0 Bumps [pytest-env](https://github.com/pytest-dev/pytest-env) from 1.6.0 to 1.7.0. - [Release notes](https://github.com/pytest-dev/pytest-env/releases) - [Commits](pytest-dev/pytest-env@1.6.0...1.7.0) --- updated-dependencies: - dependency-name: pytest-env dependency-version: 1.7.0 dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Bump mmh3 from 5.2.0 to 5.2.1 Bumps [mmh3](https://github.com/hajimes/mmh3) from 5.2.0 to 5.2.1. - [Release notes](https://github.com/hajimes/mmh3/releases) - [Changelog](https://github.com/hajimes/mmh3/blob/master/CHANGELOG.md) - [Commits](hajimes/mmh3@v5.2.0...v5.2.1) --- updated-dependencies: - dependency-name: mmh3 dependency-version: 5.2.1 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Bump cachetools from 7.1.4 to 7.1.6 Bumps [cachetools](https://github.com/tkem/cachetools) from 7.1.4 to 7.1.6. - [Changelog](https://github.com/tkem/cachetools/blob/master/CHANGELOG.rst) - [Commits](tkem/cachetools@v7.1.4...v7.1.6) --- updated-dependencies: - dependency-name: cachetools dependency-version: 7.1.6 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Bump orjson from 3.11.7 to 3.11.9 Bumps [orjson](https://github.com/ijl/orjson) from 3.11.7 to 3.11.9. - [Release notes](https://github.com/ijl/orjson/releases) - [Changelog](https://github.com/ijl/orjson/blob/master/CHANGELOG.md) - [Commits](ijl/orjson@3.11.7...3.11.9) --- updated-dependencies: - dependency-name: orjson dependency-version: 3.11.9 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Bump regex from 2026.7.10 to 2026.7.19 Bumps [regex](https://github.com/mrabarnett/mrab-regex) from 2026.7.10 to 2026.7.19. - [Changelog](https://github.com/mrabarnett/mrab-regex/blob/hg/changelog.txt) - [Commits](mrabarnett/mrab-regex@2026.7.10...2026.7.19) --- updated-dependencies: - dependency-name: regex dependency-version: 2026.7.19 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Fix docker_pull crash on failed registry request * Bump uvicorn from 0.39.0 to 0.51.0 Bumps [uvicorn](https://github.com/Kludex/uvicorn) from 0.39.0 to 0.51.0. - [Release notes](https://github.com/Kludex/uvicorn/releases) - [Changelog](https://github.com/Kludex/uvicorn/blob/main/docs/release-notes.md) - [Commits](Kludex/uvicorn@0.39.0...0.51.0) --- updated-dependencies: - dependency-name: uvicorn dependency-version: 0.51.0 dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * ci: pass benchmark refs through env and quote them (#3354) Co-authored-by: Paul Mueller <paul.mueller08@gmail.com> * Reinstall module deps when they're missing from the environment (#3338) * Reinstall module deps when they're missing from the environment The setup status cache recorded a module's deps as installed and never rechecked them, so a rebuilt virtualenv (e.g. `uv sync`) silently left modules broken until ~/.bbot was wiped by hand. Verify a module's pip packages are actually present before honoring a cache hit. Also promote `packaging` to a direct dependency; it was only present transitively via ansible. * Remove dead else branch, honor custom pip constraints The else branch in install()'s dependency loop was unreachable structure; the preceding branch always continues. Collapsing it also flattens the nested if/else into elif. pip_install() overwrote custom constraints with bbot's own instead of using them as a fallback. No module sets deps_pip_constraints today, so this changes no current behavior. * Bump packaging from 26.0 to 26.2 (#3365) Bumps [packaging](https://github.com/pypa/packaging) from 26.0 to 26.2. - [Release notes](https://github.com/pypa/packaging/releases) - [Changelog](https://github.com/pypa/packaging/blob/main/CHANGELOG.rst) - [Commits](pypa/packaging@26.0...26.2) --- updated-dependencies: - dependency-name: packaging dependency-version: '26.2' dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump pytest-httpserver from 1.1.4 to 1.1.5 (#3364) Bumps [pytest-httpserver](https://github.com/csernazs/pytest-httpserver) from 1.1.4 to 1.1.5. - [Release notes](https://github.com/csernazs/pytest-httpserver/releases) - [Changelog](https://github.com/csernazs/pytest-httpserver/blob/master/CHANGES.rst) - [Commits](csernazs/pytest-httpserver@1.1.4...1.1.5) --- updated-dependencies: - dependency-name: pytest-httpserver dependency-version: 1.1.5 dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump lxml from 6.1.0 to 6.1.1 (#3362) Bumps [lxml](https://github.com/lxml/lxml) from 6.1.0 to 6.1.1. - [Release notes](https://github.com/lxml/lxml/releases) - [Changelog](https://github.com/lxml/lxml/blob/master/CHANGES.txt) - [Commits](lxml/lxml@lxml-6.1.0...lxml-6.1.1) --- updated-dependencies: - dependency-name: lxml dependency-version: 6.1.1 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump pip from 26.1.2 to 26.2 (#3360) Bumps [pip](https://github.com/pypa/pip) from 26.1.2 to 26.2. - [Changelog](https://github.com/pypa/pip/blob/main/NEWS.rst) - [Commits](pypa/pip@26.1.2...26.2) --- updated-dependencies: - dependency-name: pip dependency-version: '26.2' dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump uvicorn from 0.51.0 to 0.52.0 (#3363) Bumps [uvicorn](https://github.com/Kludex/uvicorn) from 0.51.0 to 0.52.0. - [Release notes](https://github.com/Kludex/uvicorn/releases) - [Changelog](https://github.com/Kludex/uvicorn/blob/main/docs/release-notes.md) - [Commits](Kludex/uvicorn@0.51.0...0.52.0) --- updated-dependencies: - dependency-name: uvicorn dependency-version: 0.52.0 dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump fastapi from 0.140.0 to 0.141.1 (#3361) Bumps [fastapi](https://github.com/fastapi/fastapi) from 0.140.0 to 0.141.1. - [Release notes](https://github.com/fastapi/fastapi/releases) - [Commits](fastapi/fastapi@0.140.0...0.141.1) --- updated-dependencies: - dependency-name: fastapi dependency-version: 0.141.1 dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump pip from 26.2 to 26.2.1 (#3369) Bumps [pip](https://github.com/pypa/pip) from 26.2 to 26.2.1. - [Changelog](https://github.com/pypa/pip/blob/main/NEWS.rst) - [Commits](pypa/pip@26.2...26.2.1) --- updated-dependencies: - dependency-name: pip dependency-version: 26.2.1 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump packaging from 26.2 to 26.3 (#3371) Bumps [packaging](https://github.com/pypa/packaging) from 26.2 to 26.3. - [Release notes](https://github.com/pypa/packaging/releases) - [Changelog](https://github.com/pypa/packaging/blob/main/CHANGELOG.rst) - [Commits](pypa/packaging@26.2...26.3) --- updated-dependencies: - dependency-name: packaging dependency-version: '26.3' dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Parallelize test suite with pytest-xdist (#3353) * test: parallelize suite with pytest-xdist Test servers bound fixed ports (8888/9999/5556) and 58 test files hardcoded those in URLs, so the suite could only ever run serially. CI spent ~58 min in a single pytest call while using one of four cores. - bbot/test/ports.py: resolve server ports per xdist worker - conftest.py: tolerate listener=None (workers inherit _BBOT_LOGGING_SETUP) - 57 test files: hardcoded host:port -> constants from bbot.test.ports localhost and 127.0.0.1 spellings are kept distinct; virtualhost tests assert on the Host header, where they are not interchangeable. test_regexes.py is untouched, its literals are regex fixtures, not references to the test server. * test: isolate per-worker state for parallel runs Beyond the server ports, three things were process-global and would corrupt each other under xdist: - BBOT home (/tmp/.bbot_test): shared caches and scan output, and pytest_sessionfinish rmtree's it. The first worker to finish would delete the directory out from under every worker still running. - uvicorn (8978) and websockets (8765) servers: bound fixed ports. - docker-backed tests (kafka, elastic, mongo, mysql, nats, postgres, rabbitmq): fixed container names and fixed host port bindings, so they genuinely cannot run concurrently. They are now pinned to a single worker via xdist_group, keyed off the skip_distro_tests flag they already set. Renamed ports.py -> worker.py since it now covers more than ports. Port bases were chosen with distinct mod-100 residues; verified collision-free and clear of the ephemeral range for up to 64 workers. * ci: run tests in parallel pytest-xdist added to the dev group; workflow runs with -n logical --dist loadgroup. loadgroup (not loadfile) is required so the xdist_group marker on the docker-backed tests is honored. * test: bound docker service waits and size worker count by memory Two problems surfaced running the suite 16-way on a 16-core/15GB box. Elasticsearch was OOM-killed (exit 137): 16 workers hold ~10GB, leaving under 3GB, and it wants ~2GB of heap. The test then waited for it in an unbounded 'while True', so a container that was already dead hung the run at 99% until the 1200s global timeout. Same unbounded loop in the mongo test. Both now stop after CONTAINER_READY_TIMEOUT (180s, overridable) and say what happened, including the exit-137 hint. '-n logical' is the wrong worker count when cores outnumber GB. worker_count.py bounds by cores AND by available memory, keeping 5GB free for the docker-backed services. A 4-core/16GB github runner is unaffected (still 4); this box goes 16 -> 15. Override with BBOT_TEST_WORKERS. * test: fix parallel-run regressions missed by the first port pass Running the suite under xdist surfaced failures that were my own, not pre-existing. Four separate gaps: Logging. Workers inherit _BBOT_LOGGING_SETUP from the pytest parent, so BBOTLogger skipped setup entirely and debug.log was never written. The previous 'if listener is not None' guard hid that instead of fixing it, and log-reading tests failed with 'assert -retries 1 in ""'. Workers now clear the flag so each sets up its own logging, and conftest raises if a process somehow still has no listener. Port literals the first pass did not match, because it only looked for 127.0.0.1:PORT and localhost:PORT: * synthetic hostnames carrying the port (test.example:8888, bad.dns:8888) * the bbot_other_httpservers fixture, six servers hardcoded to 8888/8889 * config values like {"ports": "8888"}, which are not URLs * 127.0.0.0:9999 (the .0 address, excluded along with .1) * filename/slug spellings: 127.0.0.1.8888 and http-127-0-0-1-8888 * mocked masscan JSON, whose port is what the module 'discovers' Hardcoded /tmp/.bbot_test paths, which my own per-worker home broke. Adds BBOT_TEST_PORT_OFFSET so a second copy of the suite can run beside one that already owns the base ports, instead of the two colliding and looking like a worker-isolation bug. * test: remove ordering dependency and fix integer port comparisons Full-suite A/B surfaced the last regressions from parallelization. asset_inventory: TestAsset_InventoryEmitPrevious runs with use_previous=True, which reads the asset-inventory.csv left behind by an earlier scan of the same name. It was relying on TestAsset_Inventory having already run. That is order-dependent, and xdist guarantees neither order nor worker affinity, so it failed whenever it was scheduled first or landed on another worker (each worker has its own BBOT home). It now seeds the CSV itself in setup_before_prep, which makes it independent and drops its runtime from ~20s to ~4s since it no longer needs the portscan. Port comparisons written as bare integers (e.port == 8888) in test_scope.py and test_module_fingerprintx.py. The earlier passes only rewrote string literals, so these were missed. Also groups tests that share a scan_name onto one worker, alongside the existing docker grouping. Not sufficient on its own for the ordering problem above, but correct for co-locating tests that share a scan home. * test: stop hardcoding the BBOT home directory name in path assertions git_clone writes cloned repos to git_repos/<bbot home name>/<repo>, so the home directory's *basename* shows up inside the paths these tests assert on. They spelled it '.bbot_test', which stopped matching once each xdist worker got its own home ('.bbot_test_gw0'). Export BBOT_TEST_DIR_NAME from worker.py and use it instead. Worth noting these reproduce with '-n 1', a single worker with no concurrency at all. They were never race conditions, just the per-worker home rename, and checking '-n 1' before '-n 4' would have found that much faster. * test: fix port in a regex literal and a self-poisoning cli precondition test_module_robots matched the server URL with a raw regex string, where the port sits behind escaped dots (r"http://127\.0\.0\.1:8888/..."). The earlier rewrites looked for plain literals and skipped it, so the test failed whenever it landed on a worker whose port was not 8888. test_cli_presets asserts a scan output dir does not exist yet, and removes it at the end of the test. Any earlier failure or interrupted run leaves it in place, and every later run against that BBOT home then fails on the assertion rather than on the real problem. It now clears the directory up front, so the precondition is established instead of assumed. Both showed up as intermittent (~25% of runs) rather than deterministic, which is what made them the last two to be found. * test: cut dead code and de-duplicate the container wait Net removal pass over the parallelization work. Removed: - BBOT_TEST_PORT_OFFSET and port_offset(). Added for running two copies of the suite side by side while benchmarking. Nothing outside worker.py used it. - LOCALHOST_SSL_HOSTPORT, the BASE_* port aliases, and the worker_index() wrapper. All unused or single-use indirection. - OTHER_HTTPSERVER_PORT, which was literally 'OTHER_HTTPSERVER_PORT = HTTPSERVER_PORT'. Renamed to HTTPSERVER_PORT/HTTPSERVER_PORT_ALT at its one call site. - The scan_name xdist grouping. It compensated for asset_inventory's ordering dependency, which the previous commit removed by having the test seed its own input, so the grouping no longer does anything. Verified: 3x green at 4 workers without it. - worker_count's -v flag and MIN_WORKERS floor. The floor guaranteed 2 workers even when the memory arithmetic said there was room for none, which is an assumption I never validated; a 2-core/4GB box now correctly gets 1. - A dead 'import time' the refactor exposed. De-duplicated: - The container readiness loop was copy-pasted between elastic (sync) and mongo (async), each with its own deadline handling and OOM hint. Now one wait_for_container() helper that takes either kind of callable. worker.py 154 -> 100 lines, worker_count.py 64 -> 38, conftest hook 40 -> 13. * test: fix gowitness hardcoded ports and shared home directory Caught by CI on this PR, not locally: gowitness needs its binary installed, so both classes error out on my machine and never reached the assertions. Two bugs: The multiport test asserted on bare port substrings ('8888' in url) rather than URL-shaped literals, so the earlier rewrites did not match them. On any worker but the first the ports are 9188/10299 and the assertion could never pass. Both classes also pinned an explicit home directory under /tmp and rmtree'd it at class-definition time, which every worker does to the same path. Workers deleted each other's dependency installs mid-run, which is where the FileNotFoundError on the depsinstaller artifact came from. Now worker-scoped. Worth noting these run with deps.behavior=force_install, so each worker installs its own copy of chrome, ~650MB. That is fine on a runner where /tmp is disk-backed, but it multiplies with worker count. * test: fix a hardcoded port in the virtualhost canary regex The force_basehost wildcard-canary check still matched a literal 8888, so on any worker but gw0 it never matched. Both that branch and the fallthrough return 404, which is why it did not show up as a failure. * test: give the remaining fixed /tmp paths a per-worker name Same class of bug as the gowitness home directory that CI caught: paths that are fine serially but shared once workers run concurrently. - test_python_api used one BBOT home across two tests, so the dep installer wrote and read the same artifacts directory from two workers - test_presets wrote a fixed custom preset dir - test_command and test_web wrote fixed scratch files, then read them back * ci: skip the benchmark PR comment on fork pull requests A pull_request run from a fork gets a read-only GITHUB_TOKEN regardless of the permissions block, so github-script fails with HttpError: Resource not accessible by integration (403) POST /repos/.../issues/{n}/comments and takes the whole job red. Every fork PR has a failing Performance Benchmarks check for a reason that has nothing to do with performance, which trains people to ignore a red check. Skip the comment step when the head repo is not this repo. The benchmark still runs and the report is still uploaded as an artifact, so nothing is lost except a comment that could never have been posted. * test: cover the worker count and isolation helpers Codecov flagged worker_count.py at 0 percent: the workflow runs it as a script, so pytest never imported it. worker.py was only covered incidentally. These two modules decide how many workers run and how each one gets its own ports and directories, so a silent regression there either overloads the runner or lets workers collide. Verified by mutation: removing the max(1, ...) clamp, dropping the memory cap, and swapping sched_getaffinity for os.cpu_count each fail these tests. * Bump uvicorn from 0.52.0 to 0.52.1 (#3368) Bumps [uvicorn](https://github.com/Kludex/uvicorn) from 0.52.0 to 0.52.1. - [Release notes](https://github.com/Kludex/uvicorn/releases) - [Changelog](https://github.com/Kludex/uvicorn/blob/main/docs/release-notes.md) - [Commits](Kludex/uvicorn@0.52.0...0.52.1) --- updated-dependencies: - dependency-name: uvicorn dependency-version: 0.52.1 dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Shane Engelman <haroboy876@gmail.com> * Bump cachetools from 7.1.6 to 7.1.7 (#3372) Bumps [cachetools](https://github.com/tkem/cachetools) from 7.1.6 to 7.1.7. - [Changelog](https://github.com/tkem/cachetools/blob/master/CHANGELOG.rst) - [Commits](tkem/cachetools@v7.1.6...v7.1.7) --- updated-dependencies: - dependency-name: cachetools dependency-version: 7.1.7 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Shane Engelman <haroboy876@gmail.com> * Bump websockets from 16.0 to 16.1.1 (#3312) Bumps [websockets](https://github.com/python-websockets/websockets) from 16.0 to 16.1.1. - [Release notes](https://github.com/python-websockets/websockets/releases) - [Commits](python-websockets/websockets@16.0...16.1.1) --- updated-dependencies: - dependency-name: websockets dependency-version: 16.1.1 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Shane Engelman <haroboy876@gmail.com> * Bump ruff from 0.15.22 to 0.16.1 (#3370) * Bump ruff from 0.15.22 to 0.16.1 Bumps [ruff](https://github.com/astral-sh/ruff) from 0.15.22 to 0.16.1. - [Release notes](https://github.com/astral-sh/ruff/releases) - [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md) - [Commits](astral-sh/ruff@0.15.22...0.16.1) --- updated-dependencies: - dependency-name: ruff dependency-version: 0.16.1 dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * style: ruff format for ruff 0.16 ruff 0.16 formats python code blocks inside markdown by default, so the existing docs fail 'ruff format --check' under the bumped version. Apply the formatter (10 files, docs only) to make the lint step pass. --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update nuclei (#3367) Co-authored-by: Shane Engelman <haroboy876@gmail.com> * [create-pull-request] automated change (#3373) * Fix custom YARA rules matching but emitting nothing (#3374) * fix custom YARA rule emission and rule splitting CustomExtractor.process() called .get() on YaraRuleSettings, which has no get(), so every custom rule match raised AttributeError before the event was emitted. The scan logged one error per match and reported zero findings. Replace the regex-based rule splitter with a brace-depth-aware parser that skips comments, string literals, and regex literals, so braces inside them no longer truncate or drop rules. Rule names now tolerate arbitrary whitespace, import statements are collected and preserved, and unparseable input fails loudly instead of silently dropping a rule. * test: use per-worker HTTPSERVER_URL in custom YARA tests The two custom-YARA module tests hardcoded http://127.0.0.1:8888/ as their target. Under pytest-xdist the httpserver port is offset per worker (worker_port(8888) in bbot/test/worker), so 8888 is only correct on gw0. On any other worker the scan hit the wrong (or no) server, so no HTTP_RESPONSE was produced, the custom rule never matched, and the test failed with 'produced no FINDING, got: []'. TestExcavateYaraCustomEdgeCases passed only because it happened to land on gw0. Point both at HTTPSERVER_URL like every other test in this file. Verified green under -n 2 --dist loadgroup, which reproduced the original failure. --------- Co-authored-by: Shane Engelman <shane.engelman@blacklanternsecurity.com> * Raise process pool max_tasks_per_child from 25 to 250 (#3375) Under spawn, retiring a worker every 25 tasks forces a cold re-import of the callback module (~742ms for badsecrets). badsecrets runs on every HTTP_RESPONSE and leaks ~0 memory (RSS flat to 5000 tasks), so 25 costs a ~25x throughput drop and ~130x p95 blowup for no benefit. 250 recovers full throughput while keeping the safety net for the heavy infrequent tenants (kreuzberg, bevigil, waf_bypass). * Update trufflehog (#3376) * Bump pip from 26.2 to 26.2.1 (#3380) Bumps [pip](https://github.com/pypa/pip) from 26.2 to 26.2.1. - [Changelog](https://github.com/pypa/pip/blob/main/NEWS.rst) - [Commits](pypa/pip@26.2...26.2.1) --- updated-dependencies: - dependency-name: pip dependency-version: 26.2.1 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump tldextract from 5.3.1 to 5.3.2 (#3381) Bumps [tldextract](https://github.com/john-kurkowski/tldextract) from 5.3.1 to 5.3.2. - [Release notes](https://github.com/john-kurkowski/tldextract/releases) - [Changelog](https://github.com/john-kurkowski/tldextract/blob/master/CHANGELOG.md) - [Commits](john-kurkowski/tldextract@5.3.1...5.3.2) --- updated-dependencies: - dependency-name: tldextract dependency-version: 5.3.2 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump packaging from 26.2 to 26.3 (#3378) Bumps [packaging](https://github.com/pypa/packaging) from 26.2 to 26.3. - [Release notes](https://github.com/pypa/packaging/releases) - [Changelog](https://github.com/pypa/packaging/blob/main/CHANGELOG.rst) - [Commits](pypa/packaging@26.2...26.3) --- updated-dependencies: - dependency-name: packaging dependency-version: '26.3' dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump ruff from 0.16.1 to 0.16.2 (#3379) Bumps [ruff](https://github.com/astral-sh/ruff) from 0.16.1 to 0.16.2. - [Release notes](https://github.com/astral-sh/ruff/releases) - [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md) - [Commits](astral-sh/ruff@0.16.1...0.16.2) --- updated-dependencies: - dependency-name: ruff dependency-version: 0.16.2 dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump pre-commit from 4.6.1 to 4.6.2 (#3383) Bumps [pre-commit](https://github.com/pre-commit/pre-commit) from 4.6.1 to 4.6.2. - [Release notes](https://github.com/pre-commit/pre-commit/releases) - [Changelog](https://github.com/pre-commit/pre-commit/blob/main/CHANGELOG.md) - [Commits](pre-commit/pre-commit@v4.6.1...v4.6.2) --- updated-dependencies: - dependency-name: pre-commit dependency-version: 4.6.2 dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump xxhash from 3.8.1 to 4.0.0 (#3382) Bumps [xxhash](https://github.com/ifduyue/python-xxhash) from 3.8.1 to 4.0.0. - [Release notes](https://github.com/ifduyue/python-xxhash/releases) - [Changelog](https://github.com/ifduyue/python-xxhash/blob/master/CHANGELOG.rst) - [Commits](ifduyue/python-xxhash@v3.8.1...v4.0.0) --- updated-dependencies: - dependency-name: xxhash dependency-version: 4.0.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * bump blasthttp to 0.10.0 (#3385) Surface Response.decode_error on HTTP_RESPONSE events and tag them decode-error, so nothing treats an undecoded body as content. Pass redirect_cookies through WebHelper.request(). * Bump version to 3.0.2 (#3387) * don't treat an undecoded response body as content (#3389) A body that fails to decode is no longer offered as `body` or parsed for a `title`; `decode_error` carries the reason instead. Drops the `decode-error` tag, which nothing read. Also bounds the blasthttp pin at <0.11, matching the other dependencies. --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: blsaccess <info@blacklanternsecurity.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Kobi Hikri <kobi.hikri@gmail.com> Co-authored-by: Shane Engelman <haroboy876@gmail.com> Co-authored-by: BLS <114500254+blsaccess@users.noreply.github.com> Co-authored-by: Shane Engelman <shane.engelman@blacklanternsecurity.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is an automated pull request to update the documentation.