Skip to content

Fix crash, hang, and resource-leak defects found in a project-wide scan#104

Merged
JE-Chen merged 8 commits into
mainfrom
dev
Jul 23, 2026
Merged

Fix crash, hang, and resource-leak defects found in a project-wide scan#104
JE-Chen merged 8 commits into
mainfrom
dev

Conversation

@JE-Chen

@JE-Chen JE-Chen commented Jul 23, 2026

Copy link
Copy Markdown
Member

Project-wide scan for crash / runtime defects, and the fixes. Static analysis
(ruff, mypy, bandit) was already clean, so these were found by hunting hazard
patterns: unbounded external calls, loops without exit conditions, and cleanup
paths that abort partway.

Hangs

  • process_supervisor, diff_shardps / tasklist / taskkill and the
    git shard-split ran with no timeout, so a wedged subprocess blocked the
    calling thread forever. A git process waiting on an index lock or a credential
    prompt would stall the whole shard split. Handlers widened to
    SubprocessError so the now-reachable TimeoutExpired is caught.

  • socket_serverrecv() had no socket timeout. ThreadingMixIn spawns a
    thread per connection, so a few clients that connect and never send could
    exhaust the server.

Infinite loop

  • BrowserPool.checkout() — destroying an unhealthy session frees a tracked
    slot, so _can_grow() flipped back to True and the loop spawned again. A
    factory that always yields unhealthy instances (crash-on-launch browser, dead
    health check) looped forever, churning real browser processes and never
    checking the timeout. Measured ~6400 spawn/destroy cycles in 0.2s before the
    fix. Retries are now capped deterministically at size * 3.

Resource leaks

  • WebdriverManager.quit() — aborted the loop on the first driver that
    raised, so every remaining browser leaked as an orphan process and the
    tracking list was never cleared. Every driver now gets a quit attempt.

  • WebdriverManager close paths — left current_webdriver pointing at a
    closed session, and never cleared the shared webdriver_wrapper singleton, so
    the next caller inherited a dead driver and a stale ActionChains. Close now
    happens before the reference is dropped, so a failed close stays tracked for
    quit() to reclaim.

Socket protocol

The handler did one recv(8192), so any request over one segment was silently
truncated. It now reads until the request is complete.

Clients can opt into WRLEN <n>\n<body> framing. It is auto-detected, so
existing clients are unaffected, and replies mirror the dialect the request
arrived in. Framing also fixes authentication when the token line straddles a
packet boundary — something the legacy format cannot resolve, since a
token-less first segment must be rejected promptly and is indistinguishable
from a partial one.

Legacy requests use an incremental bracket-balance scanner that tells a
truncated body from a malformed one, preserving the prompt error reply for bad
JSON. Requests are capped at 8 MiB. send_command / encode_frame /
read_frame are exported so the framed format is usable from client code.

Smaller fixes

  • assert_sorted_by leaked a bare TypeError for non-orderable keys
    (Hashable does not imply orderable) instead of the module exception.
  • The fan-out parallelism test asserted a 0.12s wall-clock budget and was flaky
    on loaded machines (observed 0.234s). Replaced with a barrier, which is
    timing-independent.

Verification

4375 passed, 9 skipped, 22 subtests passed   (was 4333 passed / 1 failed)
ruff F,E9,B:  All checks passed!
bandit -ll:   exit 0
C901:         no new complexity violations

41 regression tests added covering every fix above.

🤖 Generated with Claude Code

JE-Chen added 6 commits July 23, 2026 11:20
ps / tasklist / taskkill and the git shard-split invocation ran without a
timeout, so a wedged subprocess blocked the calling thread indefinitely.
Broaden the handlers from CalledProcessError to SubprocessError so the
newly reachable TimeoutExpired is caught rather than escaping.
Destroying an unhealthy session frees a tracked slot, so _can_grow() flipped
straight back to True and checkout() spawned again. A factory that always
produced unhealthy instances (crash-on-launch browser, dead health check)
looped forever, churning real browser processes and ignoring the timeout.

Cap the replacement attempts at size * 3 so the failure is deterministic
rather than a spin that only stops at the deadline.
quit() aborted its loop on the first driver that raised, leaving the
remaining browsers running as orphans and the tracking list uncleared.
Attempt every driver and report the failures afterwards.

The close paths left current_webdriver pointing at a closed session, and
neither path cleared the shared webdriver_wrapper singleton, so the next
caller inherited a dead driver and a stale ActionChains. Close before
dropping the reference too, so a failed close stays tracked for quit().
The handler did a single recv(8192), so any request larger than one segment
was silently truncated, and a client that connected without sending pinned
its thread forever (one thread per connection under ThreadingMixIn).

Add a receive timeout and read until the request is actually complete. New
clients can opt into WRLEN <n>\n<body> framing, which is auto-detected, so
existing clients keep working unchanged and replies mirror the dialect the
request arrived in. Framing also fixes authentication when the token line
straddles a packet boundary, which the legacy heuristic cannot resolve.

Legacy requests fall back to an incremental bracket-balance scanner that
distinguishes a truncated body from a malformed one, keeping the prompt
error reply for bad JSON. Requests are capped at 8 MiB.

Export send_command / encode_frame / read_frame so the framed format is
usable from client code.
items_by_page_key only guarantees Hashable, which does not imply orderable.
Mixed key types raised a bare TypeError out of assert_sorted_by instead of
the module exception callers handle.
The 0.12s wall-clock budget was flaky on loaded machines, where thread
start-up alone can exceed it. Use a barrier instead: three tasks can only
clear it when they are genuinely in flight at the same time.
@codacy-production

codacy-production Bot commented Jul 23, 2026

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 121 complexity · 0 duplication

Metric Results
Complexity 121
Duplication 0

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

JE-Chen added 2 commits July 23, 2026 11:27
Bandit B106 flagged the intentionally wrong token used to assert that
authentication rejects it, matching the suppression the surrounding
fixtures already carry.
All six were fixture data or protocol constants rather than credentials:
stubbed OAuth/FCM tokens, the "Bearer" token-type constant (the mock
server issues its real token from secrets.token_hex), and "pass_rate",
which matched only on the "pass" substring and holds a float ratio.

Bandit parses space-separated test IDs after nosec, not comma-separated,
and the marker has to sit on the flagged line itself. Both tripped up the
existing suppression on the autofill fixture.

Bandit now reports clean across the package and the test suite at every
severity level, not just the -ll threshold CI had been using.
@sonarqubecloud

Copy link
Copy Markdown

@JE-Chen
JE-Chen merged commit 3a13d89 into main Jul 23, 2026
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant