Skip to content

Fix legacy Datastore access in data_handler.py for UWORKERs#5294

Merged
jardondiego merged 7 commits into
masterfrom
fix-uworker-get-component-name
May 28, 2026
Merged

Fix legacy Datastore access in data_handler.py for UWORKERs#5294
jardondiego merged 7 commits into
masterfrom
fix-uworker-get-component-name

Conversation

@jardondiego

@jardondiego jardondiego commented May 26, 2026

Copy link
Copy Markdown
Collaborator

Background

As part of the migration of ClusterFuzz to a scalable, isolated LUCI Swarming architecture, untrusted execution logic is handled by the UWORKER. The UWORKER operates within a highly sandboxed Docker container using a restricted IAM service account (untrusted-worker@...) that explicitly lacks Google Cloud Datastore read/write permissions.

During the teardown and log-upload phases of native Android fuzzing tasks (and potentially others), the UWORKER frequently crashed with google.api_core.exceptions.PermissionDenied (403) exceptions. Investigation revealed that several legacy monolithic utility functions in data_handler.py were still attempting raw Datastore queries (e.g.,
data_types.Job.query(...).get()) to fetch metadata about the fuzzing environment.

Proposal

This PR comprehensively patches data_handler.py to enforce the UWORKER security boundary. We implement short-circuit checks (if environment.is_uworker():) at the top of the following functions:

  1. get_value_from_job_definition
  2. get_component_name
  3. get_repository_for_component
  4. get_value_from_fuzzer_environment_string

Instead of hitting the database, these functions now pull the requested metadata directly from the UWORKER's local environment variables (which are securely pre-injected by the Trusted Worker (TWORKER) before the container boots).

Testing

  • Unit Tests: Added tests to explicitly mock is_uworker and assert that Datastore queries (ndb.query) are never invoked when executing in an untrusted context
  • End-to-End Validation: Fully verified on a live chrome-development instance. Successfully executed an end-to-end Swarming task (android_content_shell_native_job), confirming the UWORKER correctly detects a timeout, packages its logs, bypasses the Datastore traps, and exits cleanly with Code: 0.

@jardondiego jardondiego marked this pull request as ready for review May 26, 2026 23:48
@jardondiego jardondiego requested a review from a team as a code owner May 26, 2026 23:48

@IvanBM18 IvanBM18 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jardondiego

Copy link
Copy Markdown
Collaborator Author

Changes have been merged into dev for testing, will monitor metrics and logs for stability.

@fernandofloresg fernandofloresg left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@jardondiego

Copy link
Copy Markdown
Collaborator Author

Metrics are logs are ok. Changes have been in dev for at least 16 hours.

@javanlacerda javanlacerda left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@jardondiego jardondiego merged commit a1492bf into master May 28, 2026
14 checks passed
@jardondiego jardondiego deleted the fix-uworker-get-component-name branch May 28, 2026 17:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants