Fix legacy Datastore access in data_handler.py for UWORKERs#5294
Merged
Conversation
Collaborator
Author
|
Changes have been merged into |
Collaborator
Author
|
Metrics are logs are ok. Changes have been in |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background
As part of the migration of ClusterFuzz to a scalable, isolated LUCI Swarming architecture, untrusted execution logic is handled by the UWORKER. The UWORKER operates within a highly sandboxed Docker container using a restricted IAM service account (untrusted-worker@...) that explicitly lacks Google Cloud Datastore read/write permissions.
During the teardown and log-upload phases of native Android fuzzing tasks (and potentially others), the UWORKER frequently crashed with google.api_core.exceptions.PermissionDenied (403) exceptions. Investigation revealed that several legacy monolithic utility functions in data_handler.py were still attempting raw Datastore queries (e.g.,
data_types.Job.query(...).get()) to fetch metadata about the fuzzing environment.
Proposal
This PR comprehensively patches data_handler.py to enforce the UWORKER security boundary. We implement short-circuit checks (if environment.is_uworker():) at the top of the following functions:
Instead of hitting the database, these functions now pull the requested metadata directly from the UWORKER's local environment variables (which are securely pre-injected by the Trusted Worker (TWORKER) before the container boots).
Testing