fix(integrations): Chunk the repo-activity RPC in sync_repos_for_org

[billyvg]

Fixes an unbounded cross-silo RPC payload in the periodic SCM repo sync.

• Sentry event: 1986a344223545f0af045736530b1efb
• Transaction: sentry.integrations.source_code_management.sync_repos.sync_repos_for_org (env control)
• Triage session: https://claude.ai/code/session_017YpUohVVykHZnEEdAS77dw

Root cause

_sync_repos_for_org diffs the provider's repo list against Sentry's Repository rows, then — before disabling anything — asks the cell silo which of the removed repos still have recent commit/PR/code-review activity:

removed_id_list = list(removed_ids)
if removed_id_list:
    active_skipped = set(
        repository_service.find_recently_active_repo_external_ids(
            ...,
            external_ids=removed_id_list,   # entire set, one request
            cutoff_days=DISABLE_ACTIVITY_CUTOFF_DAYS,
        )
    )

removed_ids is unbounded — it is sentry_active_ids - provider_external_ids. For a GitHub org with ~10K repos (99+ pages of /installation/repositories), that single call serializes every removed external id into one RPC POST body (~72KB on the failing event). Nothing caps it, so at scale the call dies on request-size limit / timeout / cell-side memory before a single repo is disabled, and the task never converges — every scheduled run re-does the same doomed request. The task's processing_deadline_duration=120 plus Retry(times=3) means it just burns retries.

Everything else in this task already batches through chunked(...) (create_repos_batch, disable_repos_batch, restore_repos_batch all dispatch in SYNC_BATCH_SIZE slices). This activity lookup was the one remaining unbounded payload on the path.

Fix

Chunk the candidate list and accumulate the results:

removed_id_list = sorted(eid for eid in removed_ids if eid is not None)
active_skipped: set[str] = set()
for candidate_batch in chunked(removed_id_list, ACTIVITY_LOOKUP_BATCH_SIZE):
    active_skipped.update(
        repository_service.find_recently_active_repo_external_ids(
            ..., external_ids=candidate_batch, ...
        )
    )

Notes on the shape of the change:

• New ACTIVITY_LOOKUP_BATCH_SIZE = 500 rather than reusing SYNC_BATCH_SIZE = 100. SYNC_BATCH_SIZE sizes async task dispatch, where extra batches are cheap. This lookup is a blocking round trip inside a task with a 120s processing deadline, so 100 would mean ~99 sequential RPCs for a 9.9K-repo org. 500 keeps each body around 4KB while holding the worst case to ~20 calls.
• sorted(...) instead of list(...) makes batch boundaries deterministic (easier to correlate logs/retries) and narrows the element type to str — RpcRepository.external_id is str | None, and the if eid is not None filter is a no-op at runtime since removed_ids is built only from repos with a truthy external_id.
• The if removed_id_list: guard is gone because the loop body simply doesn't execute for an empty set; the find_recently_active_repo_external_ids impl also already short-circuits on an empty list. The logging/metrics block moved out one level, which is behaviour-preserving (active_skipped is empty whenever there were no candidates).

Ruled out

• querybuilder.name: UnresolvedQuery on the event is a red herring — scope contamination from a previous task in the same worker process. No query-builder code is reachable from this task.
• Not the provider fetch. installation.get_repositories(raise_on_page_limit=True) already has a pagination cap and its ApiPaginationTruncated path is handled (partial results are used for create/restore and the disable path is skipped). That's the path that would produce a provider failure, and the event isn't it.
• Not a broken integration. Expired-token / suspended-install cases are already funnelled into _halt_broken_integration and halt without an issue, so this error is a genuine failure, not a known terminal state.
• Not fixed by a guard at the crash site. Skipping or truncating the lookup when the list is long would silently disable repos that still have activity — the opposite of what DISABLE_ACTIVITY_CUTOFF_DAYS exists to prevent. Chunking preserves the exact result set.

Known remaining risk (not addressed here)

repository_service.get_repositories(organization_id=..., integration_id=..., providers=[...]) has no limit or pagination (src/sentry/integrations/services/repository/impl.py) — it serializes every matching Repository row into the RPC response (~78KB for the org in this event). It's a read, so it doesn't have the same failure mode as the oversized request body, but it will keep growing with repo count, and disable_repos_batch / restore_repos_batch each call it again per batch. Fixing it properly means a paginated or external_ids-filtered variant of the RPC method, which is a bigger change than this hotfix and is deliberately out of scope.

Tests

Added test_activity_lookup_is_chunked to tests/sentry/integrations/source_code_management/test_sync_repos.py. It patches ACTIVITY_LOOKUP_BATCH_SIZE down to 2, removes 5 repos from the provider listing (one of which has a recent commit), spies on the RPC while delegating to the real implementation, and asserts:

• 3 requests are made and no request carries more than 2 external ids,
• the union of the batches is the full candidate set (nothing dropped),
• results accumulate across batches — the repo with recent activity stays ACTIVE, the other four end up DISABLED.

⚠️ Not run locally: this environment has no Python virtualenv (no .venv, no devenv, no Django installed), so pytest and prek could not be executed. What was verified locally: ruff check and ruff format --check clean on both files, py_compile clean, and the chunk/accumulate logic exercised standalone against a 9,900-id set (20 calls, max 500 ids per call, correct skip/disable partition). CI needs to confirm the pytest run.

Legal Boilerplate

Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. and is gonna need some rights from me in order to utilize my contributions in this here PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.

🤖 Generated with Claude Code

https://claude.ai/code/session_017YpUohVVykHZnEEdAS77dw

---

Generated by Claude Code

[github-actions]

Backend Test Failures

Failures on 29ad287 in this run:

tests/sentry/integrations/source_code_management/test_sync_repos.py::SyncReposForOrgTestCase::test_activity_lookup_is_chunked — log

[gw0] linux -- Python 3.13.1 /home/runner/work/sentry/sentry/.venv/bin/python3
tests/sentry/integrations/source_code_management/test_sync_repos.py:243: in test_activity_lookup_is_chunked
    sync_repos_for_org(self.oi.id)
src/sentry/silo/base.py:157: in override
    return original_method(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/taskbroker_client/task.py:142: in __call__
    return self._func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
src/sentry/integrations/source_code_management/sync_repos.py:159: in sync_repos_for_org
    _sync_repos_for_org(organization_integration_id)
src/sentry/integrations/source_code_management/sync_repos.py:309: in _sync_repos_for_org
    repository_service.find_recently_active_repo_external_ids(
/opt/hostedtoolcache/Python/3.13.1/x64/lib/python3.13/unittest/mock.py:1167: in __call__
    return self._mock_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/opt/hostedtoolcache/Python/3.13.1/x64/lib/python3.13/unittest/mock.py:1171: in _mock_call
    return self._execute_mock_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/opt/hostedtoolcache/Python/3.13.1/x64/lib/python3.13/unittest/mock.py:1232: in _execute_mock_call
    result = effect(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^
tests/sentry/integrations/source_code_management/test_sync_repos.py:227: in spy
    return real_find(
src/sentry/hybridcloud/rpc/service.py:357: in remote_method
    return dispatch_remote_call(
src/sentry/hybridcloud/rpc/service.py:496: in dispatch_remote_call
    return remote_silo_call.dispatch(use_test_client)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
src/sentry/hybridcloud/rpc/service.py:566: in dispatch
    serial_response = self._send_to_remote_silo(use_test_client)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
src/sentry/hybridcloud/rpc/service.py:654: in _send_to_remote_silo
    self._raise_from_response_status_error(response)
src/sentry/hybridcloud/rpc/service.py:689: in _raise_from_response_status_error
    raise self._remote_exception(
E   sentry.hybridcloud.rpc.service.RpcRemoteException: repository.find_recently_active_repo_external_ids: Error invoking rpc at '/api/0/internal/rpc/repository/find_recently_active_repo_external_ids/': check error logs for more details