Skip to content

feat: add search by attached data asset for capsules and pipelines - #78

Open
arielleleon wants to merge 1 commit into
codeocean:mainfrom
arielleleon:feat/search-by-data-asset
Open

feat: add search by attached data asset for capsules and pipelines#78
arielleleon wants to merge 1 commit into
codeocean:mainfrom
arielleleon:feat/search-by-data-asset

Conversation

@arielleleon

Copy link
Copy Markdown

Closes #77.

What

Adds a convenience method to each resource client for finding the capsules or pipelines that
have a given data asset attached:

capsules  = client.capsules.search_capsules_by_data_asset(data_asset_id)   # -> list[Capsule]
pipelines = client.pipelines.search_pipelines_by_data_asset(data_asset_id) # -> list[Capsule]

An empty list means the data asset is not attached to anything the caller can access, so one
call answers both "is it attached" and "what is it attached to". The main use case is checking
whether anything still depends on a data asset before archiving or deleting it.

Why

The pieces already exist, but the caller has to know that a bare data asset ID works as a
CapsuleSearchParams.query which isn't discoverable from that field's documentation and
has to drive pagination themselves, separately for capsules and pipelines.

Implementation notes

  • Uses the existing POST capsules/search and POST pipelines/search endpoints, so
    MIN_SERVER_VERSION is unchanged.
  • Delegates to search_capsules_iterator, so all pages are collected rather than just the
    first.
  • Follows the Capsules._route pattern, so Pipelines.search_pipelines_by_data_asset is a
    one-line forwarder and both resources stay in sync.
  • No new models, so no additions to the backward-compatibility re-export blocks.
  • No CHANGELOG.md or version changes, matching how other feature PRs here leave those to the
    release PR. Happy to add a changelog entry if you'd rather have it in this PR.

Testing

tests/test_search_by_data_asset.py follows the mock-session style of test_git_sync.py and
covers: the data asset ID being sent as the search query on the correct route, the
not-attached case returning an empty list, pagination across multiple pages, and the pipeline
variant hitting pipelines/search.

The assertions check the route and the query/next_token keys rather than the whole request
body, so adding fields to CapsuleSearchParams later won't break them.

hatch run lint   # clean
hatch run test   # 20 tests, OK

Tested locally on Python 3.11; the 3.9, 3.13 matrix is left to CI.

Verification against a live deployment

The unit tests are mock-based, so I also exercised the methods against a real 4.x deployment
(read-only calls) to check that a search for a data asset ID behaves like an attachment lookup:

  • No false positives observed. For a data asset attached in practice, the search returned
    5 capsules, and every one of them had a computation that had mounted that asset.
  • Matching is ID-specific. A search for a random UUID returned 0 hits.
  • Free text does cover capsule text. Searching for a capsule's own name returns that
    capsule, so the same query field matches names/descriptions/tags as well as attachments.
  • Two capsules were not returned. Across 7 capsule/asset pairs where a computation had
    mounted the asset, 5 capsules came back from a search for that asset and 2 did not. That is
    consistent with the asset having been detached from those capsules after those runs, which
    would be the correct result for a "currently attached" lookup.

Open questions

  1. Naming and placement. I put these on Capsules/Pipelines to keep each module aligned
    with its route, but client.data_assets.list_attached_capsules(...) may read better given
    the question starts from a data asset. Easy to move.
  2. Return type. Returns list[Capsule] for a straightforward truthiness check. I can add
    an ..._iterator variant instead, or as well, to match the existing search method pairs.
  3. Search index completeness. Is a search for a data asset ID guaranteed to return every
    capsule that currently has it attached, or can the index lag or miss cases? This decides how
    strongly the method can be documented: the intended use is checking whether anything still
    depends on a data asset before archiving it, and there an empty result that should not have
    been empty is the costly direction. If the index isn't authoritative for this, I'm happy to
    document the caveat, or to build the check on a different endpoint if there's a better one.

Add Capsules.search_capsules_by_data_asset and
Pipelines.search_pipelines_by_data_asset, which answer "is this data
asset attached to anything, and to what" by searching the capsules and
pipelines routes for a data asset ID and collecting every page of
matches.

Uses the existing POST capsules/search and pipelines/search endpoints,
so no Min-Server-Version change is required.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings September 4, 2026 23:27

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The new public docstrings and one test fixture are slightly misleading/unclear and should be tightened for accuracy/readability before merge.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds a convenient, paginated “search by attached data asset” helper on the Capsules and Pipelines resource clients (closing #77), plus unit tests covering request routing and pagination behavior.

Changes:

  • Add Capsules.search_capsules_by_data_asset(data_asset_id) -> list[Capsule] implemented via the existing search iterator.
  • Add Pipelines.search_pipelines_by_data_asset(data_asset_id) -> list[Capsule] as a one-line forwarder following the existing route-delegation pattern.
  • Add mock-session tests validating query placement, empty results, pagination, and the pipelines route.
File summaries
File Description
src/codeocean/capsule.py Adds search_capsules_by_data_asset convenience wrapper around the existing search iterator.
src/codeocean/pipeline.py Adds search_pipelines_by_data_asset forwarding helper to keep pipelines behavior in sync with capsules.
tests/test_search_by_data_asset.py Adds mock-based tests for both capsule and pipeline variants, including pagination.
Review details
  • Files reviewed: 3/3 changed files
  • Comments generated: 3
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/codeocean/capsule.py
Comment on lines +130 to +142
"""Find the capsules that currently have a given data asset attached.

Searches for the data asset ID across all capsules accessible to the caller,
following pagination until every match is collected. Results are whatever capsule
search reports for the ID, so an empty list means no capsule accessible to the
caller is reported as having the data asset attached.

Args:
data_asset_id: ID of the data asset to look for

Returns:
Capsules reported as having the data asset attached
"""
Comment thread src/codeocean/pipeline.py
Comment on lines +83 to +95
"""Find the pipelines that currently have a given data asset attached.

Searches for the data asset ID across all pipelines accessible to the caller,
following pagination until every match is collected. Results are whatever pipeline
search reports for the ID, so an empty list means no pipeline accessible to the
caller is reported as having the data asset attached.

Args:
data_asset_id: ID of the data asset to look for

Returns:
Pipelines reported as having the data asset attached
"""
Comment on lines +87 to +96
"""The pipeline variant searches the pipelines route."""
session = self._mock_session({"has_more": False, "results": [CAPSULE]})
pipelines = Pipelines(client=session)

results = pipelines.search_pipelines_by_data_asset("asset-789")

[(route, body)] = self._posted(session)
self.assertEqual(route, "pipelines/search")
self.assertEqual(body["query"], "asset-789")
self.assertEqual([p.id for p in results], ["cap-123"])
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature request: find which capsules/pipelines a data asset is attached to

2 participants