feat: Search created by user in pipeline run API by yuechao-qin · Pull Request #108 · TangleML/tangle

yuechao-qin · 2026-02-24T13:08:03Z

TL;DR

Implemented search created_by user in Pipeline Runs.

What changed?

Functionality

API GET /api/pipeline_runs/
- Search created_by user in filter_query. Example query (not URL encoded for example):
```
/api?filter_query={"and": [{"value_equals": {"key": "system/pipeline_run.created_by", "value": "alice@example.com"}}]}
```
- Annotation value me is supported for created_by search.
API POST /api/pipeline_runs/
- Copies the created_by data to the annotations table for future searching. Example pipeline run annotations table after API is called:
  
  pipeline_run_id key value
  
  abc123 system/pipeline_run.created_by alice@example.com
API [PUT|DELETE] /api/pipeline_runs/{id}/annotations/{key}
- Prevent (create, update, delete) of annotations with system/ prefix. Reserved for system annotation usage.

Other

Database migration: Added backfill logic to populate existing pipeline runs with created_by annotations. Example what a pipeline run's created by user would look like in the annotations table:

pipeline_run_id	key	value
42	system/pipeline_run.created_by	alice@example.com

How to test?

uv run pytest tests/test_api_server_sql.py  tests/test_filter_query_sql.py tests/test_database_ops.py

Create pipeline runs with different created_by values
Use filter queries like {"and": [{"value_equals": {"key": "system/pipeline_run.created_by", "value": "alice"}}]} to search by creator
Test the "me" placeholder: {"and": [{"value_equals": {"key": "system/pipeline_run.created_by", "value": "me"}}]}
Verify that attempts to set/delete system annotations return 422 errors
Test that unsupported predicates on system keys (like value_contains) are rejected

Why make this change?

This enables users to search for pipeline runs by creator using the new filter query system.
Following safety guards and synchronization will allow old and new data (create_by) to be searchable with the new filter.
- Preventing (create, delete, update) system prefix in annotations
- Saving created_by to annotations table when starting a Pipeline Run

yuechao-qin · 2026-02-24T13:08:22Z

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

Volv-G

looks good to me

cloud_pipelines_backend/database_ops.py

Ark-kun · 2026-03-03T08:08:52Z

cloud_pipelines_backend/database_ops.py

+    backfill_created_by_annotations(db_engine=db_engine)
+
+
+def is_annotation_key_already_backfilled(


It's a public function. Let's make it private.

The name mentions "annotation", but that work is used everywhere. We need to be more specific.

_is_pipeline_run_annotation_key_already_backfilled

Ark-kun · 2026-03-03T08:09:41Z

cloud_pipelines_backend/database_ops.py

+        ).scalar()
+
+
+def backfill_created_by_annotations(*, db_engine: sqlalchemy.Engine):


It's a public function. Let's make it private.

The name mentions "annotation", but that work is used everywhere. We need to be more specific.

_backfill_pipeline_run_created_by_annotations

Ark-kun · 2026-03-03T08:11:37Z

cloud_pipelines_backend/filter_query_sql.py

+SYSTEM_KEY_PREFIX: Final[str] = "system/"
+
+
+class SystemKey(enum.StrEnum):


PipelineRunAnnotationSystemKey

Ark-kun · 2026-03-03T08:14:56Z

cloud_pipelines_backend/filter_query_sql.py

+def _get_predicate_key(*, predicate: filter_query_models.Predicate) -> str | None:
+    """Extract the annotation key from a leaf predicate, or None for logical operators."""
+    match predicate:
+        case filter_query_models.KeyExistsPredicate():


You can have some base class like filter_query_models.KeyPredicateBase with field key. Then getting the key from the predicates would be trivial.

Ark-kun · 2026-03-03T08:24:01Z

cloud_pipelines_backend/filter_query_sql.py

    *,
-    predicate,
+    predicate: filter_query_models.Predicate,
+    current_user: str | None = None,


Maybe it's simpler to first preprocess the predicates so that you don't need to do that in functions like _predicate_to_clause ?

Ark-kun · 2026-03-03T08:26:03Z

tests/test_database_ops.py

+
+@pytest.fixture()
+def session_factory():
+    engine = sqlalchemy.create_engine(


Let's use the create_db_engine function from database_ops (without migration).

Ark-kun · 2026-03-03T08:29:12Z

tests/test_database_ops.py

+
+class TestIsAnnotationKeyAlreadyBackfilled:
+    def test_false_on_empty_db(self, session_factory):
+        engine = session_factory.kw["bind"]


Maybe, is_annotation_key_already_backfilled can just accept session_factory instead of engine?

Ark-kun · 2026-03-03T08:32:11Z

cloud_pipelines_backend/api_server_sql.py

        skip_user_check: bool = False,
    ):
+        if key.startswith(filter_query_sql.SYSTEM_KEY_PREFIX):
+            raise errors.InvalidAnnotationKeyError(_SYSTEM_KEY_RESERVED_MSG)


Maybe, self._fail_if_changing_system_annotation(key)?

This was referenced Feb 24, 2026

refactor: Search pipeline run refactoring and baseline unit tests #105

Open

feat: Create Pydantic models for Search Pipeline Run API #106

Open

feat: Search annotations in pipeline run API #107

Open

yuechao-qin marked this pull request as ready for review February 24, 2026 17:06

yuechao-qin requested a review from Ark-kun as a code owner February 24, 2026 17:06

Volv-G approved these changes Feb 25, 2026

View reviewed changes

yuechao-qin changed the base branch from ycq/search-pipeline-run-annotations to graphite-base/108 February 25, 2026 06:34

yuechao-qin force-pushed the ycq/search-pipeline-run-created-by branch from 6789217 to 01de682 Compare February 25, 2026 06:39

yuechao-qin force-pushed the graphite-base/108 branch from 41a18a2 to 7606c83 Compare February 25, 2026 06:39

yuechao-qin changed the base branch from graphite-base/108 to ycq/search-pipeline-run-annotations February 25, 2026 06:39

yuechao-qin force-pushed the ycq/search-pipeline-run-created-by branch from 01de682 to 9a3927b Compare February 25, 2026 06:43

yuechao-qin force-pushed the ycq/search-pipeline-run-annotations branch from 7606c83 to 98e7d41 Compare February 25, 2026 18:45

yuechao-qin force-pushed the ycq/search-pipeline-run-created-by branch from 9a3927b to 0e3ac9b Compare February 25, 2026 18:45

yuechao-qin force-pushed the ycq/search-pipeline-run-annotations branch from 98e7d41 to ecd7842 Compare February 26, 2026 05:34

yuechao-qin force-pushed the ycq/search-pipeline-run-created-by branch from 0e3ac9b to df1b586 Compare February 26, 2026 05:34

This was referenced Feb 26, 2026

refactor: Convert legacy filter to be filter_query #128

Open

feat: Search pipeline name in pipeline run API #129

Open

feat: Search within date range in pipeline run API #130

Open

feat: Search pipeline run API uses pagination cursor #131

Open

yuechao-qin commented Feb 27, 2026

View reviewed changes

cloud_pipelines_backend/database_ops.py Outdated Show resolved Hide resolved

yuechao-qin force-pushed the ycq/search-pipeline-run-annotations branch from ecd7842 to ba5594e Compare February 27, 2026 21:55

yuechao-qin force-pushed the ycq/search-pipeline-run-created-by branch from df1b586 to 05e42b2 Compare February 27, 2026 21:55

yuechao-qin changed the base branch from ycq/search-pipeline-run-annotations to graphite-base/108 February 28, 2026 10:23

yuechao-qin force-pushed the ycq/search-pipeline-run-created-by branch from 05e42b2 to afa68d5 Compare February 28, 2026 10:24

yuechao-qin force-pushed the graphite-base/108 branch from ba5594e to d6f04df Compare February 28, 2026 10:24

yuechao-qin changed the base branch from graphite-base/108 to ycq/search-pipeline-run-annotations February 28, 2026 10:24

yuechao-qin mentioned this pull request Feb 28, 2026

feat: Add include_sql to Search Pipeline Run API #135

Open

yuechao-qin force-pushed the ycq/search-pipeline-run-created-by branch from afa68d5 to 527caa1 Compare February 28, 2026 19:15

feat: Search created by user in pipeline run API

7b5d7cb

yuechao-qin force-pushed the ycq/search-pipeline-run-created-by branch from 527caa1 to 7b5d7cb Compare February 28, 2026 20:29

Ark-kun reviewed Mar 3, 2026

View reviewed changes

Ark-kun requested changes Mar 3, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Search created by user in pipeline run API#108

feat: Search created by user in pipeline run API#108
yuechao-qin wants to merge 1 commit intoycq/search-pipeline-run-annotationsfrom
ycq/search-pipeline-run-created-by

yuechao-qin commented Feb 24, 2026 •

edited

Loading

Uh oh!

yuechao-qin commented Feb 24, 2026 •

edited

Loading

Uh oh!

Volv-G left a comment

Uh oh!

Uh oh!

Ark-kun Mar 3, 2026

Uh oh!

Ark-kun Mar 3, 2026

Uh oh!

Ark-kun Mar 3, 2026

Uh oh!

Ark-kun Mar 3, 2026

Uh oh!

Ark-kun Mar 3, 2026

Uh oh!

Ark-kun Mar 3, 2026

Uh oh!

Ark-kun Mar 3, 2026

Uh oh!

Ark-kun Mar 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		backfill_created_by_annotations(db_engine=db_engine)


		def is_annotation_key_already_backfilled(

		).scalar()


		def backfill_created_by_annotations(*, db_engine: sqlalchemy.Engine):

		SYSTEM_KEY_PREFIX: Final[str] = "system/"


		class SystemKey(enum.StrEnum):

Conversation

yuechao-qin commented Feb 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TL;DR

What changed?

Functionality

Other

How to test?

Why make this change?

Uh oh!

yuechao-qin commented Feb 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Volv-G left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

yuechao-qin commented Feb 24, 2026 •

edited

Loading

yuechao-qin commented Feb 24, 2026 •

edited

Loading