feat: Create Pydantic models for Search Pipeline Run API

[yuechao-qin]

TL;DR

Added a new filter_query parameter to the pipeline runs API with Pydantic validation models, while maintaining backward compatibility with the existing filter parameter.

What's new?

JSON definition for filter_query:

{
  // logical (any predicate)
  "<and|or>": [
    // leaf
    {"key_exists": {"key": "<KEY>"}},
    // leaf
    {"value_contains": {"key": "<KEY>", "value_substring": "<VALUE>"}},
    // leaf
    {"value_in": {"key": "<KEY>", "values": ["<V1>", "<V2>", "..."]}},
    // leaf
    {"value_equals": {"key": "<KEY>", "value": "<VALUE>"}},
    // leaf
    {"time_range": {"key": "system/pipeline_run.date.created_at", "start_time": "<START_DATE>", "end_time": "<END_DATE>"}},
    // logical (leaf only)
    {"not": {"<LEAF_PREDICATE>": {}}},
    // logical (any predicate)
    {"<and|or>": [...]},
    ...
  ]
}

Example JSON for filter_query:

{
  "and": [
    {"key_exists": {"key": "team"}},
    {"value_equals": {"key": "env", "value": "prod"}},
    {"value_in": {"key": "region", "values": ["us-east", "us-west", "eu-west"]}},
    {"not": {"key_exists": {"key": "deprecated"}}},
    {"or": [{"value_contains": {"key": "name", "value_substring": "nightly"}}]}
  ]
}

What changed?

API GET /api/pipeline_runs/

Functional

• New filter_query if used in API will return an HTTP 501 (unimplemented error)
• Only filter or filter_query (mutual exclusive) can be in the API, else it returns a HTTP 422.

Other

• Added a new filter_query parameter to ListPipelineRunsParams that accepts structured JSON queries
• Created comprehensive Pydantic models in filter_query_models.py for validating filter queries, including:
  • Leaf predicates: key_exists, value_equals, value_contains, value_in, time_range
  • Logical operators: and, or, not
  • Recursive nesting support for complex queries
• Implemented JSON validation for filter_query with proper error handling
• Enforced strict validation rules including non-empty strings and timezone-aware datetimes

How to test?

uv run pytest tests/test_api_server_sql.py tests/test_filter_query_models.py

• Test the new filter_query parameter with valid JSON structures like {"and": [{"key_exists": {"key": "team"}}]}
• Verify that using both filter and filter_query simultaneously returns a 422 error
• Confirm that invalid JSON in filter_query triggers proper validation errors
• Check that valid filter_query usage returns a 501 "not yet implemented" response
• Test edge cases like empty strings, missing timezone info, and nested logical operators

Why make this change?

This PR introduces the Pydantic models and API wiring that upstream PRs depend on to implement the Search Pipeline Run API.

[yuechao-qin]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• feat: Add include_sql to Search Pipeline Run API #135
• feat: Search pipeline run API uses pagination cursor #131
• feat: Search within date range in pipeline run API #130
• feat: Search pipeline name in pipeline run API #129
• refactor: Convert legacy filter to be filter_query #128
• feat: Search created by user in pipeline run API #108
• feat: Search annotations in pipeline run API #107
• feat: Create Pydantic models for Search Pipeline Run API #106 👈 (View in Graphite)
• refactor: Search pipeline run refactoring and baseline unit tests #105
• master

This stack of pull requests is managed by Graphite. Learn more about stacking.

[Volv-G]

looks good, just 1 question on the design choice

[Ark-kun]

Hmm. Maybe empty value is OK.

[Ark-kun]

start_time can be optional too

[Ark-kun]

if you want, you can construct Python dict and then use json.dumps to serialize it.
But JSON string like you have is OK too.

[Ark-kun]

MutuallyExclusiveFilterError is pretty specific to pipeline run search. Maybe it should be declared in that module? Or you can just use ApiValidationError
Or if you envision more APIs with mutually exclusive parameters (we avoid this), you could rename the class to MutuallyExclusiveParametersError.

[Ark-kun]

Approved with comments.