Skip to content

feat(ml): filter algorithm list by project pipelines#1249

Merged
mihow merged 2 commits into
mainfrom
worktree-algorithms-filtered-by-project-pipelines
Apr 17, 2026
Merged

feat(ml): filter algorithm list by project pipelines#1249
mihow merged 2 commits into
mainfrom
worktree-algorithms-filtered-by-project-pipelines

Conversation

@mihow

@mihow mihow commented Apr 17, 2026

Copy link
Copy Markdown
Collaborator

Summary

Scope the /api/v2/ml/algorithms/?project_id=<id> list to algorithms that belong to pipelines with an enabled ProjectPipelineConfig for that project. Previously the endpoint returned every Algorithm in the database regardless of project_id, while ml/pipelines/ and ml/processing_services/ already filtered by project. This aligns the algorithms page at /projects/<id>/algorithms with the pipelines and processing services pages.

  • ami/ml/views.py: AlgorithmViewSet.get_queryset() filters by pipelines__project_pipeline_configs__project=<project>, enabled=True when a project_id is supplied. No filter when omitted, so admin/debug views and the AlgorithmCategoryMap cross-reference keep working.
  • AlgorithmViewSet.list() gets the project_id_doc_param schema annotation for parity with the pipeline/processing-service endpoints.
  • Detail (/ml/algorithms/<id>/) is intentionally left unscoped — clicking an algorithm reference from a classification in a project whose pipeline is no longer enabled still resolves.
  • ami/ml/tests.py: adds TestAlgorithmViewSetProjectFilter covering the enabled-only scoping, isolation between projects, and unchanged behavior when project_id is omitted.

Scope kept deliberately narrow

The user flagged that algorithms can also be referenced from historical classifications on a project whose pipeline has since been disabled. I kept this first pass simple: the list is scoped to currently enabled pipelines only, and the detail endpoint stays unscoped so those historical classification links don't 404.

Follow-up options (not in this PR)

In order of increasing effort / risk, if we want the list to also reflect historical usage:

  1. Include disabled configs. Drop enabled=True so the list shows every pipeline that has any ProjectPipelineConfig for the project, enabled or not. One-line change. Matches the UX where a disabled pipeline's algorithm still appears because the project has touched it.
  2. Union with pipelines touched by occurrences in the project. filter(Q(pipelines__project_pipeline_configs__project=<p>, enabled=True) | Q(classifications__detection__occurrence__project=<p>)).distinct(). Lets the list surface algorithms that processed data in the project even if the pipeline was never formally configured. The classifications → detection → occurrence join is the expensive piece; worth benchmarking on a project with many occurrences before shipping.
  3. Scope detail endpoint too, with a 'public pipeline' escape hatch. Treat pipelines attached to >1 project (or a public flag) as globally visible; otherwise gate detail on membership in at least one project whose configs include the pipeline. More invasive — needs a policy decision on what 'generally available' means for a pipeline.

Happy to follow up with option 1 or 2 once the team picks a direction.

Test plan

  • New unit tests pass: docker compose -f docker-compose.ci.yml run --rm django python manage.py test ami.ml.tests.TestAlgorithmViewSetProjectFilter --keepdb
  • TestProjectPipelinesAPI still passes (no regression in neighboring project-scoping logic)
  • Manual UI check: visit /projects/<id>/algorithms on the local stack for a project with both enabled and disabled pipelines; confirm only enabled-pipeline algorithms appear
  • Manual check: click through to an algorithm detail page from a classification whose pipeline is disabled in the current project; confirm it still loads

Summary by CodeRabbit

  • New Features
    • The algorithms endpoint now supports filtering by project_id query parameter, returning only algorithms from pipelines enabled for that specific project.
    • Improved API documentation for project filtering functionality.

Scope AlgorithmViewSet.list() by the active project: only return
algorithms belonging to pipelines that have an enabled
ProjectPipelineConfig for the requested project_id. Mirrors the
existing PipelineViewSet filtering pattern so the /ml/algorithms/
list in the UI matches /ml/pipelines/ and /ml/processing_services/.

Unscoped requests (no project_id) are unchanged — they still list all
algorithms — so admin/debug usage and existing AlgorithmCategoryMap
endpoints that cross-reference algorithms keep working. Detail
(/ml/algorithms/<id>/) is also unchanged, so clicking an algorithm
reference from a classification in a project where the pipeline is
now disabled continues to resolve.

Co-Authored-By: Claude <noreply@anthropic.com>
@netlify

netlify Bot commented Apr 17, 2026

Copy link
Copy Markdown

Deploy Preview for antenna-preview canceled.

Name Link
🔨 Latest commit 494af31
🔍 Latest deploy log https://app.netlify.com/projects/antenna-preview/deploys/69e29b63ba8dcc00083cc699

@netlify

netlify Bot commented Apr 17, 2026

Copy link
Copy Markdown

Deploy Preview for antenna-ssec canceled.

Name Link
🔨 Latest commit 494af31
🔍 Latest deploy log https://app.netlify.com/projects/antenna-ssec/deploys/69e29b63ea7c64000861b70e

@coderabbitai

coderabbitai Bot commented Apr 17, 2026

Copy link
Copy Markdown
Contributor
📝 Walkthrough

Walkthrough

Adds project-based filtering to the algorithms endpoint. The AlgorithmViewSet.get_queryset() method now filters algorithms to those in enabled pipelines for a specific project when provided, and the list() method is explicitly overridden with OpenAPI documentation. Test suite verifies filtering behavior with and without project context.

Changes

Cohort / File(s) Summary
Production Logic
ami/ml/views.py
Modified get_queryset() to filter algorithms by active project through enabled pipeline configs; added explicit list() override with @extend_schema for OpenAPI parameter documentation.
Test Coverage
ami/ml/tests.py
New TestAlgorithmViewSetProjectFilter test suite with setup of dual projects/pipelines and assertions validating algorithm filtering behavior with and without project_id parameter.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~15 minutes

Suggested labels

ml, backend

Poem

🐰 A filter so fine, by project divine!
Algorithms now grouped in pipelines that line.
With enabled and true, the test suite's debut,
No queries run wild—just the right view! 🎯

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 16.67% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarizes the main change: filtering the algorithm list by project pipelines, matching the core functionality implemented in the PR.
Description check ✅ Passed The pull request description comprehensively covers all required sections: a clear summary of the changes, a detailed list of modifications, related technical context, testing approach, and thoughtful discussion of scope and follow-up options.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch worktree-algorithms-filtered-by-project-pipelines

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@mihow mihow marked this pull request as ready for review April 17, 2026 20:31
Copilot AI review requested due to automatic review settings April 17, 2026 20:31
@mihow

mihow commented Apr 17, 2026

Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Apr 17, 2026

Copy link
Copy Markdown
Contributor
✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Scopes the ML algorithms list endpoint to the active project so the algorithms page aligns with existing project-scoped ML pipeline and processing-service endpoints.

Changes:

  • Update AlgorithmViewSet.get_queryset() to filter algorithms to those in pipelines with an enabled ProjectPipelineConfig for the requested project_id.
  • Add OpenAPI schema annotation for project_id on the algorithms list() endpoint.
  • Add API tests to verify enabled-only scoping by project and unchanged unscoped behavior.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
ami/ml/views.py Applies project-based filtering for algorithm listings and documents project_id on the list endpoint.
ami/ml/tests.py Adds test coverage for project-scoped algorithm listing behavior (enabled-only, per-project isolation, unscoped unchanged).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread ami/ml/views.py Outdated
Comment thread ami/ml/tests.py Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@ami/ml/views.py`:
- Around line 59-64: The project-scoped filter in AlgorithmViewSet.get_queryset
unconditionally narrows results causing detail/retrieve endpoints to 404; modify
get_queryset to only apply the pipelines__project_pipeline_configs__project and
enabled=True filter when self.action == 'list' (i.e., keep detail/retrieve
unscoped). Locate the get_queryset method and wrap or gate the existing qs =
qs.filter(...) block so it only runs for the 'list' action, leaving other
actions (retrieve) to return the unfiltered queryset.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 36183fdf-864a-4e96-8060-1b5736dec613

📥 Commits

Reviewing files that changed from the base of the PR and between 2714195 and c639bbc.

📒 Files selected for processing (2)
  • ami/ml/tests.py
  • ami/ml/views.py

Comment thread ami/ml/views.py Outdated
…_with_params

Addresses review feedback on #1249:

- AlgorithmViewSet.get_queryset now applies the project-pipeline filter
  only when self.action == "list". Detail (retrieve) stays unscoped so
  /ml/algorithms/<id>/?project_id=<p> still resolves for historical
  classification links whose pipeline is no longer enabled.
- Test helper now builds URLs via reverse_with_params("api:algorithm-list")
  to match the pattern used elsewhere in ami/ml/tests.py.
- New test confirms the detail endpoint returns the algorithm even when
  project_id refers to a project whose pipeline is disabled for it.

Co-Authored-By: Claude <noreply@anthropic.com>
@mihow

mihow commented Apr 17, 2026

Copy link
Copy Markdown
Collaborator Author

Works! Before & after:

Screenshot from 2026-04-17 16-52-33 Screenshot from 2026-04-17 16-52-44

@mihow mihow merged commit d0277da into main Apr 17, 2026
7 checks passed
@mihow mihow deleted the worktree-algorithms-filtered-by-project-pipelines branch April 17, 2026 23:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants