feat: Deploy & Run History follow-ups — polling, metrics, deep-links, DB columns, server-side filters

[abhizipstack]

What

Implements 6 of 7 items from the deferred follow-ups ticket, plus bug fixes found during testing.

Deep-link toast → Run History (#5)

• Quick Deploy success toast now includes a clickable "View in Run History →" link (with renderMarkdown: false so JSX renders correctly).
• Run History page auto-expands the most recent run when arriving via deep-link.

Pre-fill create-job from 0-candidates (#6)

• "Go to Scheduler" CTA navigates to /project/job/list?create=1&project=<pid>&model=<name>.
• Jobs List reads params: auto-opens the create drawer with the model pre-enabled in Model Configuration.

First-class trigger + scope DB columns (#4)

• Added trigger (scheduled/manual) and scope (job/model) as real CharField columns on TaskRunHistory with DB indexes.
• Migration 0002_taskrunhistory_trigger_scope.
• trigger_scheduled_run writes both columns and kwargs (backward compat).
• Frontend getRunTriggerScope prefers top-level fields, falls back to kwargs for pre-migration rows.

Server-side Run History filtering (#7)

• task_run_history endpoint accepts ?trigger=, ?scope=, ?status= query params.
• Frontend filter changes now trigger a server-side refetch instead of client-side filtering, so pagination is accurate across all pages.

Live deploy progress polling (#3)

• After dispatch, Quick Deploy button flips to "Deploying…" with spinner.
• Polls latest run status every 5s via getLatestRunStatus.
• On terminal state: completion toast (success/failure) with Run History link, explorer refresh, recent-runs cache clear.
• Auto-cleans on unmount.

Runtime metrics in Run History (#1)

• After DAG execution (success or failure), trigger_scheduled_run serializes BASE_RESULT into TaskRunHistory.result as JSON.
• Per-model name, status, end_status; aggregate total/passed/failed counts.
• Source classes (e.g. SourceMdoela) filtered out — only user-created models appear.
• Frontend insights panel renders a metrics bar when result.total > 0.

Deferred to separate tickets:

• User-facing activity logs → OR-1462
• Row counts per model → OR-1461

Why

These follow-ups close gaps identified during the Quick Deploy and Run History work: no way to navigate from a toast to the specific run, no way to create a job from the 0-candidates flow, filters were client-side only (broke pagination), no live feedback during deploy, and no execution metrics on completed runs.

How

• Backend: migration for trigger/scope columns, query-param filtering in task_run_history, BASE_RESULT serialization in trigger_scheduled_run + _mark_failure.
• Frontend: renderMarkdown: false for JSX toasts, useSearchParams for deep-link + pre-fill, polling via setInterval + getLatestRunStatus, metrics bar in insights panel.

Can this PR break any existing features?

Low risk:

• New DB columns have defaults (scheduled / job) so existing rows are valid post-migration. Serializer uses fields = "__all__" so new columns auto-expose.
• Server-side filtering is additive — no params = no filter = same behavior as before.
• Polling is self-contained in component state; auto-cleans on unmount.
• result field was already on the model (never written); now populated. Frontend guards with result?.total > 0.
• Source class filtering uses startswith("Source") convention from the no-code model generator.

Database Migrations

• backend/backend/core/scheduler/migrations/0002_taskrunhistory_trigger_scope.py — adds trigger, scope columns + indexes.

Env Config

None.

Relevant Docs

None.

Related Issues or PRs

• PR FIX: Reduce logs panel noise and make execution messages readable (OR-1457) #59, feat: Deploy, Jobs & Run History UX improvements #61, feat: Quick Deploy button in model tab with scope selection and Run History integration #62 (merged) — this PR builds on their work
• OR-1461 — row counts per model (separate ticket, adapter-level changes)
• OR-1462 — user-facing activity logs (separate ticket, needs product sign-off)

Dependencies Versions

No changes.

Notes on Testing

Tested locally (gunicorn + Celery worker + React dev server):

1. Quick Deploy → success toast shows clickable "View in Run History →" link → navigates to Run History with job preselected + first run auto-expanded.
2. Quick Deploy on model with no job → "Go to Scheduler" → Jobs List opens with create drawer, model pre-checked.
3. Run History filters (Trigger/Scope/Status) now refetch from server — verified pagination accuracy.
4. Quick Deploy → button shows "Deploying…" spinner → polls → completion toast appears on SUCCESS/FAILURE.
5. Run History expanded row shows metrics bar: "1 model attempted · 1 passed · 0 failed · Mdoela (OK)". Source classes filtered out.
6. Migration applied cleanly; old runs render with kwargs fallback.

Checklist

I have read and understood the Contribution Guidelines.

🤖 Generated with Claude Code

[greptile-apps]

Greptile Summary

This PR completes six follow-up items from the Deploy & Run History track: live deploy polling, deep-link navigation to Run History, pre-fill create-job from the 0-candidates CTA, first-class trigger/scope DB columns with migration, server-side Run History filtering, and per-model execution metrics. Prior review concerns (stale BASE_RESULT, _mark_failure reading a cleared global, pagination reset, missing project URL param) have all been addressed.

• P1 — filterQueries.job in the server-side filter effect's dep array causes a concurrent double-fetch race on every job change, because handleJobChange and getJobList also call getRunHistoryList directly. If the stale-page response resolves last it overwrites the correct page-1 data.
• P2 — searchParams.has(\"task\") never clears, so the deep-link auto-expand fires on every data refresh rather than only on first arrival.

Confidence Score: 4/5

Safe to merge after resolving the double-fetch race in Runhistory.jsx; all prior P1 issues are fixed.

One unresolved P1 (double-fetch race on job change that can leave stale page-N data visible) prevents a 5. Backend changes, migration, polling, and pre-fill flows are solid.

frontend/src/ide/run-history/Runhistory.jsx — filter effect dep array + direct getRunHistoryList calls create concurrent requests.

Important Files Changed

Filename	Overview
frontend/src/ide/run-history/Runhistory.jsx	Adds server-side filtering, deep-link auto-expand, and metrics bar; introduces a double-fetch race condition on job change.
backend/backend/core/scheduler/celery_tasks.py	Adds BASE_RESULT metrics capture with snapshotting before clear; introduces duplicate _clean_name and _clean inner functions.
backend/backend/core/scheduler/migrations/0002_taskrunhistory_trigger_scope.py	Adds trigger/scope CharField columns with indexes; default=scheduled/job means old manual runs will be labelled scheduled in DB.
backend/backend/core/scheduler/models.py	Adds trigger and scope CharField fields + indexes matching the migration; clean and straightforward.
backend/backend/core/scheduler/views.py	Adds server-side trigger/scope/status filtering to task_run_history endpoint; additive and non-breaking.
frontend/src/ide/editor/no-code-model/no-code-model.jsx	Adds polling loop for live deploy status, JSX toast with Run History deep-link, and goToScheduler pre-fill params; cleanup on unmount is correct.
frontend/src/ide/scheduler/JobDeploy.jsx	Adds prefillModel and prefillProject props with useEffect handlers; straightforward and well-guarded.
frontend/src/ide/scheduler/JobList.jsx	Reads create/model/project URL params, opens JobDeploy drawer with pre-fill, clears params via replace.
frontend/src/ide/scheduler/service.js	Adds getLatestRunStatus fetching page 1 limit 1; correctly extracts the first run from the response.

Sequence Diagram

sequenceDiagram
    participant U as User
    participant NCM as NoCodeModel
    participant SVC as service.js
    participant API as Django API
    participant RH as RunHistory

    U->>NCM: Click Quick Deploy
    NCM->>API: POST runTask / runTaskForModel
    API-->>NCM: 200 OK dispatched
    NCM->>U: Toast Deploy Triggered + View in Run History link
    NCM->>NCM: startDeployPolling(taskId)

    loop Every 5s while non-terminal
        NCM->>SVC: getLatestRunStatus(projectId, taskId)
        SVC->>API: GET /run-history/{taskId}?page=1&limit=1
        API-->>SVC: run status object
        SVC-->>NCM: run object
    end

    NCM->>U: Toast Deploy Completed/Failed + View in Run History link
    NCM->>NCM: clearInterval + refresh caches

    U->>RH: Navigate to /project/job/history?task=taskId
    RH->>API: GET /run-history/{taskId}?page=1&limit=10
    API-->>RH: runs with trigger/scope columns
    RH->>RH: Auto-expand most recent run
    U->>RH: Change filter
    RH->>API: GET /run-history/{taskId}?trigger=X&scope=Y
    API-->>RH: Server-side filtered runs

Loading

Prompt To Fix All With AI

This is a comment left during a code review.
Path: frontend/src/ide/run-history/Runhistory.jsx
Line: 195-209

Comment:
**Double-fetch race condition on job change**

Adding `filterQueries.job` to this effect's dep array means the effect fires whenever the job changes — but `handleJobChange` (line 229) and `getJobList` (line 181) already call `getRunHistoryList` directly after calling `setFilterQuery`. This produces two concurrent requests on every job switch. The direct call uses `currentPage` (which may be > 1 from the previous job), while the effect always passes `page=1`. Whichever response resolves last wins; if the stale-page response lands second it overwrites the correct page-1 data.

The fix is to remove the direct `getRunHistoryList(value)` calls from `handleJobChange` and `getJobList` and let this effect be the sole fetch trigger.

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: frontend/src/ide/run-history/Runhistory.jsx
Line: 212-224

Comment:
**Deep-link auto-expand fires on every data refresh, not just initial arrival**

`searchParams.has("task")` stays `true` for the lifetime of the page. Because this effect depends on `backUpData`, it re-runs after every API response — page changes, filter changes, refreshes — and auto-expands `backUpData[0].id` each time. A user who navigates to page 2 will find the first item of that page unexpectedly expanded.

A `useRef` one-shot flag would limit the behaviour to arrival only:

```js
const deepLinkExpandedRef = useRef(false);
useEffect(() => {
  const ids = [];
  const fromDeepLink = !deepLinkExpandedRef.current && searchParams.has("task");
  if (fromDeepLink && backUpData.length > 0) {
    ids.push(backUpData[0].id);
    deepLinkExpandedRef.current = true;
  }
  (backUpData || [])
    .filter((r) => r.status === "FAILURE" && r.error_message)
    .forEach((r) => { if (!ids.includes(r.id)) ids.push(r.id); });
  setExpandedRowKeys(ids);
}, [backUpData, searchParams]);
```

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: backend/backend/core/scheduler/celery_tasks.py
Line: 348-353

Comment:
**Duplicate name-cleaning logic defined twice**

`_clean_name` (success path) and `_clean` (`_mark_failure`) are byte-for-byte identical. A future edit to one will silently diverge from the other. Extract to a single module-level helper:

```python
def _clean_node_name(raw: str) -> str:
    if "'" in raw:
        return raw.split("'")[1].split(".")[-1]
    return raw
```

How can I resolve this? If you propose a fix, please make it concise.

_{Reviews (5): Last reviewed commit: "fix: use filterQueries.job instead of en..." | Re-trigger Greptile}

[abhizipstack]

All 6 issues addressed in 3c44484:

1. Filters dropped on pagination (P1) — handlePagination now passes active filterQueries to getRunHistoryList.
2. BASE_RESULT stale global (P1) — Added snapshot-then-clear pattern + _clear_base_result() helper called in all exit paths (success, timeout, exception).
3. handleRefresh stale closure (P2) — Updated deps to include filterQueries, currentPage, pageSize, getRunHistoryList.
4. project URL param ignored (P2) — JobList now reads project from URL params and forwards prefillProject to JobDeploy, which pre-selects it.
5. list_recent_runs_for_model kwargs (P2) — Now prefers run.trigger/run.scope DB columns, falling back to kwargs for pre-migration rows.
6. CodeQL XSS (DOM text as HTML) — Applied encodeURIComponent() to taskId in toast deep-link URLs.

[wicky-zipstack]

A few items to consider:

1. Same repr(cls) parsing anti-pattern as #54 / #59 — celery_tasks.py:47

def _clean_name(raw):
    if "'" in raw:
        return raw.split("'")[1].split(".")[-1]

PR #59 already fixed this with cls.__name__ elsewhere. Better to write a clean name into BASE_RESULT.node_name upstream and parse zero times.

2. _clean_name duplicated — defined in both trigger_scheduled_run and _mark_failure. Extract to module level.

3. Polling has no max duration / backoff — no-code-model.jsx:370. Hardcoded 5s interval = 360 backend hits per 30-min deploy. Add a max-poll-count or exponential backoff. Also no upper-bound timeout — if backend never returns terminal status, polling runs until unmount.

4. RETRY status not handled in terminal list — no-code-model.jsx:329 — only ["SUCCESS", "FAILURE", "REVOKED"]. Button stays "Deploying…" during retries — fine if intentional, worth confirming.

5. BASE_RESULT is a module-level global — works for single-threaded Celery worker. If you ever run prefork pool with concurrency > 1 in same worker, the global will be shared/contaminated across concurrent jobs. Worth verifying worker config.

6. startswith("Source") filter is brittle — celery_tasks.py:54. A user-named SourceData model would be hidden. Use a flag on the node instead. (Tech debt — fine for now.)

[tahierhussain]

@abhizipstack Please add screenshots for the UI changes addressed in this PR.

[greptile-apps]

Double-fetch race condition on job change

Adding filterQueries.job to this effect's dep array means the effect fires whenever the job changes — but handleJobChange (line 229) and getJobList (line 181) already call getRunHistoryList directly after calling setFilterQuery. This produces two concurrent requests on every job switch. The direct call uses currentPage (which may be > 1 from the previous job), while the effect always passes page=1. Whichever response resolves last wins; if the stale-page response lands second it overwrites the correct page-1 data.

The fix is to remove the direct getRunHistoryList(value) calls from handleJobChange and getJobList and let this effect be the sole fetch trigger.

Prompt To Fix With AI

This is a comment left during a code review.
Path: frontend/src/ide/run-history/Runhistory.jsx
Line: 195-209

Comment:
**Double-fetch race condition on job change**

Adding `filterQueries.job` to this effect's dep array means the effect fires whenever the job changes — but `handleJobChange` (line 229) and `getJobList` (line 181) already call `getRunHistoryList` directly after calling `setFilterQuery`. This produces two concurrent requests on every job switch. The direct call uses `currentPage` (which may be > 1 from the previous job), while the effect always passes `page=1`. Whichever response resolves last wins; if the stale-page response lands second it overwrites the correct page-1 data.

The fix is to remove the direct `getRunHistoryList(value)` calls from `handleJobChange` and `getJobList` and let this effect be the sole fetch trigger.

How can I resolve this? If you propose a fix, please make it concise.