criticality: internal + assume: ["true"] silently skips check execution

[buger]

Bug Description

When a check is configured with criticality: internal and assume: ["true"], the check execution is silently skipped. Instead of running the workflow and returning real results, Visor returns empty/assumed results immediately (~0.1s instead of normal 5-30s for AI workflows).

Steps to Reproduce

checks:
  chat:
    type: workflow
    workflow: assistant
    criticality: internal
    assume: ["true"]
    args:
      question: "{{ conversation.current.text }}"

Run with --no-mocks against a real AI provider. The workflow completes in ~0.1s with empty text output instead of actually executing the AI steps.

Expected Behavior

The check should execute the workflow normally. assume: ["true"] should serve as a precondition guard (as documented for critical steps), not cause the check to be skipped entirely.

Actual Behavior

• Check returns immediately with empty results
• No AI API calls are made
• intent-router outputs are all null
• generate-response output is {"text": ""}
• No trace files are generated

Workaround

Use criticality: policy (default) instead, which executes normally and supports on_fail.retry for auto-retries on all failure types.

Environment

• Visor v0.1.175
• Model: gemini-3-flash-preview
• Node.js v22.14.0

[probelabs]

Bug Analysis: criticality: internal + assume: ["true"] Silent Skip

Thank you for the detailed bug report. I've investigated the codebase and can confirm this is a real bug in how assume expressions are evaluated for workflow-type checks with criticality: internal.

What I Found

Root Cause

The issue stems from how assume expressions are evaluated in the state machine. Looking at the execution path in visor/src/state-machine/states/level-dispatch.ts (lines 2364-2420), the assume contract is evaluated before the workflow provider executes:

// Evaluate assume contract (design-by-contract) before executing
{
  const assumeExpr = (checkConfig as any)?.assume as string | string[] | undefined;
  if (assumeExpr) {
    let ok = true;
    try {
      const evaluator = new FailureConditionEvaluator();
      const exprs = Array.isArray(assumeExpr) ? assumeExpr : [assumeExpr];
      // Get conversation from execution context (TUI/CLI) or provider event context (Slack)
      const conversation =
        (context.executionContext as any)?.conversation ||
        (providerConfig as any)?.eventContext?.conversation;
      for (const ex of exprs) {
        const res = await evaluator.evaluateIfCondition(checkId, ex, {
          event: context.event || 'manual',
          previousResults: dependencyResults as any,
          conversation,
        } as any);
        if (!res) {
          ok = false;
          break;
        }
      }
    } catch (error) {
      const msg = error instanceof Error ? error.message : String(error);
      logger.error(`Failed to evaluate assume expression for check '${checkId}': ${msg}`);
      // Fail-secure: if assume evaluation fails, skip execution
      ok = false;
    }
    if (!ok) {
      logger.info(
        `⏭  Skipped (assume: ...)`
      );
      // ... returns empty result immediately
    }
  }
}

The Problem

When assume: ["true"] is set on a workflow check:

1. The expression "true" is evaluated as a JavaScript expression in the sandbox context
2. For workflow-type checks, the evaluation context may not have the expected variables available
3. If evaluation fails or returns a falsy value, the check is immediately skipped with an empty result
4. No provider execution occurs - the workflow never runs

This explains the ~0.1s completion time (just the expression evaluation, no AI calls) and empty outputs.

Why criticality: internal Matters

The documentation in visor/docs/guides/criticality-modes.md states that criticality: internal is for "control-plane" steps that steer execution. The config validation in visor/src/config.ts (lines 1274-1305) enforces that critical steps require assume or if guards:

if (isCritical) {
  // 1) Pre-run guard: either assume (string or non-empty array) or if
  const hasAssume =
    typeof (checkConfig as any).assume === 'string' ||
    (Array.isArray((checkConfig as any).assume) && (checkConfig as any).assume.length > 0);
  const hasIf =
    typeof (checkConfig as any).if === 'string' && (checkConfig as any).if.trim().length > 0;
  if (!hasAssume && !hasIf) {
    errors.push({
      field: `checks.${checkName}.assume`,
      message: `Critical step "${checkName}" (criticality: ${crit}) requires a precondition...`,
    });
  }
}

However, the assumption that assume: ["true"] would always pass is incorrect - it's still evaluated as a JavaScript expression, and if the evaluation context doesn't support it properly, it can fail.

Clarifying Questions

To help diagnose the exact failure mode, could you provide:

1. Full debug output with --debug flag - this would show the actual assume evaluation result and any error messages
2. The workflow definition for the assistant workflow - what are its steps and what does it expect as inputs?
3. Whether the issue persists if you change assume: ["true"] to a more explicit expression like assume: ["true === true"] or remove it entirely

Workaround Analysis

Your workaround of using criticality: policy works because:

1. policy is the default criticality when not specified (see visor/src/state-machine/context/build-engine-context.ts lines 19-28)
2. Policy steps don't require assume guards during validation (only external and internal do)
3. Without the assume guard, the workflow executes normally

Recommended Solutions

Option 1: Remove assume: ["true"] (Immediate)

Since "true" as an expression doesn't add any meaningful guard, simply remove it:

checks:
  chat:
    type: workflow
    workflow: assistant
    criticality: internal
    # Remove: assume: ["true"]
    args:
      question: "{{ conversation.current.text }}"

Option 2: Use a Meaningful Guard

Replace with a condition that actually validates prerequisites:

checks:
  chat:
    type: workflow
    workflow: assistant
    criticality: internal
    assume:
      - "conversation != null"
      - "conversation.current != null"
      - "conversation.current.text != null"
    args:
      question: "{{ conversation.current.text }}"

Option 3: Use criticality: policy (Your Workaround)

If this check doesn't actually steer execution or set flags for other checks, policy may be more appropriate:

checks:
  chat:
    type: workflow
    workflow: assistant
    criticality: policy  # Instead of internal
    args:
      question: "{{ conversation.current.text }}"

Potential Code Fix

The engine should handle the literal string "true" more gracefully in assume expressions. A potential fix would be in visor/src/failure-condition-evaluator.ts to detect and short-circuit obvious tautologies:

// In evaluateIfCondition, before sandbox evaluation
if (expression.trim() === 'true') {
  return true; // Short-circuit obvious tautology
}

However, this would only be a band-aid. The real issue is that assume: ["true"] is semantically meaningless - if you want no guard, don't set assume at all.

Summary

• Confirmed bug: assume: ["true"] on workflow checks with criticality: internal causes silent skips
• Root cause: Expression evaluation fails or returns unexpected result, triggering skip before provider execution
• Immediate fix: Remove the meaningless assume: ["true"] guard or use criticality: policy
• Long-term: Engine should handle literal boolean expressions more gracefully

---

References:

• visor/src/state-machine/states/level-dispatch.ts:2364-2420 - Assume contract evaluation before provider execution
• visor/src/failure-condition-evaluator.ts:23-908 - Expression evaluation in sandbox
• visor/src/config.ts:1274-1305 - Criticality validation requiring assume/if guards
• visor/src/state-machine/context/build-engine-context.ts:19-28 - Criticality defaults application
• visor/docs/guides/criticality-modes.md - Criticality mode documentation

---

Powered by Visor from Probelabs

💡 TIP: You can chat with Visor using /visor ask <your question>