feat: implement deterministic exit codes and structured machine outpu… by KhushviB · Pull Request #173 · Azure/CosmosDBShell

KhushviB · 2026-07-11T18:15:38Z

Description (#155 )

This PR implements the deterministic machine output contract (Wave 1, P0) required for the Agentic & Automation Roadmap. It ensures that the Cosmos DB Shell can be safely consumed by downstream CI/CD pipelines, automation tools, and agentic wrappers without output corruption or ambiguous failure states.

Key Changes

Global --output and --quiet Flags:
- Added support for --output json, ndjson, table, and csv.
- Added a --quiet (-q) flag that suppresses all standard informational output (banners, connection logs, etc.).
Deterministic Exit Codes:
- 0: Success
- 1: Generic error
- 2: Bad arguments / Parser errors
- 3: Auth / Connection failure
- 4: Not found
- 5: Throttled / RU budget exceeded
Machine Mode Protections:
- When --output json|ndjson or --quiet are detected, the shell now implicitly disables ANSI color code injections (ColorSystem = NoColors) to prevent STDOUT corruption.
- When --output json|ndjson is specified, it inherently behaves as --quiet to guarantee that no interactive text logs interleave with the JSON objects.
Structured Error Handling on stderr:
- Any errors thrown during initialization (connection failures) or shell execution are gracefully mapped to their appropriate ExitCode and emitted as a structured JSON object ({"status": "error", "error": "..."}) to STDERR when in machine modes.

Validation / Acceptance Criteria Met

-c and -f runs produce parseable NDJSON implicitly.
No ANSI characters are rendered in machine modes.
Documented and stable exit codes correctly propagate to %ERRORLEVEL% / $LASTEXITCODE.
Tested parser error interception (returns Exit Code 2).
Tested early startup connection failure (returns Exit Code 1 / 3, formats to JSON).

…t (A1) - Add --output json/ndjson/table and --quiet flags - Implement deterministic exit codes (0: success, 1: generic, 2: bad args, 3: auth, 4: not found, 5: throttled) - Emit structured JSON errors on stderr during machine modes - Automatically disable ANSI colors and suppress informational output in machine modes

Copilot

Pull request overview

Implements a first-cut “machine output contract” for the Cosmos DB Shell by introducing global --output/--quiet flags, adding NDJSON output support, and mapping error conditions to deterministic exit codes intended for CI/CD and automation consumption.

Changes:

Adds root CLI flags --output/-o and --quiet/-q, and forces NoColors + quiet behavior for machine modes.
Introduces OutputFormat.Ndjson plus NDJSON formatting support for JSON-like command results.
Adds exit-code plumbing via CommandState.ExitCode, ParserErrorCommandState.ExitCode = 2, and ErrorCommandState.ExitCode mapping for Cosmos SDK failures and argument errors.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
CosmosDBShell/Program.cs	Adds root options, machine-mode detection, defaulting to NDJSON in non-interactive runs, and maps parse/connect failures to deterministic exit codes.
CosmosDBShell/lang/en.ftl	Adds help strings for `--output` and `--quiet`.
CosmosDBShell/Azure.Data.Cosmos.Shell.Core/ShellInterpreter.cs	Implements `Quiet` suppression for `ShellInterpreter.Write*` and emits structured JSON errors to stderr in machine modes.
CosmosDBShell/Azure.Data.Cosmos.Shell.Core/ParserErrorCommandState.cs	Sets parser-error command states to exit code 2.
CosmosDBShell/Azure.Data.Cosmos.Shell.Core/OutputFormat.cs	Adds `Ndjson` output format enum value.
CosmosDBShell/Azure.Data.Cosmos.Shell.Core/ErrorCommandState.cs	Adds exit-code mapping based on exception type / Cosmos status codes.
CosmosDBShell/Azure.Data.Cosmos.Shell.Core/CommandState.cs	Adds `ExitCode` and NDJSON formatting logic; extends `SetFormat` to accept `ndjson`.

Comments suppressed due to low confidence (1)

CosmosDBShell/Azure.Data.Cosmos.Shell.Core/CommandState.cs:70

For an invalid format string, SetFormat currently throws ShellException, which maps to exit code 1 via ErrorCommandState. Since an invalid --output value is a bad-arguments case, throwing an ArgumentException here will allow it to deterministically map to exit code 2.

        {
            throw new ShellException(MessageService.GetString("error-invalid_output_format", new Dictionary<string, object> { { "format", outputFormat } }));
        }

- Handle root System.CommandLine parser errors via JSON serialization to STDERR in machine modes - Propagate COSMOSDB_SHELL_FORMAT automatically to command formatters - Ignore case on JSON/NDJSON format checks - Bypass interactive RBAC prompts in quiet/machine modes - Prevent trailing blank lines in JSON serialization output - Add integration tests for deterministic machine exit codes - Update CLI argument tables in README

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 5 comments.

mkrueger · 2026-07-13T09:04:27Z

Can you fix/comment/close the copilot issues - otherwise we can't merge.

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 7 comments.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 6 comments.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

CosmosDBShell/Program.cs:165

When --output json|ndjson or --quiet is used, this early --otel validation error still writes plain text to STDOUT (and ShellInterpreter.WriteLine can't honor o.Quiet yet because ShellInterpreter.Instance.Options is assigned later). This breaks the machine-output contract by contaminating STDOUT; emit the structured error JSON to STDERR in machine mode.

                    Environment.ExitCode = 2;
                    ShellInterpreter.WriteLine(MessageService.GetArgsString(
                        "otel-error-invalid-endpoint", "endpoint", o.OtlpEndpoint));
                    return;
                }

…ne mode

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

When --output json or --output ndjson is selected, route AnsiConsole output through TextWriter.Null using a dedicated AnsiConsoleSettings instance. This prevents accidental AnsiConsole writes from commands from polluting machine-readable stdout, while command-state output still uses Console.Out. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

- Apply startup machine-mode handling to quiet mode as well as json/ndjson. - Route startup MCP errors through structured stderr output in machine mode instead of unconditional AnsiConsole stdout writes. - Keep interactive behavior unchanged for non-machine runs. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

- Treat startup machine mode consistently for quiet/json/ndjson and suppress startup informational stdout (clear-history/theme warnings). - Route startup machine-mode errors through structured stderr JSON and return exit code 2 for invalid --mcp port argument. - In PrintState, bypass AnsiConsole markup rendering in machine mode so JSON/ text outputs always go through Console.Out. - Expand process integration coverage for implicit -c ndjson output, machine- mode clear-history behavior, and invalid --mcp port behavior. - Clarify README machine-mode wording to avoid over-promising stdout guarantees. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated no new comments.

KhushviB · 2026-07-14T20:08:09Z

@mkrueger FInally defeated the endboss 🙌🏻

mkrueger · 2026-07-17T10:24:39Z

Ah ok - I enabled auto merge - let's try the slot machine.

EDIT: Integration tests are failing :( - unfortunately I need to manually starting it. Running integration tests locally would require the azure cosmos db emulator to run.

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (2)

CosmosDBShell/Azure.Data.Cosmos.Shell.Core/ShellInterpreter.cs:1917

Machine-mode error emission here returns early after writing to Console.Error, which bypasses ErrOutRedirect/AppendErrRedirection. That changes behavior for scripts that redirect stderr (e.g., 2> errors.json) and can break existing redirection semantics in automation.

        if (this.Options?.Quiet == true || string.Equals(this.Options?.Output, "json", StringComparison.OrdinalIgnoreCase) || string.Equals(this.Options?.Output, "ndjson", StringComparison.OrdinalIgnoreCase))
        {
            var errObj = new
            {
                status = "error",
                error = e.Message,
            };
            Console.Error.WriteLine(JsonSerializer.Serialize(errObj));
            return;
        }

CosmosDBShell/Azure.Data.Cosmos.Shell.Core/ShellInterpreter.cs:2166

In machine mode this branch writes parser errors directly to Console.Error and returns, bypassing the shell’s internal stderr redirection (ErrOutRedirect / 2>). That makes redirected error logs incomplete/non-deterministic when running with --quiet/--output json|ndjson.

        if ((this.Options?.Quiet == true
             || string.Equals(this.Options?.Output, "json", StringComparison.OrdinalIgnoreCase)
             || string.Equals(this.Options?.Output, "ndjson", StringComparison.OrdinalIgnoreCase))
            && errors != null && errors.Count > 0)
        {
            var errorStrings = new System.Collections.Generic.List<string>();
            foreach (var err in errors)
            {
                if (err != null && err.ErrorLevel == ErrorLevel.Error)
                {
                    errorStrings.Add(err.Message ?? "Parser error");
                }
            }

            if (errorStrings.Count > 0)
            {
                var errObj = new
                {
                    status = "error",
                    error = string.Join("; ", errorStrings),
                };
                Console.Error.WriteLine(JsonSerializer.Serialize(errObj));
                return;
            }

+            if (inMachineMode)
            {
-                AnsiConsole.MarkupLine(Markup.Escape(hinted.Hint));
+                var errObj = new
+                {
+                    status = "error",
+                    error = e.Message,
+                };
+                Console.Error.WriteLine(JsonSerializer.Serialize(errObj));
            }


KhushviB requested review from a team and Copilot July 11, 2026 18:15

Copilot started reviewing on behalf of KhushviB July 11, 2026 18:16 View session

Copilot AI reviewed Jul 11, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings July 11, 2026 18:32

Copilot started reviewing on behalf of KhushviB July 11, 2026 18:33 View session

Copilot AI reviewed Jul 11, 2026

View reviewed changes

fix/copilot-fix-suggestion

5cca375

Copilot AI review requested due to automatic review settings July 13, 2026 14:09

Merge branch 'main' into feature/a1-deterministic-output

c8cd2c8

Copilot started reviewing on behalf of KhushviB July 13, 2026 14:13 View session

Copilot AI reviewed Jul 13, 2026

View reviewed changes

Potential fix for pull request finding

2c6a7c0

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings July 13, 2026 14:18

Copilot started reviewing on behalf of KhushviB July 13, 2026 14:22 View session

Copilot AI reviewed Jul 13, 2026

View reviewed changes

Apply suggestions from code review

96c7c30

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings July 13, 2026 14:34

Copilot started reviewing on behalf of KhushviB July 13, 2026 14:37 View session

Copilot AI reviewed Jul 13, 2026

View reviewed changes

Comment thread CosmosDBShell/Program.cs

Comment thread CosmosDBShell.Tests/Integration/ShellProcessTests.cs

Apply suggestions from code review

6afcabe

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings July 13, 2026 14:47

Copilot started reviewing on behalf of KhushviB July 13, 2026 14:51 View session

Copilot AI reviewed Jul 13, 2026

View reviewed changes

Comment thread CosmosDBShell/Program.cs

fix: emit mutually-exclusive options error to STDERR as JSON in machi…

9addb34

…ne mode

Copilot AI review requested due to automatic review settings July 13, 2026 15:32

Copilot started reviewing on behalf of KhushviB July 13, 2026 15:35 View session

Copilot AI reviewed Jul 14, 2026

View reviewed changes

Comment thread CosmosDBShell/Program.cs

Copilot AI review requested due to automatic review settings July 14, 2026 19:29

Copilot started reviewing on behalf of KhushviB July 14, 2026 19:29 View session

Copilot AI reviewed Jul 14, 2026

View reviewed changes

Comment thread CosmosDBShell/Program.cs Outdated

Copilot AI review requested due to automatic review settings July 14, 2026 19:44

Copilot started reviewing on behalf of KhushviB July 14, 2026 19:44 View session

Copilot AI reviewed Jul 14, 2026

View reviewed changes

Comment thread CosmosDBShell/Program.cs

Comment thread CosmosDBShell/Program.cs

Comment thread CosmosDBShell/Program.cs

Copilot AI review requested due to automatic review settings July 14, 2026 20:01

Copilot started reviewing on behalf of KhushviB July 14, 2026 20:02 View session

Copilot AI reviewed Jul 14, 2026

View reviewed changes

mkrueger enabled auto-merge July 17, 2026 10:24

mkrueger mentioned this pull request Jul 20, 2026

Add stable exit-code contract and CI/CD documentation #182

Open

Merge branch 'main' into feature/a1-deterministic-output

3c8693f

Copilot AI review requested due to automatic review settings July 23, 2026 16:59

Copilot started reviewing on behalf of KhushviB July 23, 2026 17:00 View session

Copilot AI reviewed Jul 23, 2026

View reviewed changes

Comment thread CosmosDBShell/Azure.Data.Cosmos.Shell.Core/ErrorCommandState.cs Outdated

Potential fix for pull request finding

7ddc3d9

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

auto-merge was automatically disabled July 23, 2026 17:11
Head branch was pushed to by a user without write access

Copilot AI review requested due to automatic review settings July 23, 2026 17:11

Copilot started reviewing on behalf of KhushviB July 23, 2026 17:11 View session

Copilot AI reviewed Jul 23, 2026

View reviewed changes

Comment thread CosmosDBShell.Tests/UtilTest/OutputFormatTests.cs

Potential fix for pull request finding

c8c4252

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings July 23, 2026 17:19

Copilot started reviewing on behalf of KhushviB July 23, 2026 17:20 View session

Copilot AI reviewed Jul 23, 2026

View reviewed changes

Uh oh!

Conversation

KhushviB commented Jul 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description (#155 )

Key Changes

Validation / Acceptance Criteria Met

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mkrueger commented Jul 13, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

KhushviB commented Jul 11, 2026 •

edited

Loading

mkrueger commented Jul 17, 2026 •

edited

Loading