Skip to content

Add REST failure tests for resolve reply flow - #120

Merged
leynos merged 15 commits into
mainfrom
codex/add-negative-tests-for-rest-api-failures
Sep 12, 2025
Merged

Add REST failure tests for resolve reply flow#120
leynos merged 15 commits into
mainfrom
codex/add-negative-tests-for-rest-api-failures

Conversation

@leynos

@leynos leynos commented Sep 9, 2025

Copy link
Copy Markdown
Owner

Summary

  • test resolve reply handling for 404 responses
  • test resolve reply handling for server errors

Testing

  • make fmt
  • make lint
  • make test

https://chatgpt.com/codex/tasks/task_e_68c064bdbb988322bb5cbcabfb63b237

Summary by Sourcery

Add REST failure tests for the resolve reply flow to verify CLI behavior on 404 and server error responses

Tests:

  • Test that a 404 from the REST replies endpoint triggers a GraphQL fallback and overall success
  • Test that a 500 from the REST replies endpoint causes the CLI resolve command to fail

@sourcery-ai

sourcery-ai Bot commented Sep 9, 2025

Copy link
Copy Markdown
Contributor

Reviewer's Guide

Adds feature-gated REST failure tests for the 'resolve' reply flow by mocking server responses (404 and 500) and asserting correct command behavior and call sequences.

File-Level Changes

Change Details Files
Implement REST failure tests for resolve reply flow
  • Add resolve_flows_reply_rest_not_found test that returns 404 on /replies, asserts success, and verifies logs of POST calls including GraphQL requests
  • Add resolve_flows_reply_rest_error test that returns 500 on /replies, asserts failure, and verifies only the initial POST call is made
tests/resolve.rs

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@coderabbitai

coderabbitai Bot commented Sep 9, 2025

Copy link
Copy Markdown

Note

Reviews paused

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Summary by CodeRabbit

  • New Features

    • None
  • Tests

    • Expanded coverage for resolve workflows under an opt-in unstable feature, validating success and failure paths and output behaviour.
    • Improved test reliability with stricter assertions on exit status and logs.
  • Chores

    • Added a feature-gated CI job to lint and run the unstable resolve test suite with warnings treated as errors.
    • Ensured required system dependencies are installed during CI.

Walkthrough

Add a crate-level feature gate and helper in tests/resolve.rs to exercise vk resolve reply flows against a MITM server; parameterise reply-flow tests to cover REST OK/404/403/500 outcomes and assert stdout/stderr/exit behaviour; add a CI job to lint and run the feature-gated tests with warnings-as-errors.

Changes

Cohort / File(s) Summary
Resolve tests & helper
tests/resolve.rs
Add crate-level feature gate #![cfg(feature = "unstable-rest-resolve")]. Import predicates::prelude::*. Add async fn run_reply_flow(rest_status: StatusCode, _should_succeed: bool) to spin up a MITM server, log requests, stub /repos/.../replies with rest_status, drive GraphQL responses, invoke vk resolve "...#discussion_r1" -m done, and return logs, stdout, stderr, and exit status. Replace previous single reply test with parameterised resolve_flows_reply(#[case] rest_status, #[case] should_succeed, #[case] expected) covering OK, NOT_FOUND, FORBIDDEN, INTERNAL_SERVER_ERROR and asserting exit status, stdout/stderr emptiness or error content, and request sequence. Retain existing resolve_flows test (now non-parameterised).
CI workflow
.github/workflows/coverage.yml
Add unstable-rest-resolve job: checkout repo, set up Rust, install OpenSSL, run clippy linting with --features unstable-rest-resolve and -D warnings, and run tests with --features unstable-rest-resolve and RUSTFLAGS="-D warnings".

Sequence Diagram(s)

sequenceDiagram
    autonumber
    participant T as Test
    participant VK as vk resolve
    participant REST as REST /repos/.../replies
    participant GQL as GraphQL /graphql

    rect rgba(230,240,255,0.45)
    note over T,VK: REST 404 path — fallback to GraphQL
    T->>VK: invoke resolve (discussion)
    VK->>REST: POST /repos/o/r/pulls/.../replies
    REST-->>VK: 404 Not Found
    VK->>GQL: POST /graphql (fetch thread id)
    GQL-->>VK: thread id
    VK->>GQL: POST /graphql (resolve payload)
    GQL-->>VK: resolution result
    VK-->>T: success
    end
Loading
sequenceDiagram
    autonumber
    participant T as Test
    participant VK as vk resolve
    participant REST as REST /repos/.../replies

    rect rgba(255,235,235,0.45)
    note over T,VK: REST 500 path — propagate error
    T->>VK: invoke resolve (discussion)
    VK->>REST: POST /repos/o/r/pulls/.../replies
    REST-->>VK: 500 Internal Server Error
    VK-->>T: fail (propagate error)
    end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20–30 minutes

Poem

A MITM hums where tests convene,
404s whisper, GraphQL leans,
500 thunders, run aborts,
Logs fall neat in ordered sorts,
CI wakes to guard the scenes.

Prompt for AI agent: Summarise changes, generate compact review sections, produce sequence diagrams, estimate review effort, and write a short celebratory poem.
✨ Finishing touches
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch codex/add-negative-tests-for-rest-api-failures

Comment @coderabbitai help to get the list of available commands and usage tips.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there - I've reviewed your changes - here's some feedback:

  • There’s a lot of duplicated setup between the 404 and error tests—consider extracting the MITM handler and vk_cmd invocation into a shared helper to reduce boilerplate and improve maintainability.
  • The tests currently only assert the API call sequence; you might also capture and validate the CLI output or error message to ensure users get clear feedback on resolve failures.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- There’s a lot of duplicated setup between the 404 and error tests—consider extracting the MITM handler and vk_cmd invocation into a shared helper to reduce boilerplate and improve maintainability.
- The tests currently only assert the API call sequence; you might also capture and validate the CLI output or error message to ensure users get clear feedback on resolve failures.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

codescene-delta-analysis[bot]

This comment was marked as outdated.

@leynos

leynos commented Sep 10, 2025

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Sep 10, 2025

Copy link
Copy Markdown
✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 53ba608 and c9c1cb8.

📒 Files selected for processing (1)
  • tests/resolve.rs (1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.rs

⚙️ CodeRabbit configuration file

**/*.rs: * Seek to keep the cyclomatic complexity of functions no more than 12.

  • Adhere to single responsibility and CQRS

  • Place function attributes after doc comments.

  • Do not use return in single-line functions.

  • Move conditionals with >2 branches into a predicate function.

  • Avoid unsafe unless absolutely necessary.

  • Every module must begin with a //! doc comment that explains the module's purpose and utility.

  • Comments and docs must follow en-GB-oxendict (-ize / -our) spelling and grammar

  • Lints must not be silenced except as a last resort.

    • #[allow] is forbidden.
    • Only narrowly scoped #[expect(lint, reason = "...")] is allowed.
    • No lint groups, no blanket or file-wide suppression.
    • Include FIXME: with link if a fix is expected.
  • Where code is only used by specific features, it must be conditionally compiled or a conditional expectation for unused_code applied.

  • Use rstest fixtures for shared setup and to avoid repetition between tests.

  • Replace duplicated tests with #[rstest(...)] parameterised cases.

  • Prefer mockall for mocks/stubs.

  • Prefer .expect() over .unwrap()

  • Ensure that any API or behavioural changes are reflected in the documentation in docs/

  • Ensure that any completed roadmap steps are recorded in the appropriate roadmap in docs/

  • Files must not exceed 400 lines in length

    • Large modules must be decomposed
    • Long match statements or dispatch tables should be decomposed by domain and collocated with targets
    • Large blocks of inline data (e.g., test fixtures, constants or templates) must be moved to external files and inlined at compile-time or loaded at run-time.
  • Environment access (env::set_var and env::remove_var) are always unsafe in Rust 2024 and MUST be marked as such

    • For testing of functionality depending upon environment variables, dependency injection and the mockable crate are the preferred option.
    • If mockable cannot be used, env mutations in tests ...

Files:

  • tests/resolve.rs

Comment thread tests/resolve.rs Outdated
Comment thread tests/resolve.rs Outdated
codescene-delta-analysis[bot]

This comment was marked as outdated.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (2)
tests/resolve.rs (2)

131-145: Assert stderr on failure to validate UX, not just exit code.

Augment the 500-path to check an actionable error message so regressions surface early.

Apply this diff:

 tokio::task::spawn_blocking(move || {
     let assert = vk_cmd(addr)
         .args([
             "resolve",
             "https://github.com/o/r/pull/83#discussion_r1",
             "-m",
             "done",
         ])
         .assert();
     if expect_success {
         assert.success();
     } else {
-        assert.failure();
+        use predicates::prelude::*;
+        assert
+            .failure()
+            .stderr(
+                predicate::str::contains("replies")
+                    .and(predicate::str::contains(rest_status.as_u16().to_string())),
+            );
     }
 })

Add this import at the top of the file (outside the selected range):

use predicates::prelude::*;

152-171: Parametrise the 404/500 tests to remove duplication.

Collapse the two near-identical tests into one #[rstest] with cases for 404 and 500.

Apply this diff:

-#[cfg(feature = "unstable-rest-resolve")]
-#[tokio::test]
-async fn resolve_flows_reply_rest_not_found() {
-    let calls = run_reply_flow(StatusCode::NOT_FOUND, true).await;
-    assert_eq!(
-        calls,
-        [
-            "POST /repos/o/r/pulls/83/comments/1/replies",
-            "POST /graphql",
-            "POST /graphql",
-        ],
-    );
-}
-
-#[cfg(feature = "unstable-rest-resolve")]
-#[tokio::test]
-async fn resolve_flows_reply_rest_error() {
-    let calls = run_reply_flow(StatusCode::INTERNAL_SERVER_ERROR, false).await;
-    assert_eq!(calls, ["POST /repos/o/r/pulls/83/comments/1/replies"],);
-}
+#[cfg(feature = "unstable-rest-resolve")]
+#[rstest::rstest]
+#[case(
+    StatusCode::NOT_FOUND,
+    true,
+    &[
+        "POST /repos/o/r/pulls/83/comments/1/replies",
+        "POST /graphql",
+        "POST /graphql",
+    ]
+)]
+#[case(
+    StatusCode::INTERNAL_SERVER_ERROR,
+    false,
+    &[
+        "POST /repos/o/r/pulls/83/comments/1/replies",
+    ]
+)]
+#[tokio::test]
+async fn resolve_flows_reply_rest(
+    #[case] rest_status: StatusCode,
+    #[case] should_succeed: bool,
+    #[case] expected: &'static [&'static str],
+) {
+    let calls = run_reply_flow(rest_status, should_succeed).await;
+    assert_eq!(calls.as_slice(), expected);
+}
📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between c9c1cb8 and d759f78.

📒 Files selected for processing (1)
  • tests/resolve.rs (1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.rs

📄 CodeRabbit inference engine (AGENTS.md)

**/*.rs: Every module must begin with a module level (//!) comment explaining the module's purpose and utility.
Document public APIs using Rustdoc comments (///) so documentation can be generated with cargo doc.
Place function attributes after doc comments.
Do not use return in single-line functions.
Use predicate functions for conditional criteria with more than two branches.
Prefer immutable data and avoid unnecessary mut bindings.
Handle errors with the Result type instead of panicking where feasible.
Prefer semantic error enums: Derive std::error::Error (via the thiserror crate) for any condition the caller might inspect, retry, or map to an HTTP status.
Use an opaque error only at the app boundary: Use eyre::Report for human-readable logs; these should not be exposed in public APIs.
Never export the opaque type from a library: Convert to domain enums at API boundaries, and to eyre only in the main main() entrypoint or top-level async task.
Clippy warnings MUST be disallowed.
Fix any warnings emitted during tests in the code itself rather than silencing them.
Where a function is too long, extract meaningfully named helper functions adhering to separation of concerns and CQRS.
Where a function has too many parameters, group related parameters in meaningfully named structs.
Where a function is returning a large error consider using Arc to reduce the amount of data returned.
Write unit and behavioural tests for new functionality. Run both before and after making any change.
Avoid unsafe code unless absolutely necessary and document any usage clearly.
Lints must not be silenced except as a last resort.
Lint rule suppressions must be tightly scoped and include a clear reason.
Prefer expect over allow.
Prefer .expect() over .unwrap().
Use concat!() to combine long string literals rather than escaping newlines with a backslash.

Files:

  • tests/resolve.rs

⚙️ CodeRabbit configuration file

**/*.rs: * Seek to keep the cyclomatic complexity of functions no more than 12.

  • Adhere to single responsibility and CQRS

  • Place function attributes after doc comments.

  • Do not use return in single-line functions.

  • Move conditionals with >2 branches into a predicate function.

  • Avoid unsafe unless absolutely necessary.

  • Every module must begin with a //! doc comment that explains the module's purpose and utility.

  • Comments and docs must follow en-GB-oxendict (-ize / -our) spelling and grammar

  • Lints must not be silenced except as a last resort.

    • #[allow] is forbidden.
    • Only narrowly scoped #[expect(lint, reason = "...")] is allowed.
    • No lint groups, no blanket or file-wide suppression.
    • Include FIXME: with link if a fix is expected.
  • Where code is only used by specific features, it must be conditionally compiled or a conditional expectation for unused_code applied.

  • Use rstest fixtures for shared setup and to avoid repetition between tests.

  • Replace duplicated tests with #[rstest(...)] parameterised cases.

  • Prefer mockall for mocks/stubs.

  • Prefer .expect() over .unwrap()

  • Ensure that any API or behavioural changes are reflected in the documentation in docs/

  • Ensure that any completed roadmap steps are recorded in the appropriate roadmap in docs/

  • Files must not exceed 400 lines in length

    • Large modules must be decomposed
    • Long match statements or dispatch tables should be decomposed by domain and collocated with targets
    • Large blocks of inline data (e.g., test fixtures, constants or templates) must be moved to external files and inlined at compile-time or loaded at run-time.
  • Environment access (env::set_var and env::remove_var) are always unsafe in Rust 2024 and MUST be marked as such

    • For testing of functionality depending upon environment variables, dependency injection and the mockable crate are the preferred option.
    • If mockable cannot be used, env mutations in tests ...

Files:

  • tests/resolve.rs
🧬 Code graph analysis (1)
tests/resolve.rs (1)
tests/utils/mod.rs (3)
  • shutdown (30-33)
  • start_mitm (49-98)
  • vk_cmd (109-117)
🔍 Remote MCP

Here are a few extra points that I didn’t see in the AI-generated summary but which will be important when reviewing this PR:

  1. Tests use start_mitm() directly, not a helper called run_reply_flow
    The new tests spin up a mock HTTP server via start_mitm().await, then install a closure into its handler to inspect each incoming request and return either the REST status or one of two GraphQL JSON payloads. There is no run_reply_flow helper in the diff—each test inlines the MITM setup and response logic. (github.com)

  2. The two originally separate tests have been collapsed into one parametrised #[rstest]-driven test
    Rather than two nearly identical async fn resolve_flows_reply_rest_not_found() and async fn resolve_flows_reply_rest_error(), the PR replaces them with a single async fn resolve_flows_reply_rest(#[case] rest_status: StatusCode, #[case] should_succeed: bool, #[case] expected: &'static [&'static str]) test, annotated with two #[case(...)] attributes for the 404 and 500 scenarios. (github.com)

  3. Both tests remain behind the same feature gate
    Each #[rstest] case is still guarded by #[cfg(feature = "unstable-rest-resolve")], so you’ll need to confirm that Cargo.toml actually defines the "unstable-rest-resolve" feature; otherwise these tests won’t compile or run. (github.com)

⏰ Context from checks skipped due to timeout of 120000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: build-test
🔇 Additional comments (1)
tests/resolve.rs (1)

101-150: Good extraction of shared flow into a helper.

Encapsulate the MITM wiring and CLI invocation once; this reduces duplication and makes the scenario-driven tests clearer.

Comment thread tests/resolve.rs Outdated
codescene-delta-analysis[bot]

This comment was marked as outdated.

codescene-delta-analysis[bot]

This comment was marked as outdated.

@leynos

leynos commented Sep 11, 2025

Copy link
Copy Markdown
Owner Author

@coderabbitai Have the following now been resolved?

  • There’s a lot of duplicated setup between the 404 and error tests—consider extracting the MITM handler and vk_cmd invocation into a shared helper to reduce boilerplate and improve maintainability.
  • The tests currently only assert the API call sequence; you might also capture and validate the CLI output or error message to ensure users get clear feedback on resolve failures.

@coderabbitai

coderabbitai Bot commented Sep 11, 2025

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

@leynos Looking at the current implementation in tests/resolve.rs, here's the status of your two concerns:

Resolved: Duplicated setup between tests

The code duplication issue has been resolved. The implementation now includes:

  1. Shared helper function: run_reply_flow(rest_status: StatusCode, expect_success: bool) that centralizes:

    • MITM server setup and handler configuration
    • Request logging logic
    • GraphQL response mocking
    • The vk_cmd invocation with standard parameters
    • Success/failure assertion logic
  2. Clean test functions: Both resolve_flows_reply_rest_not_found() and resolve_flows_reply_rest_error() now simply call the helper with different parameters:

    // 404 test
    let requests = run_reply_flow(StatusCode::NOT_FOUND, true).await;
    
    // 500 test  
    let requests = run_reply_flow(StatusCode::INTERNAL_SERVER_ERROR, false).await;

This significantly reduces boilerplate and improves maintainability.

Not resolved: CLI output validation

The second concern has not been addressed. The tests still only:

  • Assert whether vk_cmd succeeds or fails via the expect_success parameter
  • Validate the API call sequence through request logging

The tests do not capture or validate:

  • CLI output messages
  • Error message content
  • User feedback quality

The vk_cmd calls still use .unwrap() or basic success/failure checks without examining the actual output that users would see.

Summary: The code duplication has been excellently resolved with a clean helper function, but CLI output validation remains unimplemented.


🏁 Script executed:

#!/bin/bash
# Let's examine the actual test file to see the current implementation
cat tests/resolve.rs

Length of output: 6506


@leynos I need to correct my previous assessment. After examining the actual implementation, both concerns have been resolved:

Resolved: Duplicated setup between tests

The duplication has been eliminated using a different but effective approach:

  1. Consolidated into a single parameterized test: The two separate test functions have been merged into one resolve_flows_reply_rest test using #[rstest] with #[case] annotations
  2. Shared helper function: run_reply_flow(rest_status, expect_success) centralizes all the setup logic including MITM server, request handling, and command execution
  3. Clean parameterization: Each test case specifies the REST status, expected success/failure, and expected request sequence

Resolved: CLI output validation

The CLI output validation has been fully implemented:

  1. Output capture: The run_reply_flow helper now returns (calls, stdout, stderr) - capturing both stdout and stderr from the command execution
  2. Comprehensive validation:
    • stdout: Asserts it should be empty in all cases: assert!(stdout.trim().is_empty())
    • stderr for success: Validates stderr is empty for successful operations
    • stderr for errors: Uses predicates to validate error messages contain both "replies" and "500":
      predicate::str::contains("replies")
          .and(predicate::str::contains("500"))
          .eval(&stderr)

The implementation now provides clear user feedback validation, ensuring that users receive appropriate error messages when REST endpoints fail.

Summary: Both concerns have been excellently addressed. The code is now well-structured with shared helpers and includes comprehensive CLI output validation for user experience quality.

codescene-delta-analysis[bot]

This comment was marked as outdated.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
tests/resolve.rs (2)

45-49: Assert clean output on the happy path.

Also validate UX by asserting empty stdout/stderr on success, matching the style used below.

Apply this diff:

   tokio::task::spawn_blocking(move || {
-        vk_cmd(addr).args(args).assert().success();
+        vk_cmd(addr)
+            .args(args)
+            .assert()
+            .success()
+            .stdout(predicate::str::is_empty())
+            .stderr(predicate::str::is_empty());
   })

60-104: DRY the reply success test via the shared helper.

Reuse run_reply_flow(StatusCode::OK, true) to eliminate duplicate MITM/server code and align assertions.

Apply this diff:

-async fn resolve_flows_reply() {
-    let (addr, handler, shutdown) = start_mitm().await.expect("start server");
-    let calls = Arc::new(Mutex::new(Vec::<String>::new()));
-    let clone = Arc::clone(&calls);
-    *handler.lock().expect("lock handler") = Box::new(move |req| {
-        let mut vec = clone.lock().expect("lock");
-        let gql_calls = vec.iter().filter(|c| c.ends_with("/graphql")).count();
-        vec.push(format!("{} {}", req.method(), req.uri().path()));
-        let body = if req.uri().path() == "/graphql" {
-            if gql_calls == 0 {
-                r#"{"data":{"node":{"pullRequestReviewThread":{"id":"t"}}}}"#
-            } else {
-                r#"{"data":{"resolveReviewThread":{"clientMutationId":null}}}"#
-            }
-        } else {
-            "{}"
-        };
-        Response::builder()
-            .status(StatusCode::OK)
-            .header("Content-Type", "application/json")
-            .body(Full::from(body))
-            .expect("response")
-    });
-    tokio::task::spawn_blocking(move || {
-        vk_cmd(addr)
-            .args([
-                "resolve",
-                "https://github.com/o/r/pull/83#discussion_r1",
-                "-m",
-                "done",
-            ])
-            .assert()
-            .success();
-    })
-    .await
-    .expect("spawn blocking");
-    shutdown.shutdown().await;
-    assert_eq!(
-        calls.lock().expect("lock").as_slice(),
-        [
-            "POST /repos/o/r/pulls/83/comments/1/replies",
-            "POST /graphql",
-            "POST /graphql",
-        ],
-    );
-}
+async fn resolve_flows_reply() {
+    let (calls, stdout, stderr) = run_reply_flow(StatusCode::OK, true).await;
+    assert!(String::from_utf8_lossy(&stdout).trim().is_empty(), "unexpected stdout: {}", String::from_utf8_lossy(&stdout));
+    assert!(String::from_utf8_lossy(&stderr).trim().is_empty(), "unexpected stderr: {}", String::from_utf8_lossy(&stderr));
+    assert_eq!(
+        calls.as_slice(),
+        [
+            "POST /repos/o/r/pulls/83/comments/1/replies",
+            "POST /graphql",
+            "POST /graphql",
+        ],
+    );
+}
📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d759f78 and 98a4f8a.

📒 Files selected for processing (2)
  • .github/workflows/coverage.yml (1 hunks)
  • tests/resolve.rs (2 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.rs

📄 CodeRabbit inference engine (AGENTS.md)

**/*.rs: Every module must begin with a module level (//!) comment explaining the module's purpose and utility.
Document public APIs using Rustdoc comments (///) so documentation can be generated with cargo doc.
Place function attributes after doc comments.
Do not use return in single-line functions.
Use predicate functions for conditional criteria with more than two branches.
Prefer immutable data and avoid unnecessary mut bindings.
Handle errors with the Result type instead of panicking where feasible.
Prefer semantic error enums: Derive std::error::Error (via the thiserror crate) for any condition the caller might inspect, retry, or map to an HTTP status.
Use an opaque error only at the app boundary: Use eyre::Report for human-readable logs; these should not be exposed in public APIs.
Never export the opaque type from a library: Convert to domain enums at API boundaries, and to eyre only in the main main() entrypoint or top-level async task.
Clippy warnings MUST be disallowed.
Fix any warnings emitted during tests in the code itself rather than silencing them.
Where a function is too long, extract meaningfully named helper functions adhering to separation of concerns and CQRS.
Where a function has too many parameters, group related parameters in meaningfully named structs.
Where a function is returning a large error consider using Arc to reduce the amount of data returned.
Write unit and behavioural tests for new functionality. Run both before and after making any change.
Avoid unsafe code unless absolutely necessary and document any usage clearly.
Lints must not be silenced except as a last resort.
Lint rule suppressions must be tightly scoped and include a clear reason.
Prefer expect over allow.
Prefer .expect() over .unwrap().
Use concat!() to combine long string literals rather than escaping newlines with a backslash.

Files:

  • tests/resolve.rs

⚙️ CodeRabbit configuration file

**/*.rs: * Seek to keep the cyclomatic complexity of functions no more than 12.

  • Adhere to single responsibility and CQRS

  • Place function attributes after doc comments.

  • Do not use return in single-line functions.

  • Move conditionals with >2 branches into a predicate function.

  • Avoid unsafe unless absolutely necessary.

  • Every module must begin with a //! doc comment that explains the module's purpose and utility.

  • Comments and docs must follow en-GB-oxendict (-ize / -our) spelling and grammar

  • Lints must not be silenced except as a last resort.

    • #[allow] is forbidden.
    • Only narrowly scoped #[expect(lint, reason = "...")] is allowed.
    • No lint groups, no blanket or file-wide suppression.
    • Include FIXME: with link if a fix is expected.
  • Where code is only used by specific features, it must be conditionally compiled or a conditional expectation for unused_code applied.

  • Use rstest fixtures for shared setup and to avoid repetition between tests.

  • Replace duplicated tests with #[rstest(...)] parameterised cases.

  • Prefer mockall for mocks/stubs.

  • Prefer .expect() over .unwrap()

  • Ensure that any API or behavioural changes are reflected in the documentation in docs/

  • Ensure that any completed roadmap steps are recorded in the appropriate roadmap in docs/

  • Files must not exceed 400 lines in length

    • Large modules must be decomposed
    • Long match statements or dispatch tables should be decomposed by domain and collocated with targets
    • Large blocks of inline data (e.g., test fixtures, constants or templates) must be moved to external files and inlined at compile-time or loaded at run-time.
  • Environment access (env::set_var and env::remove_var) are always unsafe in Rust 2024 and MUST be marked as such

    • For testing of functionality depending upon environment variables, dependency injection and the mockable crate are the preferred option.
    • If mockable cannot be used, env mutations in tests ...

Files:

  • tests/resolve.rs
🧬 Code graph analysis (1)
tests/resolve.rs (1)
tests/utils/mod.rs (3)
  • start_mitm (49-98)
  • vk_cmd (109-117)
  • shutdown (30-33)
🔍 Remote MCP ## Issue summary for EXAMPLE-123

The issue is about implementing a new feature in the product that allows users to customize their dashboard. The issue links to a Notion page with detailed product requirements.

Product requirements for EXAMPLE-123

The product requirements for EXAMPLE-123 include:

  • The dashboard must be customizable by the user.
  • Users should be able to add, remove, and rearrange widgets on the dashboard.
  • The feature should be intuitive and easy to use.,
🔇 Additional comments (1)
tests/resolve.rs (1)

183-201: LGTM: Strong negative-path assertions and call-sequence checks.

The parametrised cases correctly validate fallback on 404 vs failure on 500, including actionable stderr content.

Comment thread .github/workflows/coverage.yml Outdated
Comment thread tests/resolve.rs Outdated
Comment thread tests/resolve.rs Outdated
codescene-delta-analysis[bot]

This comment was marked as outdated.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
tests/resolve.rs (2)

14-54: Drop rstest on the single-case test and simplify args.

This test has one case; remove the parameter boilerplate and inline args.

-#[tokio::test]
-#[rstest::rstest]
-#[case(None)]
-async fn resolve_flows(#[case] msg: Option<&'static str>) {
+#[tokio::test]
+async fn resolve_flows() {
@@
-    let mut args = vec!["resolve", "https://github.com/o/r/pull/83#discussion_r1"];
-    if let Some(m) = msg {
-        args.extend(["-m", m]);
-    }
+    let args = vec!["resolve", "https://github.com/o/r/pull/83#discussion_r1"];

56-102: Fold this success-path test into the parameterised suite.

Unify success and failure paths under one #[rstest] with a 200-OK case; delete this duplicate.

-#[tokio::test]
-async fn resolve_flows_reply() {
-    let (addr, handler, shutdown) = start_mitm().await expect("start server");
-    let calls = Arc::new(Mutex::new(Vec::<String>::new()));
-    let clone = Arc::clone(&calls);
-    *handler.lock().expect("lock handler") = Box::new(move |req| {
-        let mut vec = clone.lock().expect("lock");
-        let gql_calls = vec.iter().filter(|c| c.ends_with("/graphql")).count();
-        vec.push(format!("{} {}", req.method(), req.uri().path()));
-        let body = if req.uri().path() == "/graphql" {
-            if gql_calls == 0 {
-                r#"{"data":{"node":{"pullRequestReviewThread":{"id":"t"}}}}"#
-            } else {
-                r#"{"data":{"resolveReviewThread":{"clientMutationId":null}}}"#
-            }
-        } else {
-            "{}"
-        };
-        Response::builder()
-            .status(StatusCode::OK)
-            .header("Content-Type", "application/json")
-            .body(Full::from(body))
-            .expect("response")
-    });
-    tokio::task::spawn_blocking(move || {
-        vk_cmd(addr)
-            .args([
-                "resolve",
-                "https://github.com/o/r/pull/83#discussion_r1",
-                "-m",
-                "done",
-            ])
-            .assert()
-            .success();
-    })
-    .await
-    .expect("spawn blocking");
-    shutdown.shutdown().await;
-    assert_eq!(
-        calls.lock().expect("lock").as_slice(),
-        [
-            "POST /repos/o/r/pulls/83/comments/1/replies",
-            "POST /graphql",
-            "POST /graphql",
-        ],
-    );
-}
♻️ Duplicate comments (2)
.github/workflows/coverage.yml (1)

35-42: Wire-up of feature-gated tests is correct — good work.

This job ensures the unstable-rest-resolve tests actually execute in CI.

tests/resolve.rs (1)

3-4: Crate-level cfg gate reduces noise — keep.

This replaces per-item gates and keeps the file cohesive.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 98a4f8a and 5329f88.

📒 Files selected for processing (2)
  • .github/workflows/coverage.yml (1 hunks)
  • tests/resolve.rs (2 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.rs

📄 CodeRabbit inference engine (AGENTS.md)

**/*.rs: Every module must begin with a module level (//!) comment explaining the module's purpose and utility.
Document public APIs using Rustdoc comments (///) so documentation can be generated with cargo doc.
Place function attributes after doc comments.
Do not use return in single-line functions.
Use predicate functions for conditional criteria with more than two branches.
Prefer immutable data and avoid unnecessary mut bindings.
Handle errors with the Result type instead of panicking where feasible.
Prefer semantic error enums: Derive std::error::Error (via the thiserror crate) for any condition the caller might inspect, retry, or map to an HTTP status.
Use an opaque error only at the app boundary: Use eyre::Report for human-readable logs; these should not be exposed in public APIs.
Never export the opaque type from a library: Convert to domain enums at API boundaries, and to eyre only in the main main() entrypoint or top-level async task.
Clippy warnings MUST be disallowed.
Fix any warnings emitted during tests in the code itself rather than silencing them.
Where a function is too long, extract meaningfully named helper functions adhering to separation of concerns and CQRS.
Where a function has too many parameters, group related parameters in meaningfully named structs.
Where a function is returning a large error consider using Arc to reduce the amount of data returned.
Write unit and behavioural tests for new functionality. Run both before and after making any change.
Avoid unsafe code unless absolutely necessary and document any usage clearly.
Lints must not be silenced except as a last resort.
Lint rule suppressions must be tightly scoped and include a clear reason.
Prefer expect over allow.
Prefer .expect() over .unwrap().
Use concat!() to combine long string literals rather than escaping newlines with a backslash.

Files:

  • tests/resolve.rs

⚙️ CodeRabbit configuration file

**/*.rs: * Seek to keep the cyclomatic complexity of functions no more than 12.

  • Adhere to single responsibility and CQRS

  • Place function attributes after doc comments.

  • Do not use return in single-line functions.

  • Move conditionals with >2 branches into a predicate function.

  • Avoid unsafe unless absolutely necessary.

  • Every module must begin with a //! doc comment that explains the module's purpose and utility.

  • Comments and docs must follow en-GB-oxendict (-ize / -our) spelling and grammar

  • Lints must not be silenced except as a last resort.

    • #[allow] is forbidden.
    • Only narrowly scoped #[expect(lint, reason = "...")] is allowed.
    • No lint groups, no blanket or file-wide suppression.
    • Include FIXME: with link if a fix is expected.
  • Where code is only used by specific features, it must be conditionally compiled or a conditional expectation for unused_code applied.

  • Use rstest fixtures for shared setup and to avoid repetition between tests.

  • Replace duplicated tests with #[rstest(...)] parameterised cases.

  • Prefer mockall for mocks/stubs.

  • Prefer .expect() over .unwrap()

  • Ensure that any API or behavioural changes are reflected in the documentation in docs/

  • Ensure that any completed roadmap steps are recorded in the appropriate roadmap in docs/

  • Files must not exceed 400 lines in length

    • Large modules must be decomposed
    • Long match statements or dispatch tables should be decomposed by domain and collocated with targets
    • Large blocks of inline data (e.g., test fixtures, constants or templates) must be moved to external files and inlined at compile-time or loaded at run-time.
  • Environment access (env::set_var and env::remove_var) are always unsafe in Rust 2024 and MUST be marked as such

    • For testing of functionality depending upon environment variables, dependency injection and the mockable crate are the preferred option.
    • If mockable cannot be used, env mutations in tests ...

Files:

  • tests/resolve.rs
🧬 Code graph analysis (1)
tests/resolve.rs (1)
tests/utils/mod.rs (3)
  • start_mitm (49-98)
  • vk_cmd (109-117)
  • shutdown (30-33)
🔇 Additional comments (3)
tests/resolve.rs (3)

8-8: Predicates import for stderr/stdout assertions — OK.


104-150: Side-effect-free helper returning stdout/stderr/status — good.

Capturing exit status and leaving assertions to callers improves diagnostics.


152-193: Parametrised REST-error coverage with CLI stderr checks — good.

This validates call sequence, exit status, and user-facing errors.

Comment thread .github/workflows/coverage.yml Outdated
Comment thread tests/resolve.rs Outdated
codescene-delta-analysis[bot]

This comment was marked as outdated.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
tests/resolve.rs (1)

14-54: Assert stdout/stderr on the happy path to align with reply-flow checks.

Capture and verify the CLI emits no output on success.

-    tokio::task::spawn_blocking(move || {
-        vk_cmd(addr).args(args).assert().success();
-    })
-    .await
-    .expect("spawn blocking");
+    let (stdout, stderr, status) = tokio::task::spawn_blocking(move || {
+        let output = vk_cmd(addr).args(args).output().expect("run command");
+        (output.stdout, output.stderr, output.status)
+    })
+    .await
+    .expect("spawn blocking");
+    assert!(status.success());
+    let stdout = String::from_utf8_lossy(&stdout);
+    let stderr = String::from_utf8_lossy(&stderr);
+    assert!(predicate::str::is_empty().eval(&stdout), "unexpected stdout: {stdout}");
+    assert!(predicate::str::is_empty().eval(&stderr), "unexpected stderr: {stderr}");
📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 5329f88 and 2c07ec6.

📒 Files selected for processing (2)
  • .github/workflows/coverage.yml (1 hunks)
  • tests/resolve.rs (3 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.rs

📄 CodeRabbit inference engine (AGENTS.md)

**/*.rs: Every module must begin with a module level (//!) comment explaining the module's purpose and utility.
Document public APIs using Rustdoc comments (///) so documentation can be generated with cargo doc.
Place function attributes after doc comments.
Do not use return in single-line functions.
Use predicate functions for conditional criteria with more than two branches.
Prefer immutable data and avoid unnecessary mut bindings.
Handle errors with the Result type instead of panicking where feasible.
Prefer semantic error enums: Derive std::error::Error (via the thiserror crate) for any condition the caller might inspect, retry, or map to an HTTP status.
Use an opaque error only at the app boundary: Use eyre::Report for human-readable logs; these should not be exposed in public APIs.
Never export the opaque type from a library: Convert to domain enums at API boundaries, and to eyre only in the main main() entrypoint or top-level async task.
Clippy warnings MUST be disallowed.
Fix any warnings emitted during tests in the code itself rather than silencing them.
Where a function is too long, extract meaningfully named helper functions adhering to separation of concerns and CQRS.
Where a function has too many parameters, group related parameters in meaningfully named structs.
Where a function is returning a large error consider using Arc to reduce the amount of data returned.
Write unit and behavioural tests for new functionality. Run both before and after making any change.
Avoid unsafe code unless absolutely necessary and document any usage clearly.
Lints must not be silenced except as a last resort.
Lint rule suppressions must be tightly scoped and include a clear reason.
Prefer expect over allow.
Prefer .expect() over .unwrap().
Use concat!() to combine long string literals rather than escaping newlines with a backslash.

Files:

  • tests/resolve.rs

⚙️ CodeRabbit configuration file

**/*.rs: * Seek to keep the cyclomatic complexity of functions no more than 12.

  • Adhere to single responsibility and CQRS

  • Place function attributes after doc comments.

  • Do not use return in single-line functions.

  • Move conditionals with >2 branches into a predicate function.

  • Avoid unsafe unless absolutely necessary.

  • Every module must begin with a //! doc comment that explains the module's purpose and utility.

  • Comments and docs must follow en-GB-oxendict (-ize / -our) spelling and grammar

  • Lints must not be silenced except as a last resort.

    • #[allow] is forbidden.
    • Only narrowly scoped #[expect(lint, reason = "...")] is allowed.
    • No lint groups, no blanket or file-wide suppression.
    • Include FIXME: with link if a fix is expected.
  • Where code is only used by specific features, it must be conditionally compiled or a conditional expectation for unused_code applied.

  • Use rstest fixtures for shared setup and to avoid repetition between tests.

  • Replace duplicated tests with #[rstest(...)] parameterised cases.

  • Prefer mockall for mocks/stubs.

  • Prefer .expect() over .unwrap()

  • Ensure that any API or behavioural changes are reflected in the documentation in docs/

  • Ensure that any completed roadmap steps are recorded in the appropriate roadmap in docs/

  • Files must not exceed 400 lines in length

    • Large modules must be decomposed
    • Long match statements or dispatch tables should be decomposed by domain and collocated with targets
    • Large blocks of inline data (e.g., test fixtures, constants or templates) must be moved to external files and inlined at compile-time or loaded at run-time.
  • Environment access (env::set_var and env::remove_var) are always unsafe in Rust 2024 and MUST be marked as such

    • For testing of functionality depending upon environment variables, dependency injection and the mockable crate are the preferred option.
    • If mockable cannot be used, env mutations in tests ...

Files:

  • tests/resolve.rs
🧬 Code graph analysis (1)
tests/resolve.rs (1)
tests/utils/mod.rs (3)
  • start_mitm (49-98)
  • vk_cmd (109-117)
  • shutdown (30-33)
🔍 Remote MCP Deepwiki

Summary — additional repo facts relevant to this PR (concise)

  • Test infra: repo E2E tests use third-wheel / MITM proxy + recorded JSON fixtures and run the compiled binary via assert_cmd; tests are async (tokio) and use rstest / predicates for assertions. This is the pattern the new resolve-reply tests follow.

  • E2E specifics: tests start a local MITM and generate a temporary CA (OpenSSL) for HTTPS interception; tests override GITHUB_GRAPHQL_URL and set a dummy GITHUB_TOKEN. CI/runner must support openssl/CA generation for these tests.

  • Strict CI/test flags: the Makefile test target and CI treat warnings as errors (RUSTFLAGS="-D warnings"); clippy is run with -D warnings. Any new tests (and new feature-gated code) must compile cleanly under those flags.

  • Coverage/CI: the repository has a coverage workflow (.github/workflows/coverage.yml) that runs cargo-llvm-cov on PRs and uploads to Codecov. If the PR added a new coverage job (unstable-rest-resolve), ensure it uses the same runner/environment and that its commands/flags match project expectations.

  • Cargo features: the documented Cargo.toml features in the repository (default = ["toml"], plus config-format features) do not show an "unstable-rest-resolve" feature in the docs I read — verify the PR also declares that feature in Cargo.toml (otherwise CI invocations using --features unstable-rest-resolve will fail).

  • Review checklist items implied by the repo context:

    • Confirm Cargo.toml adds the unstable-rest-resolve feature.
    • Confirm CI job(s) added to run tests/clippy with --features unstable-rest-resolve use ubuntu-latest and have OpenSSL available.
    • Verify the new tests are isolated/serialized if they share the MITM/CA (avoid flaky concurrent runs).
    • Ensure tests compile/run with RUSTFLAGS="-D warnings" and clippy -D warnings.

Sources/tools used:,

🔇 Additional comments (3)
tests/resolve.rs (3)

3-4: Gate the suite by feature. LGTM.

This keeps the new E2E behind an explicit flag and prevents accidental CI drift.


8-8: Ensure predicates is in dev-dependencies.

Import is correct. Verify Cargo.toml lists predicates to avoid CI failures.

#!/usr/bin/env bash
set -euo pipefail
fd -a Cargo.toml | xargs -I{} rg -n '^\[dev-dependencies\]|\bpredicates\s*=' {}

56-102: Helper design is clean and side-effect free.

Returning calls/stdout/stderr/status gives precise assertions at the call site. No issues.

Comment thread .github/workflows/coverage.yml Outdated
Comment thread tests/resolve.rs
codescene-delta-analysis[bot]

This comment was marked as outdated.

codescene-delta-analysis[bot]

This comment was marked as outdated.

codescene-delta-analysis[bot]

This comment was marked as outdated.

codescene-delta-analysis[bot]

This comment was marked as outdated.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
.github/workflows/coverage.yml (1)

35-47: Enable the feature for tests; otherwise these tests never run.

The test step invokes make without passing the unstable feature, so feature-gated tests are skipped. Enable the feature in the CARGO shim.

Apply:

   unstable-rest-resolve:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v5
       - name: Setup Rust
         uses: leynos/shared-actions/.github/actions/setup-rust@ebbaaa0d7a776ec3187f3940667e5c7c9379cdf5
       - name: Install OpenSSL
         run: sudo apt-get update && sudo apt-get install -y openssl ca-certificates
       - name: Lint (feature-gated)
         run: CARGO="cargo --locked" make lint CLIPPY_FLAGS="--all-targets --features unstable-rest-resolve -- -D warnings"
       - name: Test unstable REST resolve (feature-gated)
-        run: RUSTFLAGS="-D warnings" CARGO="cargo --locked" make test
+        run: RUSTFLAGS="-D warnings" CARGO="cargo --locked --features unstable-rest-resolve" make test
♻️ Duplicate comments (1)
tests/resolve.rs (1)

61-65: Remove assertions from the helper; return ExitStatus and assert at call site.

Keep helpers side-effect free to localise failures and improve diagnostics.

Apply:

-async fn run_reply_flow(
-    rest_status: StatusCode,
-    should_succeed: bool,
-) -> (Vec<String>, Vec<u8>, Vec<u8>) {
+async fn run_reply_flow(
+    rest_status: StatusCode,
+    should_succeed: bool,
+) -> (Vec<String>, Vec<u8>, Vec<u8>, std::process::ExitStatus) {
@@
-    let (stdout, stderr, status) = tokio::task::spawn_blocking(move || {
+    let (stdout, stderr, status) = tokio::task::spawn_blocking(move || {
         let output = vk_cmd(addr)
@@
-            .output()
-            .expect("run command");
-        (output.stdout, output.stderr, output.status)
+            .output()
+            .expect("run command");
+        (output.stdout, output.stderr, output.status)
     })
@@
-    if should_succeed {
-        assert!(status.success());
-    } else {
-        assert!(!status.success());
-    }
     shutdown.shutdown().await;
-    (calls.lock().expect("lock").clone(), stdout, stderr)
+    (calls.lock().expect("lock").clone(), stdout, stderr, status)
 }
@@
-async fn resolve_flows_reply(
+async fn resolve_flows_reply(
     #[case] rest_status: StatusCode,
     #[case] should_succeed: bool,
     #[case] expected: &'static [&'static str],
 ) {
-    let (calls, stdout, stderr) = run_reply_flow(rest_status, should_succeed).await;
+    let (calls, stdout, stderr, status) = run_reply_flow(rest_status, should_succeed).await;
     let stdout = String::from_utf8_lossy(&stdout);
     let stderr = String::from_utf8_lossy(&stderr);
     let code = rest_status.as_u16().to_string();
     assert!(stdout.trim().is_empty(), "unexpected stdout: {stdout}");
-    if should_succeed {
+    if should_succeed {
+        assert!(status.success(), "status: {status:?}, stderr: {stderr}");
         assert!(stderr.trim().is_empty(), "unexpected stderr: {stderr}");
-    } else {
+    } else {
+        assert!(!status.success(), "expected failure; got success");
         assert!(
             predicate::str::contains("replies")
                 .and(predicate::str::contains(code.as_str()))
                 .eval(&stderr),
             "stderr: {stderr}"
         );
     }
     assert_eq!(calls.as_slice(), expected);
 }

Also applies to: 92-113, 145-166

📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 58c750f and ba41dfc.

📒 Files selected for processing (2)
  • .github/workflows/coverage.yml (1 hunks)
  • tests/resolve.rs (4 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.rs

📄 CodeRabbit inference engine (AGENTS.md)

**/*.rs: Every module must begin with a module level (//!) comment explaining the module's purpose and utility.
Document public APIs using Rustdoc comments (///) so documentation can be generated with cargo doc.
Place function attributes after doc comments.
Do not use return in single-line functions.
Use predicate functions for conditional criteria with more than two branches.
Prefer immutable data and avoid unnecessary mut bindings.
Handle errors with the Result type instead of panicking where feasible.
Prefer semantic error enums: Derive std::error::Error (via the thiserror crate) for any condition the caller might inspect, retry, or map to an HTTP status.
Use an opaque error only at the app boundary: Use eyre::Report for human-readable logs; these should not be exposed in public APIs.
Never export the opaque type from a library: Convert to domain enums at API boundaries, and to eyre only in the main main() entrypoint or top-level async task.
Clippy warnings MUST be disallowed.
Fix any warnings emitted during tests in the code itself rather than silencing them.
Where a function is too long, extract meaningfully named helper functions adhering to separation of concerns and CQRS.
Where a function has too many parameters, group related parameters in meaningfully named structs.
Where a function is returning a large error consider using Arc to reduce the amount of data returned.
Write unit and behavioural tests for new functionality. Run both before and after making any change.
Avoid unsafe code unless absolutely necessary and document any usage clearly.
Lints must not be silenced except as a last resort.
Lint rule suppressions must be tightly scoped and include a clear reason.
Prefer expect over allow.
Prefer .expect() over .unwrap().
Use concat!() to combine long string literals rather than escaping newlines with a backslash.

Files:

  • tests/resolve.rs

⚙️ CodeRabbit configuration file

**/*.rs: * Seek to keep the cyclomatic complexity of functions no more than 12.

  • Adhere to single responsibility and CQRS

  • Place function attributes after doc comments.

  • Do not use return in single-line functions.

  • Move conditionals with >2 branches into a predicate function.

  • Avoid unsafe unless absolutely necessary.

  • Every module must begin with a //! doc comment that explains the module's purpose and utility.

  • Comments and docs must follow en-GB-oxendict (-ize / -our) spelling and grammar

  • Lints must not be silenced except as a last resort.

    • #[allow] is forbidden.
    • Only narrowly scoped #[expect(lint, reason = "...")] is allowed.
    • No lint groups, no blanket or file-wide suppression.
    • Include FIXME: with link if a fix is expected.
  • Where code is only used by specific features, it must be conditionally compiled or a conditional expectation for unused_code applied.

  • Use rstest fixtures for shared setup and to avoid repetition between tests.

  • Replace duplicated tests with #[rstest(...)] parameterised cases.

  • Prefer mockall for mocks/stubs.

  • Prefer .expect() over .unwrap()

  • Ensure that any API or behavioural changes are reflected in the documentation in docs/

  • Ensure that any completed roadmap steps are recorded in the appropriate roadmap in docs/

  • Files must not exceed 400 lines in length

    • Large modules must be decomposed
    • Long match statements or dispatch tables should be decomposed by domain and collocated with targets
    • Large blocks of inline data (e.g., test fixtures, constants or templates) must be moved to external files and inlined at compile-time or loaded at run-time.
  • Environment access (env::set_var and env::remove_var) are always unsafe in Rust 2024 and MUST be marked as such

    • For testing of functionality depending upon environment variables, dependency injection and the mockable crate are the preferred option.
    • If mockable cannot be used, env mutations in tests ...

Files:

  • tests/resolve.rs
🧬 Code graph analysis (1)
tests/resolve.rs (1)
tests/utils/mod.rs (3)
  • start_mitm (49-98)
  • vk_cmd (109-117)
  • shutdown (30-33)
🔍 Remote MCP Deepwiki

Relevant facts for reviewing PR #120

  • The repository’s Cargo features do NOT define unstable-rest-resolve (default features include only "toml"); running cargo with --features unstable-rest-resolve will fail unless Cargo.toml is updated.

  • New tests spin up a MITM proxy (third-wheel) and generate an OpenSSL CA at runtime; CI runners must have OpenSSL available (or the workflow must install it) for tests to run. Tests also rely on third-wheel/mitm proxy and assert_cmd/predicates for CLI output checks.

  • The added GitHub Actions job (coverage.yml change) includes an explicit Install OpenSSL step for the unstable-rest-resolve job — this mirrors the runtime requirement of the new tests and is necessary in CI to avoid failures. Confirm the PR’s workflow adds that step (otherwise CI will fail).

  • Tests and CI enforce strict -D warnings and clippy rules; feature-gated lint/test runs must match Cargo features and CLIPPY_FLAGS in Makefile/workflows to avoid unexpected failures.

  • E2E tests use recorded fixtures and the MITM handler; they expect deterministic REST/GraphQL interactions and validate stdout/stderr via predicates — the new parameterized tests return (calls, stdout, stderr) so reviewers should verify assertions on stderr/stdout match expected behavior for each status case.

🔇 Additional comments (6)
tests/resolve.rs (6)

3-4: Gate the module at the file level — LGTM.

Keep the single crate-level cfg; avoid reintroducing per-item cfg noise.


8-8: Import predicates for stderr/stdout assertions — LGTM.

This enables strong UX checks without custom helpers.


45-50: Assert clean TTY on success — LGTM.

Keep stdout/stderr empty checks to prevent regressions in user-visible noise.


72-76: Status routing for /replies vs. /graphql — LGTM.

Keep GraphQL OK and drive /replies via the case parameter to exercise fallback and fail-fast paths.

Also applies to: 87-87


117-126: Cover the 200-OK reply path in the matrix — LGTM.

This keeps the happy path co-located with failure cases.


136-139: Exercise the 403 path — LGTM.

Auth failures are now validated in the CLI error surface.

Comment thread tests/resolve.rs Outdated
codescene-delta-analysis[bot]

This comment was marked as outdated.

codescene-delta-analysis[bot]

This comment was marked as outdated.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
tests/resolve.rs (1)

14-16: Parametrise or drop rstest for single-case test.

Drive both “no message” and “-m done” or make this a plain test.

Apply:

 #[tokio::test]
 #[rstest::rstest]
-#[case(None)]
+#[case(None)]
+#[case(Some("done"))]
 async fn resolve_flows(#[case] msg: Option<&'static str>) {
📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between ba41dfc and 75a3e87.

📒 Files selected for processing (2)
  • .github/workflows/coverage.yml (1 hunks)
  • tests/resolve.rs (4 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.rs

📄 CodeRabbit inference engine (AGENTS.md)

**/*.rs: Every module must begin with a module level (//!) comment explaining the module's purpose and utility.
Document public APIs using Rustdoc comments (///) so documentation can be generated with cargo doc.
Place function attributes after doc comments.
Do not use return in single-line functions.
Use predicate functions for conditional criteria with more than two branches.
Prefer immutable data and avoid unnecessary mut bindings.
Handle errors with the Result type instead of panicking where feasible.
Prefer semantic error enums: Derive std::error::Error (via the thiserror crate) for any condition the caller might inspect, retry, or map to an HTTP status.
Use an opaque error only at the app boundary: Use eyre::Report for human-readable logs; these should not be exposed in public APIs.
Never export the opaque type from a library: Convert to domain enums at API boundaries, and to eyre only in the main main() entrypoint or top-level async task.
Clippy warnings MUST be disallowed.
Fix any warnings emitted during tests in the code itself rather than silencing them.
Where a function is too long, extract meaningfully named helper functions adhering to separation of concerns and CQRS.
Where a function has too many parameters, group related parameters in meaningfully named structs.
Where a function is returning a large error consider using Arc to reduce the amount of data returned.
Write unit and behavioural tests for new functionality. Run both before and after making any change.
Avoid unsafe code unless absolutely necessary and document any usage clearly.
Lints must not be silenced except as a last resort.
Lint rule suppressions must be tightly scoped and include a clear reason.
Prefer expect over allow.
Prefer .expect() over .unwrap().
Use concat!() to combine long string literals rather than escaping newlines with a backslash.

Files:

  • tests/resolve.rs

⚙️ CodeRabbit configuration file

**/*.rs: * Seek to keep the cyclomatic complexity of functions no more than 12.

  • Adhere to single responsibility and CQRS

  • Place function attributes after doc comments.

  • Do not use return in single-line functions.

  • Move conditionals with >2 branches into a predicate function.

  • Avoid unsafe unless absolutely necessary.

  • Every module must begin with a //! doc comment that explains the module's purpose and utility.

  • Comments and docs must follow en-GB-oxendict (-ize / -our) spelling and grammar

  • Lints must not be silenced except as a last resort.

    • #[allow] is forbidden.
    • Only narrowly scoped #[expect(lint, reason = "...")] is allowed.
    • No lint groups, no blanket or file-wide suppression.
    • Include FIXME: with link if a fix is expected.
  • Where code is only used by specific features, it must be conditionally compiled or a conditional expectation for unused_code applied.

  • Use rstest fixtures for shared setup and to avoid repetition between tests.

  • Replace duplicated tests with #[rstest(...)] parameterised cases.

  • Prefer mockall for mocks/stubs.

  • Prefer .expect() over .unwrap()

  • Ensure that any API or behavioural changes are reflected in the documentation in docs/

  • Ensure that any completed roadmap steps are recorded in the appropriate roadmap in docs/

  • Files must not exceed 400 lines in length

    • Large modules must be decomposed
    • Long match statements or dispatch tables should be decomposed by domain and collocated with targets
    • Large blocks of inline data (e.g., test fixtures, constants or templates) must be moved to external files and inlined at compile-time or loaded at run-time.
  • Environment access (env::set_var and env::remove_var) are always unsafe in Rust 2024 and MUST be marked as such

    • For testing of functionality depending upon environment variables, dependency injection and the mockable crate are the preferred option.
    • If mockable cannot be used, env mutations in tests ...

Files:

  • tests/resolve.rs
🧬 Code graph analysis (1)
tests/resolve.rs (1)
tests/utils/mod.rs (3)
  • start_mitm (49-98)
  • vk_cmd (109-117)
  • shutdown (30-33)
🔍 Remote MCP Ref

Summary of additional facts relevant to reviewing PR #120

  • Cargo features: repository Cargo.toml does not define an unstable-rest-resolve feature on main — running tests with --features unstable-rest-resolve requires adding that feature or existing feature definitions to the branch under review.,

  • Test runtime dependencies: the new tests spin up a MITM proxy and generate an OpenSSL CA at runtime; CI must install OpenSSL for tests to run. The PR’s added CI job in .github/workflows/coverage.yml includes an explicit "Install OpenSSL" step for the unstable-rest-resolve job (so workflow change is required/expected). Verify the workflow change is present in the PR branch.,

  • Lint/test flags: CI and Makefile use feature-gated CLIPPY_FLAGS and RUSTFLAGS (-D warnings). The unstable-rest-resolve feature must align with the Makefile/workflow CLIPPY_FLAGS and test invocation to avoid lint/test failures. Confirm Makefile targets referenced by the workflow accept the feature and flags as used.

  • Tests structure and assertions: tests/resolve.rs on the branch introduces run_reply_flow(...) to centralize MITM setup, logs and invoke vk_cmd; tests are parameterized (rstest) to cover REST statuses 200/404/403/500 and assert stdout/stderr presence (stdout expected empty; stderr empty for success, contains "replies" + status code for failures). Reviewers should verify expected GraphQL fallback behavior for 404 and CLI failure behavior for 500 are correctly simulated and asserted.

Tools/sources used: Ref_ref_search_documentation, Ref_ref_read_url.

🔇 Additional comments (7)
.github/workflows/coverage.yml (1)

47-48: Drop the suggested change — Makefile already enables the feature-gated tests

The Makefile's test target runs $(CARGO) test --all-targets --all-features; with CARGO="cargo --locked" the CI make test invocation already enables unstable-rest-resolve (no direct cargo test change required).

Likely an incorrect or invalid review comment.

tests/resolve.rs (6)

3-4: Gate the entire module on the feature. LGTM.


8-8: Predicate-based stderr/stdout checks imported. LGTM.


45-50: Assert silent success. LGTM.

Validate zero stdout/stderr on success.


92-103: Side-effect-free helper returning ExitStatus. LGTM.

Also applies to: 107-108


110-121: Exercise 200/404/403/500 matrix. LGTM.

Also applies to: 121-129, 130-134, 135-139


140-163: Generalised failure assertion and strict output checks. LGTM.

Comment thread tests/resolve.rs
codescene-delta-analysis[bot]

This comment was marked as outdated.

codescene-delta-analysis[bot]

This comment was marked as outdated.

@leynos

leynos commented Sep 12, 2025

Copy link
Copy Markdown
Owner Author

@coderabbitai pause

@coderabbitai

coderabbitai Bot commented Sep 12, 2025

Copy link
Copy Markdown
✅ Actions performed

Reviews paused.

@leynos
leynos force-pushed the codex/add-negative-tests-for-rest-api-failures branch from 8b03c0c to 4c0fef4 Compare September 12, 2025 15:24
@leynos
leynos merged commit f9fda35 into main Sep 12, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant