feat(sdk): Scenario

[sid-rl]

User description

⚠️ PR Title Must Follow Conventional Commits

Format: feat[optional scope]: <description>

Examples: feat: add new SDK method · feat(storage): support file uploads · feat!: breaking API change

---

Description

Motivation

Changes

Testing

• Unit tests added
• Integration tests added
• Smoke Tests added/updated
• Tested locally

Breaking Changes

Checklist

• PR title follows Conventional Commits format (feat: or feat(scope):)
• Documentation updated (if needed)
• Breaking changes documented (if applicable)

---

CodeAnt-AI Description

Add object-oriented Scenario API to SDK

What Changed

• Exposes RunloopSDK.scenario so callers can get scenarios by ID or list scenarios directly from the SDK
• Adds Scenario object with user-facing methods: getInfo (fetch details), update (modify scenario fields), runAsync (start a run and return immediately), and run (start a run and wait until the devbox is ready)
• Includes unit and smoke tests that verify scenario creation, retrieval, updates, and both run modes work as expected

Impact

✅ Direct scenario management from SDK
✅ Start-and-wait scenario runs from SDK
✅ Fewer manual API calls to start and poll runs

💡 Usage Guide

Checking Your Pull Request

Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.

Talking to CodeAnt AI

Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:

@codeant-ai ask: Your question here

This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.

Example

@codeant-ai ask: Can you suggest a safer alternative to storing this secret?

Preserve Org Learnings with CodeAnt

You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:

@codeant-ai: Your feedback here

This helps CodeAnt AI learn and adapt to your team's coding style and standards.

Example

@codeant-ai: Do not flag unused imports.

Retrigger review

Ask CodeAnt AI to review the PR again, by typing:

@codeant-ai: review

Check Your Repository Health

To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.

[codeant-ai]

CodeAnt AI is reviewing your PR.

---

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

[codeant-ai]

Nitpicks 🔍

🔒 No security issues identified

⚡ Recommended areas for review Scenario list API mismatch ScenarioOps.list assumes the response has a scenarios array (iterable via for (const scenario of result.scenarios)). Other list endpoints in this file sometimes return async iterables or a top-level page shape. Confirm the API surface of this.client.scenarios.list (async iterable vs page object with .scenarios) and make list handling consistent with the actual return type to avoid runtime errors or missed items. Params merging risk Both runAsync and run call API methods by merging { scenario_id: this._id, ...params }. While ScenarioRunParams omits scenario_id, a caller could still cast or pass a value at runtime. That may unintentionally overwrite the intended scenario_id. Consider explicitly filtering out scenario_id from params before merging, or use an object assign pattern that ensures scenario_id cannot be overwritten. Missing devbox_id check The code constructs new ScenarioRun instances from the API response (runView) without validating that devbox_id (or other required fields) is present. If the backend ever returns a run without devbox_id (or returns null/undefined), downstream code that assumes a devbox will exist may throw runtime errors. Consider validating and returning a clearer error or handling the absent devbox_id case. Resource cleanup missing The tests shut down the devbox in afterAll but do not delete the created scenario itself. This may leave test artifacts (scenarios) in the system after test runs. Consider deleting or archiving the scenario in cleanup. Type-only export compatibility The PR adds a combined value+type re-export using the type modifier. Some toolchains (older Babel setups, older TS versions, or isolatedModules configurations) can behave differently with inline type modifiers or drop type-only exports in emitted JS/typing artifacts. Verify the build pipeline, tsconfig, and that ScenarioRunParams is exported as a type from './scenario'.

[codeant-ai]

CodeAnt AI finished reviewing your PR.

[github-actions]

⚠️ Object Smoke Tests & Coverage Report

Test Results

✅ All smoke tests passed

Coverage Results

Metric	Coverage	Required	Status
Functions	97.83%	100%	❌
Lines	85.98%	-	ℹ️
Branches	65.11%	-	ℹ️
Statements	84.64%	-	ℹ️

Coverage Requirement: 100% function coverage (all public methods must be called in smoke tests)

⚠️ Some object methods are not covered in smoke tests. Please add tests that call all public methods.

View detailed coverage report

Coverage reports are available in the workflow artifacts. Lines/branches/statements coverage is tracked but not required to be 100%.

📋 View workflow run

[james-rl]

The code looks good, but the comments could do with some improvement. The top class-level comments are too vague to be useful and they're also repeated in several areas. Most importantly, they don't highlight the most important fields (particularly input_context.problem_statement and the reference solution).

More generally, I think we have to start anticipating that LLMs are going to be the ones reading these docs, not humans. They're very good at understanding code, but not good at understanding the domain or intended usage. Comments help bridge this gap.

Also, make sure you update the main README to (very briefly!) describe how benchmarks & scenarios work, as well as the various LLM files.

[github-actions]

⚠️ Object Smoke Tests & Coverage Report

Test Results

✅ All smoke tests passed

Coverage Results

Metric	Coverage	Required	Status
Functions	97.83%	100%	❌
Lines	85.98%	-	ℹ️
Branches	65.11%	-	ℹ️
Statements	84.64%	-	ℹ️

Coverage Requirement: 100% function coverage (all public methods must be called in smoke tests)

⚠️ Some object methods are not covered in smoke tests. Please add tests that call all public methods.

View detailed coverage report

Coverage reports are available in the workflow artifacts. Lines/branches/statements coverage is tracked but not required to be 100%.

📋 View workflow run

[github-actions]

⚠️ Object Smoke Tests & Coverage Report

Test Results

✅ All smoke tests passed

Coverage Results

Metric	Coverage	Required	Status
Functions	90.81%	100%	❌
Lines	82.52%	-	ℹ️
Branches	60.93%	-	ℹ️
Statements	81.13%	-	ℹ️

Coverage Requirement: 100% function coverage (all public methods must be called in smoke tests)

⚠️ Some object methods are not covered in smoke tests. Please add tests that call all public methods.

View detailed coverage report

Coverage reports are available in the workflow artifacts. Lines/branches/statements coverage is tracked but not required to be 100%.

📋 View workflow run

[github-actions]

⚠️ Object Smoke Tests & Coverage Report

Test Results

✅ All smoke tests passed

Coverage Results

Metric	Coverage	Required	Status
Functions	97.83%	100%	❌
Lines	85.98%	-	ℹ️
Branches	65.11%	-	ℹ️
Statements	84.64%	-	ℹ️

Coverage Requirement: 100% function coverage (all public methods must be called in smoke tests)

⚠️ Some object methods are not covered in smoke tests. Please add tests that call all public methods.

View detailed coverage report

Coverage reports are available in the workflow artifacts. Lines/branches/statements coverage is tracked but not required to be 100%.

📋 View workflow run

[james-rl]

looks great to me!

[github-actions]

⚠️ Object Smoke Tests & Coverage Report

Test Results

✅ All smoke tests passed

Coverage Results

Metric	Coverage	Required	Status
Functions	97.83%	100%	❌
Lines	85.98%	-	ℹ️
Branches	65.11%	-	ℹ️
Statements	84.64%	-	ℹ️

Coverage Requirement: 100% function coverage (all public methods must be called in smoke tests)

⚠️ Some object methods are not covered in smoke tests. Please add tests that call all public methods.

View detailed coverage report

Coverage reports are available in the workflow artifacts. Lines/branches/statements coverage is tracked but not required to be 100%.

📋 View workflow run

[github-actions]

⚠️ Object Smoke Tests & Coverage Report

Test Results

✅ All smoke tests passed

Coverage Results

Metric	Coverage	Required	Status
Functions	97.83%	100%	❌
Lines	85.98%	-	ℹ️
Branches	65.11%	-	ℹ️
Statements	84.64%	-	ℹ️

Coverage Requirement: 100% function coverage (all public methods must be called in smoke tests)

⚠️ Some object methods are not covered in smoke tests. Please add tests that call all public methods.

View detailed coverage report

Coverage reports are available in the workflow artifacts. Lines/branches/statements coverage is tracked but not required to be 100%.

📋 View workflow run