Skip to content

Conversation

@tofarr
Copy link
Collaborator

@tofarr tofarr commented May 15, 2025

  • This change is worth documenting at https://docs.all-hands.dev/
  • Include this change in the Release Notes. If checked, you must provide an end-user friendly description for your change below

End-user friendly description of the problem this fixes or functionality this introduces.
This PR introduces reusable runtime fixtures for tests that don't modify the container state, significantly improving test performance by reducing the number of runtime environments that need to be created and torn down during testing.


Summarize what the PR does, explaining any non-trivial design decisions.

  1. Introduces module-scoped reusable runtime fixtures:

    • reusable_runtime_and_config - Creates a shared runtime that persists across multiple tests
    • reusable_runtime and reusable_config - Convenience fixtures that extract the runtime and config
  2. Improves runtime management:

    • Renamed _load_runtime to create_runtime_and_config for better clarity
    • Enhanced close_test_runtime to properly clean up resources
    • Added helper functions for common operations
  3. Optimizes test execution:

    • Tests that don't modify container state now share a runtime instance
    • Tests that need isolation still create their own runtime
    • Significantly reduces test execution time, especially for Docker-based tests

These changes maintain test isolation where needed while improving performance by reusing runtimes when possible. This is particularly valuable for CI/CD pipelines and local development.


Link of any specific issues this addresses:


To run this PR locally, use the following command:

docker run -it --rm   -p 3000:3000   -v /var/run/docker.sock:/var/run/docker.sock   --add-host host.docker.internal:host-gateway   -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:5554cad-nikolaik   --name openhands-app-5554cad   docker.all-hands.dev/all-hands-ai/openhands:5554cad

@openhands-ai openhands-ai bot deleted a comment from openhands-staging bot Jul 1, 2025
@openhands-ai
Copy link

openhands-ai bot commented Jul 1, 2025

Looks like there are a few issues preventing this PR from being merged!

  • GitHub Actions are failing:
    • Run Python Unit Tests

If you'd like me to help, just leave a comment, like

@OpenHands please fix the failing actions on PR #8520

Feel free to include any additional details that might help me get this PR into a better state.

You can manage your notification settings

@tofarr tofarr changed the title Introduced reusable runtimes for runtime tests (DRAFT) Introduced reusable runtimes for runtime tests Jul 1, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Aug 2, 2025

This PR is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

@github-actions github-actions bot added the Stale Inactive for 40 days label Aug 2, 2025
@github-actions
Copy link
Contributor

This PR was closed because it had no activity for 50 days. If you feel this was closed in error, and you would like to continue the PR, please resubmit or let us know.

@github-actions github-actions bot closed this Aug 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Stale Inactive for 40 days

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants