Skip to content

test failed in CI: omicron-nexus::test_all integration_tests::target_release::get_set_target_release #10490

@hawkw

Description

@hawkw

This test failed on a CI run on **pull request #10486 **:

https://github.com/oxidecomputer/omicron/pull/10486/checks?check_run_id=77474398506

Log showing the specific test failure:

https://buildomat.eng.oxide.computer/wg/0/details/01KS8Y41ZSEPTKTAYFXAN8H8XT/TT22PNIXBDYhmIzKybIq9907RlqtUR4K4trkL4SyaGA3z4Jj/01KS8Y4BGGTN408FT8X5C90DPK#S8809

Excerpt from the log showing the failure:

  stdout ───

    running 1 test
    test integration_tests::target_release::get_set_target_release ... FAILED

    failures:

    failures:
        integration_tests::target_release::get_set_target_release

    test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 561 filtered out; finished in 11.10s
    
  stderr ───
    log file: /var/tmp/omicron_tmp/test_all-8578c2eb121ad81c-get_set_target_release.22254.0.log
    note: configured to log to "/var/tmp/omicron_tmp/test_all-8578c2eb121ad81c-get_set_target_release.22254.0.log"
    DB URL: postgresql://root@[::1]:43512/omicron?sslmode=disable
    DB address: [::1]:43512
    log file: /var/tmp/omicron_tmp/test_all-8578c2eb121ad81c-get_set_target_release.22254.2.log
    note: configured to log to "/var/tmp/omicron_tmp/test_all-8578c2eb121ad81c-get_set_target_release.22254.2.log"

    thread 'integration_tests::target_release::get_set_target_release' (2) panicked at nexus/test-utils/src/resource_helpers.rs:132:13:
    failed to make "GET" request to /v1/system/update/status: expected status code 200 OK, found 500 Internal Server Error
    stack backtrace:
       0: __rustc::rust_begin_unwind
                 at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/std/src/panicking.rs:689:5
       1: core::panicking::panic_fmt
                 at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/panicking.rs:80:14
       2: nexus_test_utils::resource_helpers::object_get::<nexus_types_versions::v2026_05_20_00::update::UpdateStatus>::{closure#0}::{closure#0}
                 at ./test-utils/src/resource_helpers.rs:132:13
       3: <core::result::Result<nexus_test_utils::http_testing::TestResponse, anyhow::Error>>::unwrap_or_else::<nexus_test_utils::resource_helpers::object_get<nexus_types_versions::v2026_05_20_00::update::UpdateStatus>::{closure#0}::{closure#0}>
                 at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/result.rs:1622:23
       4: {async_fn#0}<nexus_types_versions::v2026_05_20_00::update::UpdateStatus>
                 at ./test-utils/src/resource_helpers.rs:131:10
       5: {async_block#0}
                 at ./tests/integration_tests/target_release.rs:74:56
       6: poll<&mut dyn core::future::future::Future<Output=core::result::Result<(), anyhow::Error>>>
                 at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/future/future.rs:133:9
       7: {closure#0}<core::pin::Pin<&mut dyn core::future::future::Future<Output=core::result::Result<(), anyhow::Error>>>>
                 at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/park.rs:284:71
       8: with_budget<core::task::poll::Poll<core::result::Result<(), anyhow::Error>>, tokio::runtime::park::{impl#4}::block_on::{closure_env#0}<core::pin::Pin<&mut dyn core::future::future::Future<Output=core::result::Result<(), anyhow::Error>>>>>
                 at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/task/coop/mod.rs:167:5
       9: budget<core::task::poll::Poll<core::result::Result<(), anyhow::Error>>, tokio::runtime::park::{impl#4}::block_on::{closure_env#0}<core::pin::Pin<&mut dyn core::future::future::Future<Output=core::result::Result<(), anyhow::Error>>>>>
                 at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/task/coop/mod.rs:133:5
      10: block_on<core::pin::Pin<&mut dyn core::future::future::Future<Output=core::result::Result<(), anyhow::Error>>>>
                 at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/park.rs:284:31
      11: <tokio::runtime::context::blocking::BlockingRegionGuard>::block_on::<core::pin::Pin<&mut dyn core::future::future::Future<Output = core::result::Result<(), anyhow::Error>>>>
                 at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/context/blocking.rs:66:14
      12: <tokio::runtime::scheduler::multi_thread::MultiThread>::block_on::<core::pin::Pin<&mut dyn core::future::future::Future<Output = core::result::Result<(), anyhow::Error>>>>::{closure#0}
                 at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/multi_thread/mod.rs:92:22
      13: tokio::runtime::context::runtime::enter_runtime::<<tokio::runtime::scheduler::multi_thread::MultiThread>::block_on<core::pin::Pin<&mut dyn core::future::future::Future<Output = core::result::Result<(), anyhow::Error>>>>::{closure#0}, core::result::Result<(), anyhow::Error>>
                 at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/context/runtime.rs:65:16
      14: <tokio::runtime::scheduler::multi_thread::MultiThread>::block_on::<core::pin::Pin<&mut dyn core::future::future::Future<Output = core::result::Result<(), anyhow::Error>>>>
                 at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/scheduler/multi_thread/mod.rs:91:9
      15: <tokio::runtime::runtime::Runtime>::block_on_inner::<core::pin::Pin<&mut dyn core::future::future::Future<Output = core::result::Result<(), anyhow::Error>>>>
                 at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/runtime.rs:373:50
      16: block_on<core::pin::Pin<&mut dyn core::future::future::Future<Output=core::result::Result<(), anyhow::Error>>>>
                 at /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/tokio-1.52.1/src/runtime/runtime.rs:345:18
      17: get_set_target_release
                 at ./tests/integration_tests/target_release.rs:196:7
      18: test_all::integration_tests::target_release::get_set_target_release::{closure#0}
                 at ./tests/integration_tests/target_release.rs:42:38
      19: <test_all::integration_tests::target_release::get_set_target_release::{closure#0} as core::ops::function::FnOnce<()>>::call_once
                 at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/ops/function.rs:250:5
      20: core::ops::function::FnOnce::call_once
                 at /rustc/4a4ef493e3a1488c6e321570238084b38948f6db/library/core/src/ops/function.rs:250:5
    note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
    WARN: dropped CockroachInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked)
    WARN: temporary directory leaked: "/var/tmp/omicron_tmp/.tmpJjEtBQ"
    	If you would like to access the database for debugging, run the following:

    	# Run the database
    	cargo xtask db-dev run --no-populate --store-dir "/var/tmp/omicron_tmp/.tmpJjEtBQ/data"
    	# Access the database. Note the port may change if you run multiple databases.
    	cockroach sql --host=localhost:32221 --insecure
    WARN: dropped ClickHouse process without cleaning it up first (there may still be a child process running (PID 22271) and a temporary directory leaked, /var/tmp/omicron_tmp/test_all-8578c2eb121ad81c-get_set_target_release.22254.1-clickhouse-089JI6)
    failed to clean up ClickHouse data dir:
    - /var/tmp/omicron_tmp/test_all-8578c2eb121ad81c-get_set_target_release.22254.1-clickhouse-089JI6: File exists (os error 17)
    WARN: dropped DendriteInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked)
    WARN: dendrite temporary directory leaked: /var/tmp/omicron_tmp/.tmpIEDJVA
    WARN: dropped MgdInstance without cleaning it up first (there may still be a child process running and a temporary directory leaked)
    WARN: mgd temporary directory leaked: /var/tmp/omicron_tmp/.tmp9u4DOo

Metadata

Metadata

Assignees

No one assigned

    Labels

    Test FlakeTests that work. Wait, no. Actually yes. Hang on. Something is broken.

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions