Skip to content

clickhouse setup in test suite could report more info for debugging #3011

@davepacheco

Description

@davepacheco

The code that sets up a CockroachDB database for testing logs a bunch of stuff:

info!(
&log,
"cockroach temporary directory: {}",
starter.temp_dir().display()
);
// If we're going to copy the storage directory from the seed,
// it is critical we do so before starting the DB.
if matches!(storage_source, StorageSource::CopyFromSeed) {
info!(&log,
"cockroach: copying from seed directory ({}) to storage directory ({})",
seed_dir.to_string_lossy(), starter.store_dir().to_string_lossy(),
);
copy_dir(seed_dir, starter.store_dir())
.expect("Cannot copy storage from seed directory");
}
info!(&log, "cockroach command line: {}", starter.cmdline());
info!(
&log,
"cockroach environment: {}",
starter
.environment()
.map(|(k, v)| format!("{}={}", k, v))
.collect::<Vec<_>>()
.join(" ")
);
let database = starter.start().await.unwrap_or_else(|error| {
panic!("failed to start CockroachDB: {:#}", error);
});
info!(&log, "cockroach pid: {}", database.pid());
let db_url = database.pg_config();
info!(&log, "cockroach listen URL: {}", db_url);

  • the temporary directory it's using for database storage
  • the environment and command line it used to exec cockroach
  • the pid and listen URL for cockroach

This is aimed at a few goals:

  • if cockroach is still running when you want to debug it (e.g., because a test hung), you have the listen URL (so you can inspect the database) and pid (so that you can use any of the usual debugging tools on it)
  • if the test failed and cockroach is no longer running, you can use the temp directory to start up a new cockroach instance on the same database to inspect the contents of the database when the test failed (omicron-dev db-run --store-dir=... --no-populate makes this easy)
  • if we have some other weird problem that we suspect is related to the environment or how we're executing cockroach, we know exactly how it ran (and can run it again the same way)

In debugging #2994 I wanted these for Clickhouse too -- particularly, to know what the contents of the Clickhouse database were. It'd be helpful if we logged a bunch more about how we start Clickhouse.

Metadata

Metadata

Assignees

No one assigned

    Labels

    DebuggingFor when you want better data in debugging an issue (log messages, post mortem debugging, and more)good first issueIssues that are good for learning the codebase

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions