DAOS-18304 ddb: Introduce DdbAPI interface and fix db_path propagation by knard38 · Pull Request #18124 · daos-stack/daos

knard38 · 2026-04-28T14:17:04Z

Description

This PR is the second split of #17967, broken out at reviewers' request to make the review process easier. The first split #18125, contains unrelated code fixes found during the creation of the initial PR #17967. The content of this PR cannot be reduced further without breaking the build, as the Go interface and the C dv_pool_open API changes are tightly coupled: the new method signatures in the Go layer directly depend on the updated C function signature, so neither half can land independently.

This patch refactors the ddb Go CLI layer and the underlying C VOS pool open API to improve testability and fix a db_path propagation bug.

Go layer

Introduces a DdbAPI interface in commands_wrapper.go capturing all ddb operations, decoupling command dispatch from the concrete DdbContext implementation.
Converts all standalone ddbXxx(ctx *DdbContext, ...) functions to methods on *DdbContext, making DdbContext implement DdbAPI.
Passes dbPath explicitly to Open, Feature, and RmPool instead of storing it globally in ctx.ctx.dc_db_path, preventing the value from leaking across commands.
Renames Go parameters to follow camelCase conventions.

C layer

Simplifies the dv_pool_open API: callers pass a *ddb_ctx directly instead of a pre-parsed vos_file_parts; the function populates ctx->dc_poh.
Stores the full VOS file path in vf_vos_file_path within vos_file_parts, enabling callers to retrieve it without re-parsing.
Adds a VOS_PATH_SIZE overflow test and improves assertion coverage in ddb_parse_tests.c.

Steps for the author:

Commit message follows the guidelines.
Appropriate Features or Test-tag pragmas were used.
Appropriate Functional Test Stages were run.
At least two positive code reviews including at least one code owner from each category referenced in the PR.
Testing is complete. If necessary, forced-landing label added and a reason added in a comment.

After all prior steps are complete:

Gatekeeper requested (daos-gatekeeper added as a reviewer).

…limits - Add vf_vos_file_path field to vos_file_parts to retain the original VOS file path through the parsing pipeline, so callers no longer need to keep the input pointer alive after parse_vos_file_parts() returns. - Rename the enum to introduce VOS_PATH_SIZE (alongside DB_PATH_SIZE) and raise both constants from 256 to PATH_MAX. - Validate the VOS path length in parse_vos_file_parts() and return -DER_EXCEEDS_PATH_LEN when the limit is exceeded. Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@hpe.com>

…hing from ddb_ctx Remove dc_pool_path and dc_db_path from ddb_ctx: callers now pass paths as explicit arguments instead of storing them in the context, eliminating the need for the SetCString() helper. dv_pool_open() and dv_pool_destroy() signatures are simplified from (path, path_parts, poh, flags, write_mode) to (path, db_path, ctx, flags): - vos_file_parts allocation is moved into the new internal helper create_vos_file_parts(), keeping the API surface minimal. - dc_write_mode and dc_poh are set directly on the ctx, so the caller gets the pool handle back without an extra out-parameter. ddb_run_open(), ddb_run_feature() and ddb_run_rm_pool() in ddb_commands.c are simplified accordingly. ddb_run_feature() also gains an explicit error message on dv_pool_open() failure. ddb_run_dtx_stat() no longer prints ctx->dc_pool_path (which no longer exists). Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@hpe.com>

Update all dv_pool_open() and dv_pool_destroy() call sites to use the new (path, db_path, ctx) signature: path_parts is no longer allocated by callers, and the pool handle is read back from ctx->dc_poh. ddb_parse_tests.c: add vf_vos_file_path assertions in success cases; add a test for VOS paths exceeding VOS_PATH_SIZE; add a dedicated MD-on-SSD test for DB_PATH_SIZE independently of VOS_PATH_SIZE; introduce the MOCKED_VOS_PATH_STR macro to avoid repeated string literals. ddb_commands_tests.c: move dvt_vos_insert_2_records_with_dtx() into dcv_suit_setup() so dtx records are present for all tests in the suite; correct VOS tree path indices for ls, value_dump and ilog_dump tests; update the dtx_stat expected message. ddb_vos_tests.c: update open_pool_test(), pool_flags_tests(), dv_test_setup() and helper_stat_open_modify_close_stat() for the new API. Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@hpe.com>

…methods Add the DdbAPI interface that lists every ddb sub-command as a method. It decouples the command layer from the concrete CGo implementation so tests can inject a stub without a live VOS environment. Convert all ddbXxx() free functions to (ctx *DdbContext) Xxx() methods, matching the DdbAPI interface. InitDdb() becomes (ctx *DdbContext) Init() so the caller (or a stub) controls object allocation. Open(), Feature() and RmPool() now receive db_path as an explicit argument instead of reading it from ctx.ctx.dc_db_path (which was removed from the C struct in the previous commit). Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@hpe.com>

ddb_commands.go: addAppCommands() now takes a DdbAPI instead of a *DdbContext; every command Run closure calls api.Xxx() instead of the former ddbXxx(ctx, ...) free functions. The 'open' and 'feature' commands pass db_path explicitly; 'rm_pool' now requires --vos_path. main.go: - parseOpts() no longer calls os.Exit(); it returns (cliOptions, *flags.Parser, error) so the function is fully unit-testable. - A new run() function owns the DDB context lifetime: Init/defer-cleanup, auto-open logic, non-interactive command dispatch, interactive loop. - --version is handled in main() before run(), printing directly instead of synthesising a fake command. - --db_path without --vos_path returns an explicit error. - The HasPrefix loop for auto-open exclusions is replaced by a noAutoOpen map for exact command-name matching. - Pool close is consolidated into a closePoolIfOpen() helper used with defer. - Errors from runCmdStr() and runFileCmds() are now propagated. - unknownCmdError typed struct enables type-safe detection; exitWithError() prints the command list on unknown commands and fixes the duplicated ERROR: prefix in fault resolution messages. - errHelpRequested sentinel avoids os.Exit() in the --help path. Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@hpe.com>

Fix reviewers comments: - Use MOCKED_VOS_PATH_STR instead of the hardcoded "/" MOCKED_POOL_UUID_STR "/vos-0" string literal. Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@hpe.com>

Fixed reviewers comments: - Use slice instead of dictionary Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@hpe.com>

github-actions · 2026-04-28T15:32:35Z

Ticket title is 'Add unit test to ddb go code'
Status is 'In Review'
https://daosio.atlassian.net/browse/DAOS-18304

daosbuild3 · 2026-04-29T21:28:49Z

Test stage Functional Hardware Medium MD on SSD completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-18124/2/execution/node/1184/log

janekmi

A few minor issues. I am very happy with this change otherwise. 👍

janekmi · 2026-05-01T16:50:58Z


-	/* Test invalid vos paths with too long db path */
+	/* Test invalid vos paths with too long vos path */
+	D_ALLOC_ARRAY_CHECK(buf, VOS_PATH_SIZE + 1);


Nitpick. But you do not have to allocate, free and re-allocate again. You can make it big enough to handle all cases and just populate it with what is necessary.

The reason I allocate, free, and re-allocate is to ensure the buffer is always freed even when an assertion fails. From what I know, cmocka's assert_* macros use longjmp under the hood, so a failing assert exits the test immediately, bypassing any cleanup that follows.
By freeing buf before calling assert_rc_equal, I guarantee no leak regardless of the test outcome.

There is no leak when the process is terminated. AFAIK OS reclaims all of the process' memory on termination. So, you do not have to worry about it.

You may argue it is a bad practice. But IMHO the happy day scenario is even worse since each of the successful rounds do a lot more than is necessary just for the sake of testing what you want to test.

Please reconsider.

Update help text and descriptions to use "VOS" instead of "vos" for consistency. Also clarify "vos db" references to use "sys db" where appropriate. Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@hpe.com>

Fix reviewers comments. Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@hpe.com>

…/daos-18304-patch-001-split

janekmi · 2026-05-05T15:41:49Z

 		Name:      "close",
 		Aliases:   nil,
-		Help:      "Close the currently opened vos pool shard",
+		Help:      "Close the currently opened VOS pool shard",


You may chose to ignore this one. Since it is getting further and further from your core change.

We are opening a file but closing a shard? 🤔

Suggested change

Help: "Close the currently opened VOS pool shard",

Help: "Close the currently opened VOS pool file",

janekmi · 2026-05-05T15:43:26Z

 		Name:      "rm_pool",
 		Aliases:   nil,
-		Help:      "Remove a vos pool.",
+		Help:      "Remove a VOS pool.",


A little bit similar here. We are removing a pool but the argument is a file. Confusing.

kanard38 added 6 commits April 28, 2026 13:41

DAOS-18304 ddb: use MOCKED_VOS_PATH_STR macro in Go unit tests

7b3c6ac

Fix reviewers comments: - Use MOCKED_VOS_PATH_STR instead of the hardcoded "/" MOCKED_POOL_UUID_STR "/vos-0" string literal. Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@hpe.com>

knard38 changed the title ~~DAOS-18304 ddb: TODO~~ DAOS-18304 ddb: Introduce DdbAPI interface and fix db_path propagation Apr 28, 2026

knard38 mentioned this pull request Apr 28, 2026

DAOS-18304 ddb: Fix memory leaks, defer ordering, and pool handle corruption #18125

Open

6 tasks

DAOS-18304 ddb: replace noAutoOpen map with a string slice

b0f7207

Fixed reviewers comments: - Use slice instead of dictionary Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@hpe.com>

knard38 self-assigned this Apr 28, 2026

knard38 mentioned this pull request Apr 28, 2026

DAOS-18304 ddb: Introduce DdbAPI interface for Go unit testing #17967

Closed

6 tasks

knard38 marked this pull request as ready for review April 28, 2026 15:37

knard38 requested review from a team as code owners April 28, 2026 15:37

knard38 requested review from Nasf-Fan, janekmi, kjacque, mjmac and tanabarr April 28, 2026 15:38

janekmi requested changes May 1, 2026

View reviewed changes

kanard38 added 2 commits May 4, 2026 09:19

daos-18304 ddb: capitalize VOS in help strings

425fc48

Update help text and descriptions to use "VOS" instead of "vos" for consistency. Also clarify "vos db" references to use "sys db" where appropriate. Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@hpe.com>

daos-18304 ddb: improve unknown command error message

beaeed7

Fix reviewers comments. Signed-off-by: Cedric Koch-Hofer <cedric.koch-hofer@hpe.com>

knard38 requested a review from janekmi May 4, 2026 09:35

Merge remote-tracking branch 'origin/master' into ckochhof/fix/master…

c12dcbb

…/daos-18304-patch-001-split

janekmi approved these changes May 5, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DAOS-18304 ddb: Introduce DdbAPI interface and fix db_path propagation#18124

DAOS-18304 ddb: Introduce DdbAPI interface and fix db_path propagation#18124
knard38 wants to merge 10 commits intomasterfrom
ckochhof/fix/master/daos-18304-patch-001-split

knard38 commented Apr 28, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Apr 28, 2026 •

edited

Loading

Uh oh!

daosbuild3 commented Apr 29, 2026

Uh oh!

janekmi left a comment

Uh oh!

janekmi May 1, 2026

Uh oh!

knard38 May 4, 2026

Uh oh!

janekmi May 5, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

janekmi May 5, 2026

Uh oh!

janekmi May 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

4 participants

	Help: "Close the currently opened VOS pool shard",
	Help: "Close the currently opened VOS pool file",

Conversation

knard38 commented Apr 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Go layer

C layer

Steps for the author:

After all prior steps are complete:

Uh oh!

github-actions Bot commented Apr 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

daosbuild3 commented Apr 29, 2026

Uh oh!

janekmi left a comment

Choose a reason for hiding this comment

Uh oh!

janekmi May 1, 2026

Choose a reason for hiding this comment

Uh oh!

knard38 May 4, 2026

Choose a reason for hiding this comment

Uh oh!

janekmi May 5, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

janekmi May 5, 2026

Choose a reason for hiding this comment

Uh oh!

janekmi May 5, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

4 participants

knard38 commented Apr 28, 2026 •

edited

Loading

github-actions Bot commented Apr 28, 2026 •

edited

Loading