fix(test suite) virtual_file_io_engine and get_vectored_impl patametrization doesn't work

[problame]

Problem

While investigating #7124, I noticed that the benchmark was always using the DEFAULT_* virtual_file_io_engine , i.e., tokio-epoll-uring as of #7077.

The fundamental problem is that the control_plane code has its own view of PageServerConfig, which, I believe, will always be a subset of the real pageserver's pageserver/src/config.rs.

For the virtual_file_io_engine and get_vectored_impl parametrization of the test suite, we were constructing a dict on the Python side that contained these parameters, then handed it to control_plane::PageServerConfig's derived serde::Deserialize.
The default in serde is to ignore unknown fields, so, the Deserialize impl silently ignored the fields.
In consequence, the fields weren't propagated to the pageserver --init call, and the tests ended up using the pageserver/src/config.rs::DEFAULT_ values for the respective options all the time.

Tests that explicitly used overrides in env.pageserver.start() and similar were not affected by this.

But, it means that all the test suite runs where with parametrization didn't properly exercise the code path.

Changes

• use serde(deny_unknown_fields) to expose the problem
  • With this change, the Python tests that override virtual_file_io_engine and
    get_vectored_impl fail on pageserver --init, exposing the problem.
• use destructuring to uncover the issue in the future
• fix the issue by adding the missing fields to the control_plane crate's PageServerConf
  • A better solution would be for control plane to re-use a struct provided
    by the pageserver crate, so that everything is in one place in
    pageserver/src/config.rs, but, our config parsing code is (almost)
    beyond repair anyways.
• fix the pageserver_virtual_file_io_engine to be responsive to the env var
  • => required to make parametrization work in benchmarks

Testing

Before merging this PR, I re-ran the regression tests & CI with the full matrix of virtual_file_io_engine and tokio-epoll-uring, see 9c7ea36

[github-actions]

2732 tests run: 2592 passed, 1 failed, 139 skipped (full report)

---

Failures on Postgres 14

• test_bulk_insert[neon-github-actions-selfhosted]: release

# Run all failed tests locally:
scripts/pytest -vv -n $(nproc) -k "test_bulk_insert[neon-release-pg14-github-actions-selfhosted]"

Flaky tests (1)

Postgres 15

• test_vm_bit_clear_on_heap_lock: debug

Code coverage* (full report)

• functions: 28.4% (7031 of 24721 functions)
• lines: 47.2% (43470 of 92118 lines)

* collected from Rust tests only

---

_{The comment gets automatically updated with the latest test results
763c0b8 at 2024-03-14T11:26:28.864Z :recycle:}

[koivunej]

Ok, this explains the "override is always specified" for remote storage which I couldn't believe.

[problame]

Ok, this explains the "override is always specified" for remote storage which I couldn't believe.

Got more context on that?

[koivunej]

Ok, this explains the "override is always specified" for remote storage which I couldn't believe.

Got more context on that?

I think it was from last weeks huddles with you and me. You were explaining how to run pageserver against in-memory minio, configuring it in the pageserver.toml did not work because neon_local always specifies the override, which I didn't find sensible.

Expanding the code more between changes explained to me why:

neon/control_plane/src/pageserver.rs

Lines 138 to 145 in 9c7ea36

	if !cli_overrides
	.iter()
	.any(|c| c.starts_with("remote_storage"))
	{
	overrides.push(format!(
	"remote_storage={{local_path='../{PAGESERVER_REMOTE_STORAGE_DIR}'}}"
	));
	}

[problame]

test_bulk_insert` benchmark failure is a known issue => #7124