neon_local: use pageserver.toml as source of truth for struct PageServerConf
#7642
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…side of `neon_local init` The `NeonCli.init()` persists the non-default pageserver config values for remote storage & `NeonEnvBuilder.pageserver_config_override` in `pageserver.toml`. We don't need to repeat them on each pageserver start after that.
…blame/test-suite-narrow-pageserver-config-override
Rewrite the pageserver.toml instead.
…--pageserver-config-override`
This allows inlining append_pageserver_param_overrides into NeonCli.init()
…ng init, not start
…age during init, not start" This reverts commit 511f593.
…into problame/remove-pageserver-config-overrides
…akes a toml tempfile
…into problame/remove-pageserver-config-overrides
…into problame/remove-pageserver-config-overrides
Conflicts:
control_plane/src/pageserver.rs
=> pick ours
…r --init --config-override`
…ame/neon-local-init-dont-use-config-override
The neon_local's `PageServerConf` struct isn't the source of truth for pageserver config anymore.
… problame/neon-local-pageserver-toml-source-of-truth
… problame/neon-local-pageserver-toml-source-of-truth
3024 tests run: 2891 passed, 0 failed, 133 skipped (full report)Code coverage* (full report)
* collected from Rust tests only The comment gets automatically updated with the latest test results
c9f63e9 at 2024-05-08T14:40:20.773Z :recycle: |
Base automatically changed from
problame/neon-local-init-dont-use-config-override
to
main
May 8, 2024 09:03
VladLazar
approved these changes
May 8, 2024
a-masterov
pushed a commit
that referenced
this pull request
May 20, 2024
…ServerConf` (#7642) Before this PR, `neon_local` would store a copy of a subset of the initial `pageserver.toml` in its `.neon/config`, e.g, `listen_pg_addr`. That copy is represented as `struct PageServerConf`. This copy was used to inform e.g., `neon_local endpoint` and other commands that depend on Pageserver about which port to connect to. The problem with that scheme is that the duplicated information in `.neon/config` can get stale if `pageserver.toml` is changed. This PR fixes that by eliminating populating `struct PageServerConf` from the `pageserver.toml`s. The `[[pageservers]]` TOML table in the `.neon/config` is obsolete. As of this PR, `neon_local` will fail to start and print an error informing about this change. Code-level changes: - Remove the `--pg-version` flag, it was only used for some checks during `neon_local init` - Remove the warn-but-continue behavior for when auth key creation fails but auth keys are not required. It's just complexity that is unjustified for a tool like `neon_local`. - Introduce a type-system-level distinction between the runtime state and the two (!) toml formats that are almost the same but not quite. - runtime state: `struct PageServerConf`, now without `serde` derives - toml format 1: the state in `.neon/config` => `struct OnDiskState` - toml format 2: the `neon_local init --config TMPFILE` that, unlike `struct OnDiskState`, allows specifying `pageservers` - Remove `[[pageservers]]` from the `struct OnDiskState` and load the data from the individual `pageserver.toml`s instead.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Before this PR,
neon_localwould store a copy of a subset of the initialpageserver.tomlin its.neon/config, e.g,listen_pg_addr. That copy is represented asstruct PageServerConf.This copy was used to inform e.g.,
neon_local endpointand other commands that depend on Pageserver about which port to connect to.The problem with that scheme is that the duplicated information in
.neon/configcan get stale ifpageserver.tomlis changed.This PR fixes that by eliminating populating
struct PageServerConffrom thepageserver.tomls.The
[[pageservers]]TOML table in the.neon/configis obsolete.As of this PR,
neon_localwill fail to start and print an error informing about this change.Code-level changes:
--pg-versionflag, it was only used for some checks duringneon_local initneon_local.struct PageServerConf, now withoutserdederives.neon/config=>struct OnDiskStateneon_local init --config TMPFILEthat, unlikestruct OnDiskState, allows specifyingpageservers[[pageservers]]from thestruct OnDiskStateand load the data from the individualpageserver.tomls instead.