feat(server): Use global config from file if provided #2458

TBS1996 · 2023-09-04T05:27:50Z

Provides a way to load global config statically from a file.

If the relay mode is in managed mode, this file will be ignored as we want in that case to fetch from upstream.
If it's not in managed mode, it will attempt the load the file, if it exists it will serve it, if it doesn't exist, it will serve the default config. If it exists but we fail to open/parse it, an error will be logged and we will serve the default.

The location of the static config will be in the same folder as config.yml and credentials.json.

TBS1996 · 2023-09-04T05:53:01Z

relay-config/src/config.rs

@@ -1250,6 +1252,7 @@ impl ConfigObject for ConfigValues {
 pub struct Config {
    values: ConfigValues,
    credentials: Option<Credentials>,
+    global_config: Option<Arc<GlobalConfig>>,


wasn't 100% sure if i should put it here or not. My reasoning was that if we fail to parse the file, it's better to fail early than later, and it just seems idiomatic within the codebase that we don't do these kind of risky IO-stuff in the services.

We have a precedent where we defer the deserialization of a static config file to a service:

relay/relay-config/src/config.rs

Lines 1916 to 1919 in 498401a

/// Get filename for static project config.

pub fn project_configs_path(&self) -> PathBuf {

self.path.join("projects")

}

That service reloads the config every 10 seconds though, which I think is overkill. In short, I think your way of doing it is better because it fails early. I'll leave final review to @jan-auer.

I concur with this: We can load the global configs directly from within the actual service like we do it for project configs. That keeps the responsibility of the relay-config crate lower. The overall implementation in this PR does look good, so it would just have to be moved.

alright, I moved it now to the spawn handler, where I'll log an error if the file exists yet it fails to load it.

relay-server/src/actors/global_config.rs

…into tor/static_config

iker-barriocanal · 2023-09-05T15:27:56Z

relay-server/src/actors/global_config.rs

+                relay_log::info!("global config service starting");
+                // This request will trigger the request intervals when internal_rx receives the
+                // result from upstream.
+                Self::request_global_config(upstream.clone(), internal_tx.clone());


We should not make requests during initialization, as this can block relay startup. We should only do requests once the service is actually started, see my comment below in the spawn_handler.

relay-server/src/actors/global_config.rs

TBS1996 · 2023-09-06T05:35:07Z

relay-server/src/actors/global_config.rs

+                        // Start timer for making new global config request.
+                        self.schedule_fetch();


I moved the timer reset and scheduling away from the functions to make it more explicit, I think it seems a lot more clear here, lmk if any disagreements

TBS1996 · 2023-09-06T05:37:14Z

relay-server/src/actors/global_config.rs

+                (RelayMode::Proxy | RelayMode::Static | RelayMode::Capture, _, None) => {
+                    relay_log::info!(
+                        "serving default global configs due to lacking static global config file"


not a big fan of the implicit nature here, Id prefer we create the watch here, but then we'd have to pass it into handle_message and handle_upstream_result.

relay-server/src/actors/global_config.rs

iker-barriocanal

I'd merge this as-is, but I'm biased as I've also worked on this so I'll leave the approval to the team.

TBS1996 · 2023-09-11T11:17:40Z

relay-server/src/actors/global_config.rs

+                        relay_log::info!(
+                            "serving default global configs due to lacking credentials"
+                        );


doing nothing means serving default config, because we initiate the watch with default in new();

TBS1996 · 2023-09-11T11:30:04Z

relay-config/src/config.rs

+    /// Updates the credentials without creating the file, for testing purposes.
+    pub fn update_credentials(&mut self) {
+        let creds = Credentials::generate();
+        self.credentials = Some(creds);
+    }


is it possible to conditionally compile such functions only on testing? I tried the cfg(test) feature, but when i run make test-rust-all it fails to find the function, despite that it should compile in testing mode. Strangely, running the tests directly from vscode works.

I also made some attempts playing around with features and didn't manage. I'm open to updating this if somebody knows a better way.

The#[cfg(test)] does not propagate to the external crates, and only work within the current one.

It's possible to achieve this, if really really necessary with features, by e.g. adding testing feature to the crate and hiding all the functionality behind that feature. Then in the crate which has to use this testing functionality you will need to add maybe dev dependency and enable the feature.

This is now addressed by two subsequent changes:

At first, we introduced a test feature like @olksdr described above

Later, we updated the existing regenerate_credentials method and added a flag to opt out of saving to a file.

CHANGELOG.md

relay-dynamic-config/src/global.rs

jan-auer · 2023-09-11T14:02:08Z

relay-server/src/actors/global_config.rs

+                RelayMode::Managed => {
+                    if self.config.has_credentials() {


As a thought for simplifying this, what if we simply match for managed mode and catch the rest collectively?

if config.relay_mode() == RelayMode::Managed && config.has_credentials() { // ... } else { // ... }

The case that managed mode is enabled but there are no credentials is caught during startup, so that would be unreachable.

In this case, I think being explicit provides more benefits, mostly ease of understanding and being more defensive to changes.

It's harder to understand the case where relay mode is managed and there aren't credentials, besides that it requires prior knowledge of how relay startup works. We could add a comment, but I don't think that's a good alternative to avoid an if branch.

Changes to relay startup don't impact the global config service negatively.

The exhaustive match on relay modes ensures that we must consider the global config service if adding new modes to relay.

It's not a concern of the global config service to understand whether or not credentials are needed. It only needs to know that if it's in managed mode, it should make requests. Whether or not all preconditions for upstream are met to make requests is outside of its scope.

The issue with adding this here is that it duplicates validation logic or invariants across the codebase, which makes them hard to maintain or change. It creates tighter-than-necessary coupling.

jan-auer · 2023-09-11T14:05:44Z

relay-server/src/actors/global_config.rs

+        tokio::time::pause();
+
+        let (upstream, handle) = mock_service("upstream", (), |(), _| {
+            panic!();


Instead of #[should_panic], which is intended to be used for expected panics, we should rather make sure the upstream service returns a default config.

IIUC, you suggest returning a response as if it were the actual upstream service. To actually return a global config we need to do an async operation in the closure, currently unstable in Rust. I think the panic makes the test easy to understand, and removes the complexity of dealing with mock_service. Maybe I'm missing something?

I'm here with @jan-auer . The thing is with unit tests, it can also panic for completely different reason when something changes in the code, and therefore the tests will be testing completely wrong thing.

It's better to make sure the behaviour is correct, but checking the status of some operation, like in this case that resulting config is default

@TBS1996 and I spent some time on this today. In order to move forward with this PR, let's keep the should_panic for now. He'll link a follow-up issue here to clean this up.

We'll implement some utilities to make mocking upstream requests easier. The boilerplate needed to achieve this now for a single test is too complex.

olksdr · 2023-09-12T06:26:05Z

relay-config/src/config.rs

+    /// Updates the credentials without creating the file, for testing purposes.
+    pub fn update_credentials(&mut self) {
+        let creds = Credentials::generate();
+        self.credentials = Some(creds);
+    }


The#[cfg(test)] does not propagate to the external crates, and only work within the current one.

It's possible to achieve this, if really really necessary with features, by e.g. adding testing feature to the crate and hiding all the functionality behind that feature. Then in the crate which has to use this testing functionality you will need to add maybe dev dependency and enable the feature.

relay-dynamic-config/src/global.rs

olksdr · 2023-09-12T06:33:47Z

relay-server/src/actors/global_config.rs

+        tokio::time::pause();
+
+        let (upstream, handle) = mock_service("upstream", (), |(), _| {
+            panic!();


I'm here with @jan-auer . The thing is with unit tests, it can also panic for completely different reason when something changes in the code, and therefore the tests will be testing completely wrong thing.

It's better to make sure the behaviour is correct, but checking the status of some operation, like in this case that resulting config is default

olksdr · 2023-09-12T06:51:03Z

relay-server/src/actors/global_config.rs

+                    match GlobalConfig::load(self.config.path()) {
+                        Ok(Some(from_file)) => {
+                            relay_log::info!("serving static global config loaded from file");
+                            self.global_config_watch.send(Arc::new(from_file)).ok();


From the docs on subscribe

All messages sent before this call to subscribe are initially marked as seen by the new Receiver.

Can it happen, that e.g. EnvelopeProcessorService will never gets update over the watch, since it will subscribe to the watch after this static config is sent? And then we actually never send any other update?

Or maybe I'm missing something?

You're right. Every subscriber must use the initial value that the subscription has before awaiting changed(). Essentially, this would be the flow:

let subscription = global_config.send(Subscribe)?; self.config = subscription.borrow().clone(); while let Ok(()) = subscription.changed().await { self.config = subscription.borrow().clone(); }

or

let subscription = global_config.send(Subscribe)?; loop { self.config = subscription.borrow().clone(); subscription.changed().await?; }

olksdr · 2023-09-12T06:52:12Z

relay-server/src/actors/global_config.rs


            loop {
                tokio::select! {
                    biased;

-                    () = &mut self.fetch_handle => self.update_global_config(),
+                    () = &mut self.fetch_handle => self.request_global_config(),


In the static mode we actually never get this fetch_handle trigger? Is it right?

Yes, that's right. The fetch handle is only scheduled when handling a request, and a request is only made in managed modes; thus, in static mode, this fetch handle is never resolved.

…into tor/static_config

* master: fix(server): Immediatly set global config to watch contents. (#2505) feat(server): Use global config from file if provided (#2458) ref(metrics): Remove deprecated `Metric` type and improve docs (#2503) ref(projconfig): Unsupport v4 in the endpoint (#2500) docs(dashboard): Add instructions for required WASM target (#2502) release: 0.8.30

TBS1996 added 5 commits September 3, 2023 20:42

wip

dbc71f3

Merge branch 'master' into tor/static_config

a320a33

wip

dc8fe7a

wip

d631912

wip

c125ec5

TBS1996 commented Sep 4, 2023

View reviewed changes

TBS1996 added 2 commits September 4, 2023 08:03

wip

aed5e6e

wip

153c9f1

TBS1996 marked this pull request as ready for review September 4, 2023 06:33

TBS1996 requested a review from a team September 4, 2023 06:33

TBS1996 self-assigned this Sep 4, 2023

take relay modes into account

9367f89

TBS1996 commented Sep 5, 2023

View reviewed changes

relay-server/src/actors/global_config.rs Outdated Show resolved Hide resolved

TBS1996 commented Sep 5, 2023

View reviewed changes

relay-server/src/actors/global_config.rs Outdated Show resolved Hide resolved

merge

9068be4

TBS1996 requested a review from iker-barriocanal September 5, 2023 09:33

TBS1996 added 3 commits September 5, 2023 11:34

Merge branch 'master' into tor/static_config

1a43645

wip

3776ec6

Merge branch 'tor/static_config' of https://github.com/getsentry/relay …

2d56213

…into tor/static_config

iker-barriocanal suggested changes Sep 5, 2023

View reviewed changes

TBS1996 added 3 commits September 6, 2023 06:29

simplify match arms

761198e

move match to spawnhandler

c3221b7

ref

8fa4521

TBS1996 commented Sep 6, 2023

View reviewed changes

nit

387c246

TBS1996 requested review from iker-barriocanal and jjbayer September 6, 2023 05:48

TBS1996 added 2 commits September 6, 2023 07:51

self request

0fadcea

Merge branch 'master' into tor/static_config

749306a

iker-barriocanal added 2 commits September 7, 2023 18:54

Merge branch 'master' into tor/static_config

a5e9c03

update docstrings

a7030d4

iker-barriocanal requested a review from jan-auer September 7, 2023 17:25

TBS1996 commented Sep 8, 2023

View reviewed changes

relay-server/src/actors/global_config.rs Show resolved Hide resolved

TBS1996 commented Sep 8, 2023

View reviewed changes

relay-server/src/actors/global_config.rs Outdated Show resolved Hide resolved

TBS1996 requested a review from iker-barriocanal September 8, 2023 08:03

iker-barriocanal reviewed Sep 8, 2023

View reviewed changes

TBS1996 commented Sep 11, 2023

View reviewed changes

TBS1996 added 2 commits September 11, 2023 13:21

update changelog

a5c0224

Merge branch 'master' into tor/static_config

a6efd91

TBS1996 commented Sep 11, 2023

View reviewed changes

jan-auer reviewed Sep 11, 2023

View reviewed changes

address some feedback

99f2acd

iker-barriocanal requested a review from jan-auer September 11, 2023 20:37

wip

20a593e

olksdr reviewed Sep 12, 2023

View reviewed changes

TBS1996 added 6 commits September 12, 2023 10:27

Merge branch 'tor/static_config' of https://github.com/getsentry/relay …

17eba98

…into tor/static_config

wip

a6ada9f

wip

a69eb2f

t->T

a4f5a14

remove test feat/dep

c0e767d

Merge branch 'master' into tor/static_config

c9f0ef9

jan-auer approved these changes Sep 12, 2023

View reviewed changes

TBS1996 mentioned this pull request Sep 12, 2023

Watch might miss out on static global config file if subscribed after watch was updated #2504

Closed

iker-barriocanal approved these changes Sep 12, 2023

View reviewed changes

jan-auer mentioned this pull request Sep 12, 2023

fix(server): Immediatly set global config to watch contents. #2505

Merged

Merge branch 'master' into tor/static_config

c39f12c

TBS1996 merged commit 5cb3b09 into master Sep 12, 2023
17 checks passed

TBS1996 deleted the tor/static_config branch September 12, 2023 11:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(server): Use global config from file if provided #2458

feat(server): Use global config from file if provided #2458

TBS1996 commented Sep 4, 2023 •

edited

Loading

TBS1996 Sep 4, 2023

jjbayer Sep 4, 2023

jan-auer Sep 6, 2023

TBS1996 Sep 6, 2023

iker-barriocanal Sep 5, 2023

TBS1996 Sep 6, 2023

TBS1996 Sep 6, 2023

iker-barriocanal left a comment

TBS1996 Sep 11, 2023

TBS1996 Sep 11, 2023 •

edited

Loading

iker-barriocanal Sep 11, 2023

olksdr Sep 12, 2023

jan-auer Sep 12, 2023

jan-auer Sep 11, 2023 •

edited

Loading

iker-barriocanal Sep 11, 2023

jan-auer Sep 12, 2023

jan-auer Sep 11, 2023

iker-barriocanal Sep 11, 2023

olksdr Sep 12, 2023

jan-auer Sep 12, 2023

olksdr Sep 12, 2023

olksdr Sep 12, 2023

olksdr Sep 12, 2023

jan-auer Sep 12, 2023

olksdr Sep 12, 2023

iker-barriocanal Sep 12, 2023

	/// Get filename for static project config.
	pub fn project_configs_path(&self) -> PathBuf {
	self.path.join("projects")
	}

		// Start timer for making new global config request.
		self.schedule_fetch();

feat(server): Use global config from file if provided #2458

feat(server): Use global config from file if provided #2458

Conversation

TBS1996 commented Sep 4, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

iker-barriocanal left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TBS1996 Sep 11, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jan-auer Sep 11, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TBS1996 commented Sep 4, 2023 •

edited

Loading

TBS1996 Sep 11, 2023 •

edited

Loading

jan-auer Sep 11, 2023 •

edited

Loading