Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Events notification system for Nessie - Quarkus #6870

Merged
merged 14 commits into from Jun 1, 2023

Conversation

adutra
Copy link
Contributor

@adutra adutra commented May 22, 2023

This commit introduces a new module, nessie-events-quarkus. To improve isolation and facilitate testing, this new module is completely independent of other Quarkus modules; it contains:

  1. Quarkus-specific implementations of nessie-events-service classes;
  2. Asynchronous delivery based on Vert.x, both non-blocking and blocking:
    1. non-blocking executes delivery on the event loop;
    2. blocking offloads to Vert.x worker threads.
    3. In any case, delivery is never done on a Nessie thread serving HTTP requests.
  3. Delivery with optional logging, tracing and metrics.
    1. Tracing and metrics for events are enabled/disabled with the usual nessie.version.store.XYZ.enable options.
    2. Logging is for debugging purposes only, and completely disabled unless DEBUG level is enabled.
  4. Quarkus tests in this module exercise a combination of setups: auth, tracing, metrics enabled / disabled, etc.

Changes outside this module are minimal:

  1. A few cosmetic changes were introduced in nessie-events-api, nessie-events-spi and nessie-events-service.
  2. nessie-quarkus-common:
    1. VersionStoreConfig has a new isEventsEnabled() method mapped to the nessie.version.store.events.enable property, true by default;
    2. ConfigurableVersionStoreFactory changed to detect when events are enabled;
    3. New RepositoryIdProvider: creates a bean holding the repository id.
  3. nessie-quarkus (server):
    1. Compile-time dependency on nessie-events-quarkus;
    2. Smoke tests.

@Produces
@Singleton
public MeterFilter configureGlobalTags() {
return MeterFilter.commonTags(Tags.of(APPLICATION_TAG_NAME, APPLICATION_TAG_VALUE));
Copy link
Contributor Author

@adutra adutra May 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a global bean and as such, interferes with other usages of MeterRegistry throughout the application. Which is why the returned MeterFilter contains exactly the same tags used in MetricsVersionStore – this way we don't alter its behavior.

We might want to improve this later on and have a global strategy for metrics tags in Nessie.

}

@Test
void testCommitWithTracing() {
Copy link
Contributor Author

@adutra adutra May 22, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To help understand the spans used in events, here is a screenshot of Jaeger UI for the spans produced by this test:

Screenshot 2023-05-22 at 15 02 24

@adutra adutra force-pushed the events-quarkus branch 8 times, most recently from 3cbb7ad to 4891186 Compare May 24, 2023 13:20
Copy link
Member

@snazy snazy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a full review, but some higher level first-round comments.

@adutra adutra force-pushed the events-quarkus branch 2 times, most recently from aafeed5 to db893a5 Compare May 26, 2023 08:05
@adutra
Copy link
Contributor Author

adutra commented May 26, 2023

Heads up: had to rebase because of conflicts.

Copy link
Member

@snazy snazy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly LGTM - some minor comments

@snazy snazy added this to the 0.61.0 milestone May 31, 2023
adutra added 11 commits June 1, 2023 13:21
This commit introduces a new module, nessie-events-quarkus.
To improve isolation and facilitate testing, this new module
is completely independent of other Quarkus modules; it
contains:

1. Quarkus-specific implementations of nessie-events-service
   classes;
2. Asynchronous delivery based on Vert.x, both non-blocking
   and blocking;
3. Delivery with optional logging, tracing and metrics.

Changes outside this module are minimal:

1. A few changes were introduced in nessie-events-api,
   nessie-events-spi and nessie-events-service.
2. nessie-quarkus-common:
   1. ConfigurableVersionStoreFactory changed to detect when
      events are enabled;
   2. New RepositoryIdProvider.
3. nessie-quarkus (server):
   1. Compile-time dependency on nessie-events-quarkus;
   2. Smoke tests.
@adutra adutra enabled auto-merge (squash) June 1, 2023 14:01
@snazy snazy disabled auto-merge June 1, 2023 14:01
@snazy
Copy link
Member

snazy commented Jun 1, 2023

Let's merge manually to have a "nicer" commit message.

@adutra
Copy link
Contributor Author

adutra commented Jun 1, 2023

Let's merge manually to have a "nicer" commit message

I changed the commit message when I enabled auto-merge :-) – but OK, let's wait.

@adutra adutra merged commit d17a9ed into projectnessie:main Jun 1, 2023
22 checks passed
@adutra adutra deleted the events-quarkus branch June 1, 2023 14:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants