Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 32 additions & 7 deletions docs/content/en/docs/documentation/observability.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ Operator operator = new Operator(client, o -> o.withMetrics(metrics));
### Micrometer implementation

The micrometer implementation is typically created using one of the provided factory methods which, depending on which
is used, will return either a ready to use instance or a builder allowing users to customized how the implementation
is used, will return either a ready to use instance or a builder allowing users to customize how the implementation
behaves, in particular when it comes to the granularity of collected metrics. It is, for example, possible to collect
metrics on a per-resource basis via tags that are associated with meters. This is the default, historical behavior but
this will change in a future version of JOSDK because this dramatically increases the cardinality of metrics, which
Expand All @@ -62,14 +62,13 @@ instance via:

```java
MeterRegistry registry; // initialize your registry implementation
Metrics metrics = new MicrometerMetrics(registry);
Metrics metrics = MicrometerMetrics.newMicrometerMetricsBuilder(registry).build();
```

Note, however, that this constructor is deprecated and we encourage you to use the factory methods instead, which either
return a fully pre-configured instance or a builder object that will allow you to configure more easily how the instance
will behave. You can, for example, configure whether or not the implementation should collect metrics on a per-resource
basis, whether or not associated meters should be removed when a resource is deleted and how the clean-up is performed.
See the relevant classes documentation for more details.
The class provides factory methods which either return a fully pre-configured instance or a builder object that will
allow you to configure more easily how the instance will behave. You can, for example, configure whether the
implementation should collect metrics on a per-resource basis, whether associated meters should be removed when a
resource is deleted and how the clean-up is performed. See the relevant classes documentation for more details.

For example, the following will create a `MicrometerMetrics` instance configured to collect metrics on a per-resource
basis, deleting the associated meters after 5 seconds when a resource is deleted, using up to 2 threads to do so.
Expand Down Expand Up @@ -109,4 +108,30 @@ brackets (`[]`) won't be present when per-resource collection is disabled and ta
omitted if the associated value is empty. Of note, when in the context of controllers' execution metrics, these tag
names are prefixed with `resource.`. This prefix might be removed in a future version for greater consistency.

### Aggregated Metrics

The `AggregatedMetrics` class provides a way to combine multiple metrics providers into a single metrics instance using
the composite pattern. This is particularly useful when you want to simultaneously collect metrics data from different
monitoring systems or providers.

You can create an `AggregatedMetrics` instance by providing a list of existing metrics implementations:

```java
// create individual metrics instances
Metrics micrometerMetrics = MicrometerMetrics.withoutPerResourceMetrics(registry);
Metrics customMetrics = new MyCustomMetrics();
Metrics loggingMetrics = new LoggingMetrics();

// combine them into a single aggregated instance
Metrics aggregatedMetrics = new AggregatedMetrics(List.of(
micrometerMetrics,
customMetrics,
loggingMetrics
));

// use the aggregated metrics with your operator
Operator operator = new Operator(client, o -> o.withMetrics(aggregatedMetrics));
```

This approach allows you to easily combine different metrics collection strategies, such as sending metrics to both
Prometheus (via Micrometer) and a custom logging system simultaneously.
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
package io.javaoperatorsdk.operator.api.monitoring;

import java.util.List;
import java.util.Map;
import java.util.Objects;

import io.fabric8.kubernetes.api.model.HasMetadata;
import io.javaoperatorsdk.operator.api.reconciler.RetryInfo;
import io.javaoperatorsdk.operator.processing.Controller;
import io.javaoperatorsdk.operator.processing.event.Event;
import io.javaoperatorsdk.operator.processing.event.ResourceID;

/**
* An aggregated implementation of the {@link Metrics} interface that delegates method calls to a
* collection of {@link Metrics} instances using the composite pattern.
*
* <p>This class allows multiple metrics providers to be combined into a single metrics instance,
* enabling simultaneous collection of metrics data by different monitoring systems or providers.
* All method calls are delegated to each metrics instance in the list in the order they were
* provided to the constructor.
*
* <p><strong>Important:</strong> The {@link #timeControllerExecution(ControllerExecution)} method
* is handled specially - it is only invoked on the first metrics instance in the list, since it's
* not an idempotent operation and can only be executed once. The controller execution cannot be
* repeated multiple times as it would produce side effects and potentially inconsistent results.
*
* <p>All other methods are called on every metrics instance in the list, preserving the order of
* execution as specified in the constructor.
*
* @see Metrics
*/
public final class AggregatedMetrics implements Metrics {

private final List<Metrics> metricsList;

/**
* Creates a new AggregatedMetrics instance that will delegate method calls to the provided list
* of metrics instances.
*
* @param metricsList the list of metrics instances to delegate to; must not be null and must
* contain at least one metrics instance
* @throws NullPointerException if metricsList is null
* @throws IllegalArgumentException if metricsList is empty
*/
public AggregatedMetrics(List<Metrics> metricsList) {
Objects.requireNonNull(metricsList, "metricsList must not be null");
if (metricsList.isEmpty()) {
throw new IllegalArgumentException("metricsList must contain at least one Metrics instance");
}
this.metricsList = List.copyOf(metricsList);
}

@Override
public void controllerRegistered(Controller<? extends HasMetadata> controller) {
metricsList.forEach(metrics -> metrics.controllerRegistered(controller));
}

@Override
public void receivedEvent(Event event, Map<String, Object> metadata) {
metricsList.forEach(metrics -> metrics.receivedEvent(event, metadata));
}

@Override
public void reconcileCustomResource(
HasMetadata resource, RetryInfo retryInfo, Map<String, Object> metadata) {
metricsList.forEach(metrics -> metrics.reconcileCustomResource(resource, retryInfo, metadata));
}

@Override
public void failedReconciliation(
HasMetadata resource, Exception exception, Map<String, Object> metadata) {
metricsList.forEach(metrics -> metrics.failedReconciliation(resource, exception, metadata));
}

@Override
public void reconciliationExecutionStarted(HasMetadata resource, Map<String, Object> metadata) {
metricsList.forEach(metrics -> metrics.reconciliationExecutionStarted(resource, metadata));
}

@Override
public void reconciliationExecutionFinished(HasMetadata resource, Map<String, Object> metadata) {
metricsList.forEach(metrics -> metrics.reconciliationExecutionFinished(resource, metadata));
}

@Override
public void cleanupDoneFor(ResourceID resourceID, Map<String, Object> metadata) {
metricsList.forEach(metrics -> metrics.cleanupDoneFor(resourceID, metadata));
}

@Override
public void finishedReconciliation(HasMetadata resource, Map<String, Object> metadata) {
metricsList.forEach(metrics -> metrics.finishedReconciliation(resource, metadata));
}

@Override
public <T> T timeControllerExecution(ControllerExecution<T> execution) throws Exception {
return metricsList.get(0).timeControllerExecution(execution);
}

@Override
public <T extends Map<?, ?>> T monitorSizeOf(T map, String name) {
metricsList.forEach(metrics -> metrics.monitorSizeOf(map, name));
return map;
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
package io.javaoperatorsdk.operator.api.monitoring;

import java.util.List;
import java.util.Map;

import org.junit.jupiter.api.Test;

import io.fabric8.kubernetes.api.model.HasMetadata;
import io.javaoperatorsdk.operator.api.reconciler.RetryInfo;
import io.javaoperatorsdk.operator.processing.Controller;
import io.javaoperatorsdk.operator.processing.event.Event;
import io.javaoperatorsdk.operator.processing.event.ResourceID;

import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.*;

class AggregatedMetricsTest {

private final Metrics metrics1 = mock();
private final Metrics metrics2 = mock();
private final Metrics metrics3 = mock();
private final Controller<HasMetadata> controller = mock();
private final Event event = mock();
private final HasMetadata resource = mock();
private final RetryInfo retryInfo = mock();
private final ResourceID resourceID = mock();
private final Metrics.ControllerExecution<String> controllerExecution = mock();

private final Map<String, Object> metadata = Map.of("kind", "TestResource");
private final AggregatedMetrics aggregatedMetrics =
new AggregatedMetrics(List.of(metrics1, metrics2, metrics3));

@Test
void constructor_shouldThrowNullPointerExceptionWhenMetricsListIsNull() {
assertThatThrownBy(() -> new AggregatedMetrics(null))
.isInstanceOf(NullPointerException.class)
.hasMessage("metricsList must not be null");
}

@Test
void constructor_shouldThrowIllegalArgumentExceptionWhenMetricsListIsEmpty() {
assertThatThrownBy(() -> new AggregatedMetrics(List.of()))
.isInstanceOf(IllegalArgumentException.class)
.hasMessage("metricsList must contain at least one Metrics instance");
}

@Test
void controllerRegistered_shouldDelegateToAllMetricsInOrder() {
aggregatedMetrics.controllerRegistered(controller);

final var inOrder = inOrder(metrics1, metrics2, metrics3);
inOrder.verify(metrics1).controllerRegistered(controller);
inOrder.verify(metrics2).controllerRegistered(controller);
inOrder.verify(metrics3).controllerRegistered(controller);
verifyNoMoreInteractions(metrics1, metrics2, metrics3);
}

@Test
void receivedEvent_shouldDelegateToAllMetricsInOrder() {
aggregatedMetrics.receivedEvent(event, metadata);

final var inOrder = inOrder(metrics1, metrics2, metrics3);
inOrder.verify(metrics1).receivedEvent(event, metadata);
inOrder.verify(metrics2).receivedEvent(event, metadata);
inOrder.verify(metrics3).receivedEvent(event, metadata);
verifyNoMoreInteractions(metrics1, metrics2, metrics3);
}

@Test
void reconcileCustomResource_shouldDelegateToAllMetricsInOrder() {
aggregatedMetrics.reconcileCustomResource(resource, retryInfo, metadata);

final var inOrder = inOrder(metrics1, metrics2, metrics3);
inOrder.verify(metrics1).reconcileCustomResource(resource, retryInfo, metadata);
inOrder.verify(metrics2).reconcileCustomResource(resource, retryInfo, metadata);
inOrder.verify(metrics3).reconcileCustomResource(resource, retryInfo, metadata);
verifyNoMoreInteractions(metrics1, metrics2, metrics3);
}

@Test
void failedReconciliation_shouldDelegateToAllMetricsInOrder() {
final var exception = new RuntimeException("Test exception");

aggregatedMetrics.failedReconciliation(resource, exception, metadata);

final var inOrder = inOrder(metrics1, metrics2, metrics3);
inOrder.verify(metrics1).failedReconciliation(resource, exception, metadata);
inOrder.verify(metrics2).failedReconciliation(resource, exception, metadata);
inOrder.verify(metrics3).failedReconciliation(resource, exception, metadata);
verifyNoMoreInteractions(metrics1, metrics2, metrics3);
}

@Test
void reconciliationExecutionStarted_shouldDelegateToAllMetricsInOrder() {
aggregatedMetrics.reconciliationExecutionStarted(resource, metadata);

final var inOrder = inOrder(metrics1, metrics2, metrics3);
inOrder.verify(metrics1).reconciliationExecutionStarted(resource, metadata);
inOrder.verify(metrics2).reconciliationExecutionStarted(resource, metadata);
inOrder.verify(metrics3).reconciliationExecutionStarted(resource, metadata);
verifyNoMoreInteractions(metrics1, metrics2, metrics3);
}

@Test
void reconciliationExecutionFinished_shouldDelegateToAllMetricsInOrder() {
aggregatedMetrics.reconciliationExecutionFinished(resource, metadata);

final var inOrder = inOrder(metrics1, metrics2, metrics3);
inOrder.verify(metrics1).reconciliationExecutionFinished(resource, metadata);
inOrder.verify(metrics2).reconciliationExecutionFinished(resource, metadata);
inOrder.verify(metrics3).reconciliationExecutionFinished(resource, metadata);
verifyNoMoreInteractions(metrics1, metrics2, metrics3);
}

@Test
void cleanupDoneFor_shouldDelegateToAllMetricsInOrder() {
aggregatedMetrics.cleanupDoneFor(resourceID, metadata);

final var inOrder = inOrder(metrics1, metrics2, metrics3);
inOrder.verify(metrics1).cleanupDoneFor(resourceID, metadata);
inOrder.verify(metrics2).cleanupDoneFor(resourceID, metadata);
inOrder.verify(metrics3).cleanupDoneFor(resourceID, metadata);
verifyNoMoreInteractions(metrics1, metrics2, metrics3);
}

@Test
void finishedReconciliation_shouldDelegateToAllMetricsInOrder() {
aggregatedMetrics.finishedReconciliation(resource, metadata);

final var inOrder = inOrder(metrics1, metrics2, metrics3);
inOrder.verify(metrics1).finishedReconciliation(resource, metadata);
inOrder.verify(metrics2).finishedReconciliation(resource, metadata);
inOrder.verify(metrics3).finishedReconciliation(resource, metadata);
verifyNoMoreInteractions(metrics1, metrics2, metrics3);
}

@Test
void timeControllerExecution_shouldOnlyDelegateToFirstMetrics() throws Exception {
final var expectedResult = "execution result";
when(metrics1.timeControllerExecution(controllerExecution)).thenReturn(expectedResult);

final var result = aggregatedMetrics.timeControllerExecution(controllerExecution);

assertThat(result).isEqualTo(expectedResult);
verify(metrics1).timeControllerExecution(controllerExecution);
verify(metrics2, never()).timeControllerExecution(any());
verify(metrics3, never()).timeControllerExecution(any());
verifyNoMoreInteractions(metrics1, metrics2, metrics3);
}

@Test
void timeControllerExecution_shouldPropagateException() throws Exception {
final var expectedException = new RuntimeException("Controller execution failed");
when(metrics1.timeControllerExecution(controllerExecution)).thenThrow(expectedException);

assertThatThrownBy(() -> aggregatedMetrics.timeControllerExecution(controllerExecution))
.isSameAs(expectedException);

verify(metrics1).timeControllerExecution(controllerExecution);
verify(metrics2, never()).timeControllerExecution(any());
verify(metrics3, never()).timeControllerExecution(any());
verifyNoMoreInteractions(metrics1, metrics2, metrics3);
}

@Test
void monitorSizeOf_shouldDelegateToAllMetricsInOrderAndReturnOriginalMap() {
final var testMap = Map.of("key1", "value1");
final var mapName = "testMap";

final var result = aggregatedMetrics.monitorSizeOf(testMap, mapName);

assertThat(result).isSameAs(testMap);
verify(metrics1).monitorSizeOf(testMap, mapName);
verify(metrics2).monitorSizeOf(testMap, mapName);
verify(metrics3).monitorSizeOf(testMap, mapName);
verifyNoMoreInteractions(metrics1, metrics2, metrics3);
}
}