Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SelfDiscoveryResource; rename org.apache.druid.discovery.NodeType to NodeRole #6702

Merged
merged 25 commits into from Dec 8, 2019
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
d1b12d7
Add SelfDiscoveryResource
leventov Sep 25, 2018
b295155
Merge remote-tracking branch 'upstream/master' into selfDiscoveryReso…
leventov Apr 3, 2019
2239684
Rename org.apache.druid.discovery.NodeType to NodeRole. Refactor Cura…
leventov Apr 15, 2019
da617e1
Merge remote-tracking branch 'upstream/master' into selfDiscoveryReso…
leventov Apr 15, 2019
7cb1048
Extended docs
leventov Apr 15, 2019
679a303
Fix brace
leventov Apr 17, 2019
d27d0c4
Merge remote-tracking branch 'upstream/master' into selfDiscoveryReso…
leventov Apr 17, 2019
72da228
Remove redundant throws in Lifecycle.Handler.stop()
leventov Apr 17, 2019
abec2ad
Merge remote-tracking branch 'upstream/master' into selfDiscoveryReso…
leventov May 28, 2019
7f07dca
Import order
leventov May 28, 2019
4eed8d8
Remove unresolvable link
leventov May 29, 2019
bd01a6b
Address comments
leventov May 30, 2019
766288e
Merge remote-tracking branch 'upstream/master' into selfDiscoveryReso…
leventov Jun 4, 2019
f0a3c55
tmp
leventov Jun 6, 2019
21792b1
Merge remote-tracking branch 'upstream/master' into selfDiscoveryReso…
leventov Jul 16, 2019
8d02912
tmp
leventov Jul 23, 2019
e019b1c
Merge remote-tracking branch 'upstream/master' into selfDiscoveryReso…
leventov Aug 6, 2019
c4cb77b
Merge remote-tracking branch 'upstream/master' into selfDiscoveryReso…
leventov Aug 9, 2019
e4a5bb9
Merge remote-tracking branch 'upstream/master' into selfDiscoveryReso…
leventov Aug 28, 2019
6f07231
Merge remote-tracking branch 'upstream/master' into selfDiscoveryReso…
leventov Nov 22, 2019
df813e1
Rollback docker changes
leventov Nov 23, 2019
81e36f0
Remove extra .sh files
leventov Nov 23, 2019
3037e47
Merge remote-tracking branch 'upstream/master' into selfDiscoveryReso…
leventov Dec 8, 2019
3926669
Move filter
leventov Dec 8, 2019
3737cb5
Fix SecurityResourceFilterTest
leventov Dec 8, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
7 changes: 7 additions & 0 deletions docs/content/operations/api-reference.md
Expand Up @@ -54,6 +54,13 @@ An endpoint that always returns a boolean "true" value with a 200 OK response, u

Returns the current configuration properties of the node.

* `/selfDiscovered`

Returns a JSON map of the form `{"selfDiscovered": true/false}`, indicating whether the node has recieved a confirmation
from the central node discovery mechanism (currently ZooKeeper) of the Druid cluster that the node has been added to the
cluster. It is recommended to not consider a Druid node "healthy" or "ready" in automated deployment/container
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is recommended to not consider a Druid node "healthy" or "ready" in automated deployment/container
management systems until it returns {"selfDiscovered": true} from this endpoint.

Please, describe why it's not safe, probably with an example scenario.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, extended docs to describe that.

management systems until it returns `{"selfDiscovered": true}` from this endpoint.

## Coordinator

### Leadership
Expand Down
@@ -0,0 +1,117 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

package org.apache.druid.server.http;

import com.google.inject.Inject;
import com.google.inject.Singleton;
import com.sun.jersey.spi.container.ResourceFilters;
import org.apache.druid.discovery.DiscoveryDruidNode;
import org.apache.druid.discovery.DruidNodeDiscovery;
import org.apache.druid.discovery.DruidNodeDiscoveryProvider;
import org.apache.druid.discovery.NodeType;
import org.apache.druid.guice.annotations.Self;
import org.apache.druid.java.util.common.lifecycle.Lifecycle;
import org.apache.druid.server.DruidNode;
import org.apache.druid.server.http.security.StateResourceFilter;

import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
import java.util.Collection;
import java.util.Collections;

/**
* This class is annotated {@link Singleton} rather than {@link org.apache.druid.guice.LazySingleton}, because it adds
* a lifecycle handler in the constructor, that should happen before the lifecycle is started, i. e. eagerly during the
* DI configuration phase.
*/
@Singleton
@Path("/selfDiscovered")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is a kind of status/health check, could consider putting the endpoint under status/selfDiscovered to go with status and status/health

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed to status/selfDiscoveredStatus

@ResourceFilters(StateResourceFilter.class)
public class SelfDiscoveryResource
{
private boolean selfDiscovered = false;

@Inject
public SelfDiscoveryResource(
@Self DruidNode thisDruidNode,
@Self NodeType thisNodeType,
DruidNodeDiscoveryProvider nodeDiscoveryProvider,
Lifecycle lifecycle
)
{
Lifecycle.Handler selfDiscoveryListenerRegistrator = new Lifecycle.Handler()
{
@Override
public void start()
{
registerSelfDiscoveryListener(thisDruidNode, thisNodeType, nodeDiscoveryProvider);
}

@Override
public void stop()
{
// do nothing
}
};
// Using Lifecycle.Stage.LAST because DruidNodeDiscoveryProvider should be already started when
// registerSelfDiscoveryListener() is called.
lifecycle.addHandler(selfDiscoveryListenerRegistrator, Lifecycle.Stage.LAST);
}

private void registerSelfDiscoveryListener(
DruidNode thisDruidNode,
NodeType thisNodeType,
DruidNodeDiscoveryProvider nodeDiscoveryProvider
)
{
nodeDiscoveryProvider.getForNodeType(thisNodeType).registerListener(new DruidNodeDiscovery.Listener()
{
@Override
public void nodesAdded(Collection<DiscoveryDruidNode> nodes)
{
if (selfDiscovered) {
return;
}
for (DiscoveryDruidNode node : nodes) {
if (node.getDruidNode().equals(thisDruidNode)) {
selfDiscovered = true;
break;
}
}
}

@Override
public void nodesRemoved(Collection<DiscoveryDruidNode> nodes)
{
// do nothing
}
});
}

@GET
@Produces(MediaType.APPLICATION_JSON)
public Response getSelfDiscovered()
{
return Response.ok(Collections.singletonMap("selfDiscovered", selfDiscovered)).build();
}
}
Expand Up @@ -47,9 +47,7 @@
public class StateResourceFilter extends AbstractResourceFilter
{
@Inject
public StateResourceFilter(
AuthorizerMapper authorizerMapper
)
public StateResourceFilter(AuthorizerMapper authorizerMapper)
{
super(authorizerMapper);
}
Expand Down
Expand Up @@ -35,6 +35,7 @@
import org.apache.druid.server.http.IntervalsResource;
import org.apache.druid.server.http.MetadataResource;
import org.apache.druid.server.http.RulesResource;
import org.apache.druid.server.http.SelfDiscoveryResource;
import org.apache.druid.server.http.ServersResource;
import org.apache.druid.server.http.TiersResource;
import org.apache.druid.server.security.ForbiddenException;
Expand All @@ -46,13 +47,10 @@
import org.junit.runners.Parameterized;

import java.util.Collection;
import java.util.regex.Pattern;

@RunWith(Parameterized.class)
public class SecurityResourceFilterTest extends ResourceFilterTestHelper
{
private static final Pattern WORD = Pattern.compile("\\w+");

@Parameterized.Parameters
public static Collection<Object[]> data()
{
Expand All @@ -71,6 +69,7 @@ public static Collection<Object[]> data()
getRequestPathsWithAuthorizer(CoordinatorDynamicConfigsResource.class),
getRequestPathsWithAuthorizer(QueryResource.class),
getRequestPathsWithAuthorizer(StatusResource.class),
getRequestPathsWithAuthorizer(SelfDiscoveryResource.class),
getRequestPathsWithAuthorizer(BrokerQueryResource.class)
)
);
Expand Down
7 changes: 7 additions & 0 deletions services/src/main/java/org/apache/druid/cli/CliBroker.java
Expand Up @@ -44,6 +44,7 @@
import org.apache.druid.guice.LifecycleModule;
import org.apache.druid.guice.QueryRunnerFactoryModule;
import org.apache.druid.guice.QueryableModule;
import org.apache.druid.guice.annotations.Self;
import org.apache.druid.java.util.common.logger.Logger;
import org.apache.druid.query.QuerySegmentWalker;
import org.apache.druid.query.RetryQueryRunnerConfig;
Expand All @@ -53,6 +54,7 @@
import org.apache.druid.server.ClientQuerySegmentWalker;
import org.apache.druid.server.coordination.broker.DruidBroker;
import org.apache.druid.server.http.BrokerResource;
import org.apache.druid.server.http.SelfDiscoveryResource;
import org.apache.druid.server.initialization.jetty.JettyServerInitializer;
import org.apache.druid.server.metrics.MetricsModule;
import org.apache.druid.server.metrics.QueryCountStatsProvider;
Expand Down Expand Up @@ -125,11 +127,16 @@ protected List<? extends Module> getModules()

LifecycleModule.register(binder, Server.class);

binder.bind(NodeType.class).annotatedWith(Self.class).toInstance(NodeType.BROKER);

binder
.bind(DiscoverySideEffectsProvider.Child.class)
.toProvider(new DiscoverySideEffectsProvider(NodeType.BROKER, ImmutableList.of(LookupNodeService.class)))
.in(LazySingleton.class);
LifecycleModule.registerKey(binder, Key.get(DiscoverySideEffectsProvider.Child.class));

Jerseys.addResource(binder, SelfDiscoveryResource.class);
LifecycleModule.registerKey(binder, Key.get(SelfDiscoveryResource.class));
},
new LookupModule(),
new SqlModule()
Expand Down
Expand Up @@ -46,6 +46,7 @@
import org.apache.druid.guice.ManageLifecycle;
import org.apache.druid.guice.annotations.CoordinatorIndexingServiceHelper;
import org.apache.druid.guice.annotations.EscalatedGlobal;
import org.apache.druid.guice.annotations.Self;
import org.apache.druid.guice.http.JettyHttpClientModule;
import org.apache.druid.java.util.common.concurrent.ScheduledExecutorFactory;
import org.apache.druid.java.util.common.logger.Logger;
Expand Down Expand Up @@ -79,6 +80,7 @@
import org.apache.druid.server.http.RedirectFilter;
import org.apache.druid.server.http.RedirectInfo;
import org.apache.druid.server.http.RulesResource;
import org.apache.druid.server.http.SelfDiscoveryResource;
import org.apache.druid.server.http.ServersResource;
import org.apache.druid.server.http.TiersResource;
import org.apache.druid.server.initialization.ZkPathsConfig;
Expand Down Expand Up @@ -217,12 +219,17 @@ public void configure(Binder binder)
DruidCoordinatorCleanupPendingSegments.class
);

binder.bind(NodeType.class).annotatedWith(Self.class).toInstance(NodeType.COORDINATOR);

binder
.bind(DiscoverySideEffectsProvider.Child.class)
.annotatedWith(Coordinator.class)
.toProvider(new DiscoverySideEffectsProvider(NodeType.COORDINATOR, ImmutableList.of()))
.in(LazySingleton.class);
LifecycleModule.registerKey(binder, Key.get(DiscoverySideEffectsProvider.Child.class, Coordinator.class));

Jerseys.addResource(binder, SelfDiscoveryResource.class);
LifecycleModule.registerKey(binder, Key.get(SelfDiscoveryResource.class));
}

@Provides
Expand Down
Expand Up @@ -39,6 +39,7 @@
import org.apache.druid.guice.NodeTypeConfig;
import org.apache.druid.guice.QueryRunnerFactoryModule;
import org.apache.druid.guice.QueryableModule;
import org.apache.druid.guice.annotations.Self;
import org.apache.druid.java.util.common.logger.Logger;
import org.apache.druid.query.QuerySegmentWalker;
import org.apache.druid.query.lookup.LookupModule;
Expand All @@ -49,6 +50,7 @@
import org.apache.druid.server.coordination.ZkCoordinator;
import org.apache.druid.server.http.HistoricalResource;
import org.apache.druid.server.http.SegmentListerResource;
import org.apache.druid.server.http.SelfDiscoveryResource;
import org.apache.druid.server.initialization.jetty.JettyServerInitializer;
import org.apache.druid.server.metrics.MetricsModule;
import org.apache.druid.server.metrics.QueryCountStatsProvider;
Expand Down Expand Up @@ -103,6 +105,8 @@ protected List<? extends Module> getModules()
binder.install(new CacheModule());
MetricsModule.register(binder, CacheMonitor.class);

binder.bind(NodeType.class).annotatedWith(Self.class).toInstance(NodeType.HISTORICAL);

binder
.bind(DiscoverySideEffectsProvider.Child.class)
.toProvider(
Expand All @@ -113,6 +117,9 @@ protected List<? extends Module> getModules()
)
.in(LazySingleton.class);
LifecycleModule.registerKey(binder, Key.get(DiscoverySideEffectsProvider.Child.class));

Jerseys.addResource(binder, SelfDiscoveryResource.class);
LifecycleModule.registerKey(binder, Key.get(SelfDiscoveryResource.class));
},
new LookupModule()
);
Expand Down
Expand Up @@ -58,6 +58,7 @@
import org.apache.druid.java.util.common.logger.Logger;
import org.apache.druid.segment.realtime.firehose.ChatHandlerProvider;
import org.apache.druid.server.DruidNode;
import org.apache.druid.server.http.SelfDiscoveryResource;
import org.apache.druid.server.initialization.jetty.JettyServerInitializer;
import org.eclipse.jetty.server.Server;

Expand Down Expand Up @@ -130,13 +131,18 @@ public void configure(Binder binder)

LifecycleModule.register(binder, Server.class);

binder.bind(NodeType.class).annotatedWith(Self.class).toInstance(NodeType.MIDDLE_MANAGER);

binder
.bind(DiscoverySideEffectsProvider.Child.class)
.toProvider(
new DiscoverySideEffectsProvider(NodeType.MIDDLE_MANAGER, ImmutableList.of(WorkerNodeService.class))
)
.in(LazySingleton.class);
LifecycleModule.registerKey(binder, Key.get(DiscoverySideEffectsProvider.Child.class));

Jerseys.addResource(binder, SelfDiscoveryResource.class);
LifecycleModule.registerKey(binder, Key.get(SelfDiscoveryResource.class));
}

@Provides
Expand Down
10 changes: 9 additions & 1 deletion services/src/main/java/org/apache/druid/cli/CliOverlord.java
Expand Up @@ -51,6 +51,7 @@
import org.apache.druid.guice.ManageLifecycle;
import org.apache.druid.guice.PolyBind;
import org.apache.druid.guice.annotations.Json;
import org.apache.druid.guice.annotations.Self;
import org.apache.druid.indexing.common.actions.LocalTaskActionClientFactory;
import org.apache.druid.indexing.common.actions.TaskActionClientFactory;
import org.apache.druid.indexing.common.actions.TaskActionToolbox;
Expand Down Expand Up @@ -97,6 +98,7 @@
import org.apache.druid.server.coordinator.CoordinatorOverlordServiceConfig;
import org.apache.druid.server.http.RedirectFilter;
import org.apache.druid.server.http.RedirectInfo;
import org.apache.druid.server.http.SelfDiscoveryResource;
import org.apache.druid.server.initialization.ServerConfig;
import org.apache.druid.server.initialization.jetty.JettyServerInitUtils;
import org.apache.druid.server.initialization.jetty.JettyServerInitializer;
Expand Down Expand Up @@ -235,12 +237,18 @@ public void configure(Binder binder)
LifecycleModule.register(binder, Server.class);
}

binder.bind(NodeType.class).annotatedWith(Self.class).toInstance(NodeType.OVERLORD);

binder
.bind(DiscoverySideEffectsProvider.Child.class)
.annotatedWith(IndexingService.class)
.toProvider(new DiscoverySideEffectsProvider(NodeType.OVERLORD, ImmutableList.of()))
.in(LazySingleton.class);
LifecycleModule.registerKey(binder, Key.get(DiscoverySideEffectsProvider.Child.class, IndexingService.class));
LifecycleModule
.registerKey(binder, Key.get(DiscoverySideEffectsProvider.Child.class, IndexingService.class));

Jerseys.addResource(binder, SelfDiscoveryResource.class);
LifecycleModule.registerKey(binder, Key.get(SelfDiscoveryResource.class));
}

private void configureTaskStorage(Binder binder)
Expand Down
14 changes: 7 additions & 7 deletions services/src/main/java/org/apache/druid/cli/CliPeon.java
Expand Up @@ -39,6 +39,7 @@
import org.apache.druid.client.coordinator.CoordinatorClient;
import org.apache.druid.client.indexing.HttpIndexingServiceClient;
import org.apache.druid.client.indexing.IndexingServiceClient;
import org.apache.druid.discovery.NodeType;
import org.apache.druid.guice.Binders;
import org.apache.druid.guice.CacheModule;
import org.apache.druid.guice.DruidProcessingModule;
Expand All @@ -54,6 +55,7 @@
import org.apache.druid.guice.QueryableModule;
import org.apache.druid.guice.QueryablePeonModule;
import org.apache.druid.guice.annotations.Json;
import org.apache.druid.guice.annotations.Self;
import org.apache.druid.guice.annotations.Smile;
import org.apache.druid.indexing.common.RetryPolicyConfig;
import org.apache.druid.indexing.common.RetryPolicyFactory;
Expand Down Expand Up @@ -118,7 +120,8 @@
@Command(
name = "peon",
description = "Runs a Peon, this is an individual forked \"task\" used as part of the indexing service. "
+ "This should rarely, if ever, be used directly. See http://druid.io/docs/latest/design/peons.html for a description"
+ "This should rarely, if ever, be used directly. See http://druid.io/docs/latest/design/peons.html"
+ " for a description"
)
public class CliPeon extends GuiceRunnable
{
Expand Down Expand Up @@ -293,6 +296,8 @@ private void configureTaskActionClient(Binder binder)
.addBinding("remote")
.to(RemoteTaskActionClientFactory.class)
.in(LazySingleton.class);

binder.bind(NodeType.class).annotatedWith(Self.class).toInstance(NodeType.PEON);
}

@Provides
Expand Down Expand Up @@ -330,12 +335,7 @@ public SegmentListerResource getSegmentListerResource(
@Nullable BatchDataSegmentAnnouncer announcer
)
{
return new SegmentListerResource(
jsonMapper,
smileMapper,
announcer,
null
);
return new SegmentListerResource(jsonMapper, smileMapper, announcer, null);
}
},
new QueryablePeonModule(),
Expand Down