SAMZA-1788: Add LocationIdProvider abstraction. #585
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@xinyuiscool @sborya |
|
|
||
| import org.apache.samza.util.Util; | ||
|
|
||
| public class DefaultLocationIdProviderFactory implements LocationIdProviderFactory { |
There was a problem hiding this comment.
Consider adding java-docs here?
| */ | ||
| public interface LocationIdProvider { | ||
|
|
||
| LocationId getLocationId(Config config); |
There was a problem hiding this comment.
Prefer re-thinking these interfaces as follows:
public interface LocationIdProvider {
LocationId getLocationId();
}
and the corresponding factory interface as:
public interface LocationIdProviderFactory {
LocationIdProvider getLocationIdProvider(Config config);
}
shanthoosh
commented
Aug 4, 2018
| */ | ||
| public interface LocationIdProvider { | ||
|
|
||
| LocationId getLocationId(Config config); |
|
|
||
| import org.apache.samza.util.Util; | ||
|
|
||
| public class DefaultLocationIdProviderFactory implements LocationIdProviderFactory { |
|
Looks like there are test failures. can you take a look? Otherwise, LGTM |
vjagadish1989
approved these changes
Aug 10, 2018
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently in standalone, by default hostName of the standalone processor is used as LocationId. However, for containerized environments like azure cloud, kubernetes this defaulting does not work. Standalone processors can be launched from different kubernetes container on a physical machine(where each kubernetes container has different locatliyID than other kubernetes container within same machine).
To solve this problem, we introduce locationID abstraction to allow users to plugin a uniqueId identifying the execution environment of the processor.
In containerized environments, LocationId is a composite key of multiple fields: (sliceId, containerId, hostname) By default hostname will be used as LocationId(if not configured by the user).
All the processors of an application registered from an locationID should be able to share(read/write) their local state stores. Any custom LocationIdProvider is expected to honor this contract when generating the locationID.
This patch is part of SEP-11. Please refer to it for more details.