Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement auto-detection #41

Merged
merged 10 commits into from Aug 12, 2020
Merged

Implement auto-detection #41

merged 10 commits into from Aug 12, 2020

Conversation

mtyazici
Copy link
Contributor

@mtyazici mtyazici commented Aug 4, 2020

No description provided.

@mtyazici mtyazici requested a review from leszko August 4, 2020 13:27
@mtyazici mtyazici requested a review from a team as a code owner August 4, 2020 13:27
@mtyazici mtyazici requested review from pivovarit and removed request for a team August 4, 2020 13:27
@devOpsHazelcast
Copy link
Contributor

devOpsHazelcast commented Aug 4, 2020

CLA assistant check
All committers have signed the CLA.

Copy link

@leszko leszko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a few comments.

You can also check the AWS PR for a reference for tests/javadocs, etc.
https://dzone.com/articles/the-stepdown-rule

return DiscoveryStrategyLevel.CLOUD_VM;
}


Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add some unit tests?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure!

}


private String readFileContents(String fileName) {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider changing this method and others to static

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed it.

}
}

private boolean googleInternalDnsConfigured() {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please reorder the methods. They should follow Stepdown rule

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did it following the rule.

@mtyazici mtyazici marked this pull request as draft August 5, 2020 08:09
@mtyazici mtyazici marked this pull request as ready for review August 5, 2020 09:06
Co-authored-by: Rafał Leszko <rafal@hazelcast.com>
Copy link

@leszko leszko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added two comments.

}

private static boolean metadataFlavorGoogle() {
return isEndpointAvailable("metadata.google.internal");
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This part does not work for me. I get the following exception.

com.hazelcast.gcp.RestClientException: Failure in executing REST call
        at com.hazelcast.gcp.RestClient.call(RestClient.java:98)
        at com.hazelcast.gcp.RestClient.get(RestClient.java:61)
        at com.hazelcast.gcp.GcpDiscoveryStrategyFactory.isEndpointAvailable(GcpDiscoveryStrategyFactory.java:135)
        at com.hazelcast.gcp.GcpDiscoveryStrategyFactory.metadataFlavorGoogle(GcpDiscoveryStrategyFactory.java:113)
        at com.hazelcast.gcp.GcpDiscoveryStrategyFactory.isAutoDetectionApplicable(GcpDiscoveryStrategyFactory.java:78)
        at com.hazelcast.spi.discovery.impl.DefaultDiscoveryService.detectDiscoveryStrategyFactory(DefaultDiscoveryService.java:207)
        at com.hazelcast.spi.discovery.impl.DefaultDiscoveryService.loadDiscoveryStrategies(DefaultDiscoveryService.java:147)
        at com.hazelcast.spi.discovery.impl.DefaultDiscoveryService.<init>(DefaultDiscoveryService.java:58)
        at com.hazelcast.spi.discovery.impl.DefaultDiscoveryServiceProvider.newDiscoveryService(DefaultDiscoveryServiceProvider.java:29)
        at com.hazelcast.instance.impl.Node.createDiscoveryService(Node.java:339)
        at com.hazelcast.instance.impl.Node.<init>(Node.java:265)
        at com.hazelcast.instance.impl.HazelcastInstanceImpl.createNode(HazelcastInstanceImpl.java:148)
        at com.hazelcast.instance.impl.HazelcastInstanceImpl.<init>(HazelcastInstanceImpl.java:117)
        at com.hazelcast.instance.impl.HazelcastInstanceFactory.constructHazelcastInstance(HazelcastInstanceFactory.java:211)
        at com.hazelcast.instance.impl.HazelcastInstanceFactory.newHazelcastInstance(HazelcastInstanceFactory.java:190)
        at com.hazelcast.instance.impl.HazelcastInstanceFactory.newHazelcastInstance(HazelcastInstanceFactory.java:128)
        at com.hazelcast.core.Hazelcast.newHazelcastInstance(Hazelcast.java:91)
        at com.hazelcast.core.server.HazelcastMemberStarter.main(HazelcastMemberStarter.java:47)
Caused by: java.net.MalformedURLException: no protocol: metadata.google.internal
        at java.base/java.net.URL.<init>(URL.java:645)
        at java.base/java.net.URL.<init>(URL.java:541)
        at java.base/java.net.URL.<init>(URL.java:488)
        at com.hazelcast.gcp.RestClient.call(RestClient.java:72)

Could you double check?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you tell me what you ran for this to happen?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I just created a new GCP VM, copied there Hazelcast and Hazelcast Kubernetes plugin Jars and executed:

java -cp hazelcast-kubernetes.jar:hazelcast.jar com.hazelcast.core.server.HazelcastMemberStarter

Also you need to enabled debug logging or change the log level in the code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean gcp plug-ins?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, sorry, I meant hazelcast-gcp.jar

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You were right, because I used curl while testing I forgot to add "http://" to the code.


private static boolean serviceAccountAttached() {
try {
return isEndpointAvailable("metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/");
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this part is not correct. What is someone does not use default service account? Or if someone does not have persmission to check the service accounts?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If they have auto-configuration enabled, I assume they don't provide private-key-path. In that case only way to get token for compute api is through a service account. If they don't have any service accounts then GCP discovery won't work.

I don't understand what you mean by "if they don't have permissions to check service accounts"? As far as I know if they are on a GCP VM they can use instance metadata api without need for a permission, because it is only provided for that instance.

For the default service account, in "GcpMetadataApi" class we use default service account to get the token, so if they don't have a default one, discovery api wouldn't work nevertheless.

I am not confident about my findings though I am not familiar with gcp.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you define a different service account while creating a GCP VM, will it still work?
image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than rafal? If the service account has permissions for reading project and region data, then I think it would work.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you checked that it works with other service account, then it's fine.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will check it again.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It works correctly, I tried it with two different service accounts

Copy link

@leszko leszko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍 Thanks for all the changes @mtyazici !

@mtyazici mtyazici merged commit bdb845a into master Aug 12, 2020
@mtyazici mtyazici deleted the auto-config branch August 12, 2020 11:53
@leszko leszko added this to the 2.1 milestone Aug 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants