Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BROOKLYN-259: Fix JcloudsByonLocationResolver #132

Merged
merged 2 commits into from
Jun 1, 2016

Conversation

aledsage
Copy link
Contributor

@aledsage aledsage commented May 3, 2016

  • Previously, it created locations every time the resolver was called
    (even if that was just to query about what it would produce), which
    caused us to leak locations.
  • Now it defers creating the locations (though doing this with
    LocationSpec is not practical because of the relationship between
    the JcloudsLocation and the machines)!
  • Adds FixedListMachineProvisioningLocation.initialMachinesFactory,
    for deferring the machine-lookup until the byon location is really
    being created.

@aledsage aledsage changed the title Fix JcloudsByonLocationResolver BROOKLYN-259: Fix JcloudsByonLocationResolver May 3, 2016
@aledsage aledsage force-pushed the fix/JcloudsByonLocationResolver branch from bae0726 to 66be0cb Compare May 3, 2016 16:36
public static final ConfigKey<Supplier<? extends List<? extends MachineLocation>>> INITIAL_MACHINES_FACTORY = ConfigKeys.newConfigKey(
new TypeToken<Supplier<? extends List<? extends MachineLocation>>>() {},
"byon.initialMachinesFactory",
"Factory for creating the machines that should be immediatly instantiated on init",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

immediatly

@sjcorbett
Copy link
Contributor

Looks good, only a few minor comments. Believe @johnmccabe is testing this with a deployment.

@johnmccabe
Copy link
Contributor

johnmccabe commented May 9, 2016

Throwing an NPE when deploying an app to a JcloudsByonLocation that was working before this PR:

2016-05-09 14:30:30,074 WARN  o.a.b.c.m.i.LocalLocationManager [brooklyn-execmanager-K1kS1uK9-466]: Failed to create location using spec LocationSpec{type=class org.apache.brooklyn.location.byon.FixedListMachineProvisioningLocation}@4c055676 (rethrowing)
java.lang.NullPointerException: null
    at org.apache.brooklyn.location.jclouds.JcloudsLocation.createJcloudsSshMachineLocation(JcloudsLocation.java:2319) ~[brooklyn-locations-jclouds-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
    at org.apache.brooklyn.location.jclouds.JcloudsLocation.registerJcloudsSshMachineLocation(JcloudsLocation.java:2255) ~[brooklyn-locations-jclouds-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
    at org.apache.brooklyn.location.jclouds.JcloudsLocation.registerMachineLocation(JcloudsLocation.java:2106) ~[brooklyn-locations-jclouds-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
    at org.apache.brooklyn.location.jclouds.JcloudsLocation.registerMachine(JcloudsLocation.java:2097) ~[brooklyn-locations-jclouds-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
    at org.apache.brooklyn.location.jclouds.JcloudsByonLocationResolver$1.get(JcloudsByonLocationResolver.java:156) ~[brooklyn-locations-jclouds-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
    at org.apache.brooklyn.location.jclouds.JcloudsByonLocationResolver$1.get(JcloudsByonLocationResolver.java:147) ~[brooklyn-locations-jclouds-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
    at org.apache.brooklyn.location.byon.FixedListMachineProvisioningLocation.init(FixedListMachineProvisioningLocation.java:173) ~[brooklyn-core-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
    at org.apache.brooklyn.core.objs.proxy.InternalLocationFactory.createLocation(InternalLocationFactory.java:137) ~[brooklyn-core-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
    at org.apache.brooklyn.core.mgmt.internal.LocalLocationManager.createLocation(LocalLocationManager.java:94) ~[brooklyn-core-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
    at org.apache.brooklyn.core.objs.proxy.InternalEntityFactory$1.run(InternalEntityFactory.java:331) [brooklyn-core-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_99]
    at org.apache.brooklyn.util.core.task.BasicExecutionManager$SubmissionCallable.call(BasicExecutionManager.java:519) [brooklyn-core-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
    at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_99]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_99]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_99]
    at java.lang.Thread.run(Thread.java:745) [na:1.7.0_99]

Example location:

brooklyn.location.named.my-named-byon=jcloudsByon:(provider="cloudstack",endpoint="xxxxxxxxxxxxxx",region="xxxxxxxxxxxxxx",networkName="xxxxxxxxxxxxxx",user="root",password="xxxxxxxxxxxxxx",hosts="xxxxxxxxxxxxxx,yyyyyyyyyyyyy,zzzzzzzzzzzzz")
brooklyn.location.named.bt-beta-byon.identity=xxxxxxx
brooklyn.location.named.bt-beta-byon.credential=xxxxxxx

Not clear why its not getting at the creds yet.

assertEquals(machine.getPrivateAddresses(), ImmutableSet.of(nodePrivateAddress));

// TODO what user/password should we expect? Fails below because has "dummy":
// assertEquals(machine.getUser(), "myuser");
Copy link
Contributor

@johnmccabe johnmccabe May 9, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is whats throwing the NPE mentioned earlier (ie machine.getUser() returns null, previously it was these user/password values that were being used.

@johnmccabe
Copy link
Contributor

johnmccabe commented May 9, 2016

Looks like LocationConfigUtils.getOsCredential isn't overriding the null credentials returned in the NodeMetadata in JcloudsLocation.findNodeOrThrow. These should be getting replaced with the user/password set in the ConfigBag.

Theres no support for password auth currently, switching to privateKeyFile works as expected.

@johnmccabe
Copy link
Contributor

Spoke too soon on the location working with privateKey, throws an NPE when deploying a docker infrastructure via Clocker:

Caused by: java.lang.NullPointerException: id
        at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:229) ~[guava-16.0.1.jar:na]
        at org.jclouds.compute.internal.BaseComputeService.getImage(BaseComputeService.java:410) ~[jclouds-compute-1.9.2.jar:1.9.2]
        at org.apache.brooklyn.location.jclouds.JcloudsSshMachineLocation.getOptionalImage(JcloudsSshMachineLocation.java:226) ~[brooklyn-locations-jclouds-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
        at org.apache.brooklyn.location.jclouds.JcloudsSshMachineLocation.getOptionalOperatingSystem(JcloudsSshMachineLocation.java:504) ~[brooklyn-locations-jclouds-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
        at org.apache.brooklyn.location.jclouds.JcloudsSshMachineLocation.inferMachineDetails(JcloudsSshMachineLocation.java:522) ~[brooklyn-locations-jclouds-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
        at org.apache.brooklyn.location.ssh.SshMachineLocation.getMachineDetails(SshMachineLocation.java:1056) ~[brooklyn-core-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
        at clocker.docker.entity.DockerHostSshDriver.install(DockerHostSshDriver.java:183) ~[brooklyn-clocker-docker-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
        at org.apache.brooklyn.entity.software.base.AbstractSoftwareProcessDriver$2$6.run(AbstractSoftwareProcessDriver.java:159) ~[brooklyn-software-base-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[na:1.7.0_99]
        at org.apache.brooklyn.util.core.task.DynamicSequentialTask$DstJob.call(DynamicSequentialTask.java:359) ~[brooklyn-core-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
        ... 5 common frames omitted

@ahgittin
Copy link
Contributor

@aledsage important comments from @johnmccabe need addressing before can merge

- Previously, it created locations every time the resolver was called
  (even if that was just to query about what it would produce), which
  caused us to leak locations.
- Now it defers creating the locations (though doing this with
  LocationSpec is not practical because of the relationship between
  the JcloudsLocation and the machines)!
- Adds FixedListMachineProvisioningLocation.initialMachinesFactory,
  for deferring the machine-lookup until the byon location is really
  being created.
@aledsage aledsage force-pushed the fix/JcloudsByonLocationResolver branch from 66be0cb to a927f66 Compare May 19, 2016 23:19
@aledsage
Copy link
Contributor Author

Rebasing fixed the problem in my test where assertEquals(machine.getUser(), "myuser"); was failing (so that is now uncommented). I hope that whatever fixed that issue will have fixed @johnmccabe 's "NPE when deploying an app to a JcloudsByonLocation".

For "NPE when deploying a docker infrastructure via Clocker", that has been fixed in a separate PR by the looks of it (certainly we guard against that NPE). See commit bee832d.

I've confirmed that it works with username using JcloudsByonLocationResolverAwsLiveTest (though I needed to fix another couple of things there - see second commit).

@johnmccabe can you review / test again with this latest code when you get a chance?

@johnmccabe
Copy link
Contributor

@aledsage testing this at the moment

@aledsage
Copy link
Contributor Author

@johnmccabe did you get a chance to test this? Any comments? Can we merge it?

@johnmccabe
Copy link
Contributor

Hi @aledsage, sorry for the delay getting back to you, the env I had originally observed this on is still inaccessible so I had to recreate, unfortunately, still seeing an NPE when attempting to deploy to the following location (userCredentials.getUser() again):

brooklyn.location.named.my-jclouds-byon=jcloudsByon:(provider="cloudstack",endpoint="https://my-cloudstack-endpoint.domain.com/client/api",region="my-region-id",networkName="my-network-id",user="root",password="supersecretrootpassword",hosts="my-existing-host-id")
brooklyn.location.named.my-jclouds-byon.identity=MY_API_KEY
brooklyn.location.named.my-jclouds-byon.credential=MY_SECRET_KEY
2016-06-01 11:30:04,141 WARN  o.a.b.c.m.i.LocalLocationManager [brooklyn-execmanager-qQtAY4JU-1]: Failed to create location using spec LocationSpec{type=class org.apache.brooklyn.location.byon.FixedListMachineProvisioningLocation}@33a9098e (rethrowing)
java.lang.NullPointerException: null
        at org.apache.brooklyn.location.jclouds.JcloudsLocation.createJcloudsSshMachineLocation(JcloudsLocation.java:2319) ~[brooklyn-locations-jclouds-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
        at org.apache.brooklyn.location.jclouds.JcloudsLocation.registerJcloudsSshMachineLocation(JcloudsLocation.java:2255) ~[brooklyn-locations-jclouds-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
        at org.apache.brooklyn.location.jclouds.JcloudsLocation.registerMachineLocation(JcloudsLocation.java:2106) ~[brooklyn-locations-jclouds-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
        at org.apache.brooklyn.location.jclouds.JcloudsLocation.registerMachine(JcloudsLocation.java:2097) ~[brooklyn-locations-jclouds-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
        at org.apache.brooklyn.location.jclouds.JcloudsByonLocationResolver$1.get(JcloudsByonLocationResolver.java:156) ~[brooklyn-locations-jclouds-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
        at org.apache.brooklyn.location.jclouds.JcloudsByonLocationResolver$1.get(JcloudsByonLocationResolver.java:147) ~[brooklyn-locations-jclouds-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
        at org.apache.brooklyn.location.byon.FixedListMachineProvisioningLocation.init(FixedListMachineProvisioningLocation.java:173) ~[brooklyn-core-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
        at org.apache.brooklyn.core.objs.proxy.InternalLocationFactory.createLocation(InternalLocationFactory.java:137) ~[brooklyn-core-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
        at org.apache.brooklyn.core.mgmt.internal.LocalLocationManager.createLocation(LocalLocationManager.java:94) ~[brooklyn-core-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
        at org.apache.brooklyn.core.objs.proxy.InternalEntityFactory$1.run(InternalEntityFactory.java:331) [brooklyn-core-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_79]
        at org.apache.brooklyn.util.core.task.BasicExecutionManager$SubmissionCallable.call(BasicExecutionManager.java:519) [brooklyn-core-0.10.0-SNAPSHOT.jar:0.10.0-SNAPSHOT]
        at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_79]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_79]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_79]
        at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]

I'm going to rebuild to sanity check my install (I've built clocker against the #132 brooklyn-server branch).

@johnmccabe
Copy link
Contributor

Rebuild appears to have done the trick, in the process of deploying a simple test app.

location: my-jclouds-byon
services:
- type: org.apache.brooklyn.entity.webapp.tomcat.TomcatServer
  id: tomcat
  name: Tomcat
  war: http://tomcat.apache.org/tomcat-6.0-doc/appdev/sample/sample.war

@johnmccabe
Copy link
Contributor

LGTM, thanks @aledsage

@aledsage
Copy link
Contributor Author

aledsage commented Jun 1, 2016

Great, thanks @johnmccabe - merging now!

@asfgit asfgit merged commit a927f66 into apache:master Jun 1, 2016
asfgit pushed a commit that referenced this pull request Jun 1, 2016
@aledsage aledsage deleted the fix/JcloudsByonLocationResolver branch June 1, 2016 14:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants