Fix AWS error: InvalidPlacementGroup.Unknown #26

Merged
merged 1 commit into from Mar 23, 2013

Projects

None yet

2 participants

@guarly
Contributor
guarly commented Feb 11, 2013

AWS's call to AuthorizeSecurityGroupIngress doesn't
like being called specifying the same UserID as the
current user.

@guarly guarly Fix AWS error: InvalidPlacementGroup.Unknown
AWS's call to AuthorizeSecurityGroupIngress doesn't
like being called specifying the same UserID as the
current user.
ec9c125
@jasonjckn
Collaborator

authorize-group is a variadic function, and we only call the 4 arg version, so this diff has no effect.

@jasonjckn jasonjckn closed this Feb 15, 2013
@jasonjckn
Collaborator

nvm, it's called.

@jasonjckn jasonjckn reopened this Feb 15, 2013
@jasonjckn
Collaborator

Can you point to something in AWS documentation?
Are you getting a warning, or failure?

http://www.jarvana.com/jarvana/view/com/amazonaws/aws-java-sdk/1.0.10/aws-java-sdk-1.0.10-javadoc.jar!/com/amazonaws/services/ec2/model/UserIdGroupPair.html It looks like it accepts the current AWS user id.

@guarly
Contributor
guarly commented Feb 15, 2013

Failure 100% everytime I launch "lein -start" at the same line. FYI, I run storm-deploy with no changes: Amazon US-East Region, with default options in conf/clusters.yaml

The API Request to AWS requesting to add the supervisor/zookeeper groups to the nimbus security group fails consistently with an "InvalidPlacementGroup.Unknown" error, due to the fact that the API AuthorizeSecurityGroupIngress doesn't like telling a SourceSecurityGroupOwnerId when it's yourself. The SourceSecurityGroupOwnerId parameter is used to allow different users in AWS to create ingress/egress security rules between them, which is cool, but that parameter chokes when it's yourself.

The documentation always implies that you are autorizing third party IDs (and thus makes it a conditional "required" parameter): http://docs.aws.amazon.com/AWSEC2/latest/APIReference/ApiReference-query-AuthorizeSecurityGroupIngress.html
FYI, I've tried the "old" parameters:
http://docs.aws.amazon.com/AWSEC2/2009-10-31/APIReference/index.html?ApiReference-query-AuthorizeSecurityGroupIngress.html
And the "new" ones:
http://docs.aws.amazon.com/AWSEC2/2009-11-30/APIReference/index.html?ApiReference-query-AuthorizeSecurityGroupIngress.html
And always fails whenever I supply my own userid, and succeds when I don't.

2013-02-08 19:22:16,814 DEBUG [org.jclouds.http.internal.JavaUrlHttpCommandExecutorService] (i/o thread 7) Sending request -1262905079: POST https://ec2.us-east-1.amazonaws.com/ HTTP/1.1
2013-02-08 19:22:16,814 DEBUG [jclouds.wire] (i/o thread 7) >> "Action=AuthorizeSecurityGroupIngress&GroupName=jclouds%23nimbus-amplcluster%23us-east-1&Signature=8S5KsndBbwFYY8Asnd7x%2FLv1nmfBH%2B7YF63lQydRM4M%3D&SignatureMethod
=HmacSHA256&SignatureVersion=2&SourceSecurityGroupName=jclouds%23supervisor-amplcluster%23us-east-1&SourceSecurityGroupOwnerId=e26cd44119560f8e1e272d9bbc0928224c2414fde521bd81e4621789f9283ea6&Timestamp=2013-02-08T18%3A22%3A16.81
0Z&Version=2011-05-15&AWSAccessKeyId=AKIANNNNNNNNNNNNNNNNN"
2013-02-08 19:22:16,814 DEBUG [jclouds.headers] (i/o thread 7) >> POST https://ec2.us-east-1.amazonaws.com/ HTTP/1.1
2013-02-08 19:22:16,814 DEBUG [jclouds.headers] (i/o thread 7) >> Host: ec2.us-east-1.amazonaws.com
2013-02-08 19:22:16,814 DEBUG [jclouds.headers] (i/o thread 7) >> Content-Type: application/x-www-form-urlencoded
2013-02-08 19:22:16,815 DEBUG [jclouds.headers] (i/o thread 7) >> Content-Length: 449
2013-02-08 19:22:17,157 DEBUG [org.jclouds.http.internal.JavaUrlHttpCommandExecutorService] (i/o thread 7) Receiving response -1262905079: HTTP/1.1 400 Bad Request
2013-02-08 19:22:17,157 DEBUG [jclouds.headers] (i/o thread 7) << HTTP/1.1 400 Bad Request
2013-02-08 19:22:17,158 DEBUG [jclouds.headers] (i/o thread 7) << Transfer-Encoding: chunked
2013-02-08 19:22:17,158 DEBUG [jclouds.headers] (i/o thread 7) << Date: Fri, 08 Feb 2013 18:21:45 GMT
2013-02-08 19:22:17,158 DEBUG [jclouds.headers] (i/o thread 7) << Cneonction: close
2013-02-08 19:22:17,158 DEBUG [jclouds.headers] (i/o thread 7) << Server: AmazonEC2
2013-02-08 19:22:17,158 DEBUG [jclouds.headers] (i/o thread 7) << Content-Type: application/unknown
2013-02-08 19:22:17,158 DEBUG [jclouds.wire] (i/o thread 7) << "<?xml version="1.0" encoding="UTF-8"?>[\n]"
2013-02-08 19:22:17,158 DEBUG [jclouds.wire] (i/o thread 7) << "<Response><Errors><Error><Code>InvalidGroup.NotFound</Code><Message>Unable to find group 'jclouds#supervisor-amplcluster#us-east-1'</Message></Error></Errors><Reque
stID>31001939-f631-402a-a040-7ce66a25f41a</RequestID></Response>"
2013-02-08 19:22:17,177 ERROR [java.logging] (main) Exception in thread "main"
2013-02-08 19:22:17,177 ERROR [java.logging] (main) org.jclouds.rest.ResourceNotFoundException: Unable to find group 'jclouds#supervisor-amplcluster#us-east-1' (NO_SOURCE_FILE:1)
2013-02-08 19:22:17,177 ERROR [java.logging] (main) at clojure.lang.Compiler.eval(Compiler.java:5440)
2013-02-08 19:22:17,177 ERROR [java.logging] (main) at clojure.lang.Compiler.eval(Compiler.java:5415)
2013-02-08 19:22:17,177 ERROR [java.logging] (main) at clojure.lang.Compiler.eval(Compiler.java:5391)
2013-02-08 19:22:17,177 ERROR [java.logging] (main) at clojure.core$eval.invoke(core.clj:2382)
2013-02-08 19:22:17,178 ERROR [java.logging] (main) at clojure.main$eval_opt.invoke(main.clj:235)
2013-02-08 19:22:17,178 ERROR [java.logging] (main) at clojure.main$initialize.invoke(main.clj:254)
2013-02-08 19:22:17,178 ERROR [java.logging] (main) at clojure.main$null_opt.invoke(main.clj:279)
2013-02-08 19:22:17,178 ERROR [java.logging] (main) at clojure.main$main.doInvoke(main.clj:354)
2013-02-08 19:22:17,178 ERROR [java.logging] (main) at clojure.lang.RestFn.invoke(RestFn.java:422)
2013-02-08 19:22:17,178 ERROR [java.logging] (main) at clojure.lang.Var.invoke(Var.java:369)
2013-02-08 19:22:17,178 ERROR [java.logging] (main) at clojure.lang.AFn.applyToHelper(AFn.java:165)
2013-02-08 19:22:17,178 ERROR [java.logging] (main) at clojure.lang.Var.applyTo(Var.java:482)
2013-02-08 19:22:17,179 ERROR [java.logging] (main) at clojure.main.main(main.java:37)
2013-02-08 19:22:17,179 ERROR [java.logging] (main) Caused by: org.jclouds.rest.ResourceNotFoundException: Unable to find group 'jclouds#supervisor-amplcluster#us-east-1'
2013-02-08 19:22:17,179 ERROR [java.logging] (main) at org.jclouds.aws.handlers.ParseAWSErrorFromXmlContent.refineException(ParseAWSErrorFromXmlContent.java:109)
2013-02-08 19:22:17,179 ERROR [java.logging] (main) at org.jclouds.aws.handlers.ParseAWSErrorFromXmlContent.handleError(ParseAWSErrorFromXmlContent.java:92)
2013-02-08 19:22:17,179 ERROR [java.logging] (main) at org.jclouds.http.handlers.DelegatingErrorHandler.handleError(DelegatingErrorHandler.java:69)
2013-02-08 19:22:17,179 ERROR [java.logging] (main) at org.jclouds.http.internal.BaseHttpCommandExecutorService$HttpResponseCallable.shouldContinue(BaseHttpCommandExecutorService.java:191)
2013-02-08 19:22:17,179 ERROR [java.logging] (main) at org.jclouds.http.internal.BaseHttpCommandExecutorService$HttpResponseCallable.call(BaseHttpCommandExecutorService.java:161)
2013-02-08 19:22:17,180 ERROR [java.logging] (main) at org.jclouds.http.internal.BaseHttpCommandExecutorService$HttpResponseCallable.call(BaseHttpCommandExecutorService.java:130)
2013-02-08 19:22:17,180 ERROR [java.logging] (main) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
2013-02-08 19:22:17,180 ERROR [java.logging] (main) at java.util.concurrent.FutureTask.run(FutureTask.java:166)
2013-02-08 19:22:17,180 ERROR [java.logging] (main) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
2013-02-08 19:22:17,180 ERROR [java.logging] (main) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
2013-02-08 19:22:17,180 ERROR [java.logging] (main) at java.lang.Thread.run(Thread.java:679)
2013-02-08 19:22:17,180 ERROR [java.logging] (main) at org.jclouds.concurrent.config.ExecutorServiceModule$DescribingExecutorService.submit(ExecutorServiceModule.java:184)
2013-02-08 19:22:17,180 ERROR [java.logging] (main) at org.jclouds.http.internal.BaseHttpCommandExecutorService.submit(BaseHttpCommandExecutorService.java:127)
2013-02-08 19:22:17,181 ERROR [java.logging] (main) at org.jclouds.http.TransformingHttpCommandExecutorServiceImpl.submit(TransformingHttpCommandExecutorServiceImpl.java:54)
2013-02-08 19:22:17,181 ERROR [java.logging] (main) at org.jclouds.http.TransformingHttpCommandImpl.execute(TransformingHttpCommandImpl.java:73)
2013-02-08 19:22:17,181 ERROR [java.logging] (main) at org.jclouds.rest.internal.AsyncRestClientProxy.createListenableFutureForHttpRequestMappedToMethodAndArgs(AsyncRestClientProxy.java:198)
2013-02-08 19:22:17,181 ERROR [java.logging] (main) at org.jclouds.rest.internal.AsyncRestClientProxy.invoke(AsyncRestClientProxy.java:134)
2013-02-08 19:22:17,181 ERROR [java.logging] (main) at $Proxy74.authorizeSecurityGroupIngressInRegion(Unknown Source)
2013-02-08 19:22:17,181 ERROR [java.logging] (main) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2013-02-08 19:22:17,181 ERROR [java.logging] (main) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
2013-02-08 19:22:17,182 ERROR [java.logging] (main) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2013-02-08 19:22:17,182 ERROR [java.logging] (main) at java.lang.reflect.Method.invoke(Method.java:616)
2013-02-08 19:22:17,182 ERROR [java.logging] (main) at org.jclouds.concurrent.internal.SyncProxy.invoke(SyncProxy.java:137)
2013-02-08 19:22:17,182 ERROR [java.logging] (main) at $Proxy75.authorizeSecurityGroupIngressInRegion(Unknown Source)
2013-02-08 19:22:17,182 ERROR [java.logging] (main) at backtype.storm.security$authorize_group.invoke(security.clj:116)
2013-02-08 19:22:17,182 ERROR [java.logging] (main) at backtype.storm.security$authorize_group.invoke(security.clj:111)
2013-02-08 19:22:17,182 ERROR [java.logging] (main) at backtype.storm.provision$start_with_nodes_BANG_.invoke(provision.clj:81)
2013-02-08 19:22:17,182 ERROR [java.logging] (main) at backtype.storm.provision$start_BANG_.invoke(provision.clj:95)
2013-02-08 19:22:17,183 ERROR [java.logging] (main) at backtype.storm.provision$_main.doInvoke(provision.clj:147)
2013-02-08 19:22:17,183 ERROR [java.logging] (main) at clojure.lang.RestFn.invoke(RestFn.java:483)
2013-02-08 19:22:17,183 ERROR [java.logging] (main) at clojure.lang.Var.invoke(Var.java:381)
2013-02-08 19:22:17,183 ERROR [java.logging] (main) at user$eval8251.invoke(NO_SOURCE_FILE:1)
2013-02-08 19:22:17,183 ERROR [java.logging] (main) at clojure.lang.Compiler.eval(Compiler.java:5424)
2013-02-08 19:22:17,183 ERROR [java.logging] (main) at clojure.lang.Compiler.eval(Compiler.java:5415)
2013-02-08 19:22:17,183 ERROR [java.logging] (main) at clojure.lang.Compiler.eval(Compiler.java:5391)
2013-02-08 19:22:17,183 ERROR [java.logging] (main) at clojure.core$eval.invoke(core.clj:2382)
2013-02-08 19:22:17,184 ERROR [java.logging] (main) at clojure.main$eval_opt.invoke(main.clj:235)
2013-02-08 19:22:17,184 ERROR [java.logging] (main) at clojure.main$initialize.invoke(main.clj:254)
2013-02-08 19:22:17,184 ERROR [java.logging] (main) at clojure.main$null_opt.invoke(main.clj:279)
2013-02-08 19:22:17,184 ERROR [java.logging] (main) at clojure.main$main.doInvoke(main.clj:354)
2013-02-08 19:22:17,184 ERROR [java.logging] (main) at clojure.lang.RestFn.invoke(RestFn.java:422)
2013-02-08 19:22:17,184 ERROR [java.logging] (main) at clojure.lang.Var.invoke(Var.java:369)
2013-02-08 19:22:17,184 ERROR [java.logging] (main) at clojure.lang.AFn.applyToHelper(AFn.java:165)
2013-02-08 19:22:17,184 ERROR [java.logging] (main) at clojure.lang.Var.applyTo(Var.java:482)
2013-02-08 19:22:17,185 ERROR [java.logging] (main) at clojure.main.main(main.java:37)
2013-02-08 19:22:17,185 ERROR [java.logging] (main) at org.jclouds.concurrent.config.ExecutorServiceModule$DescribingExecutorService.submit(ExecutorServiceModule.java:184)
2013-02-08 19:22:17,185 ERROR [java.logging] (main) at org.jclouds.http.internal.BaseHttpCommandExecutorService.submit(BaseHttpCommandExecutorService.java:127)
2013-02-08 19:22:17,185 ERROR [java.logging] (main) at org.jclouds.http.TransformingHttpCommandExecutorServiceImpl.submit(TransformingHttpCommandExecutorServiceImpl.java:54)
2013-02-08 19:22:17,185 ERROR [java.logging] (main) at org.jclouds.http.TransformingHttpCommandImpl.execute(TransformingHttpCommandImpl.java:73)
2013-02-08 19:22:17,185 ERROR [java.logging] (main) at org.jclouds.rest.internal.AsyncRestClientProxy.createListenableFutureForHttpRequestMappedToMethodAndArgs(AsyncRestClientProxy.java:198)
2013-02-08 19:22:17,185 ERROR [java.logging] (main) at org.jclouds.rest.internal.AsyncRestClientProxy.invoke(AsyncRestClientProxy.java:134)
2013-02-08 19:22:17,185 ERROR [java.logging] (main) at $Proxy74.authorizeSecurityGroupIngressInRegion(Unknown Source)
2013-02-08 19:22:17,186 ERROR [java.logging] (main) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2013-02-08 19:22:17,186 ERROR [java.logging] (main) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
2013-02-08 19:22:17,186 ERROR [java.logging] (main) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2013-02-08 19:22:17,186 ERROR [java.logging] (main) at java.lang.reflect.Method.invoke(Method.java:616)
2013-02-08 19:22:17,186 ERROR [java.logging] (main) at org.jclouds.concurrent.internal.SyncProxy.invoke(SyncProxy.java:137)
2013-02-08 19:22:17,186 ERROR [java.logging] (main) at $Proxy75.authorizeSecurityGroupIngressInRegion(Unknown Source)
2013-02-08 19:22:17,186 ERROR [java.logging] (main) at backtype.storm.security$authorize_group.invoke(security.clj:116)
2013-02-08 19:22:17,187 ERROR [java.logging] (main) at backtype.storm.security$authorize_group.invoke(security.clj:111)
2013-02-08 19:22:17,187 ERROR [java.logging] (main) at backtype.storm.provision$start_with_nodes_BANG_.invoke(provision.clj:81)
2013-02-08 19:22:17,187 ERROR [java.logging] (main) at backtype.storm.provision$start_BANG_.invoke(provision.clj:95)
2013-02-08 19:22:17,187 ERROR [java.logging] (main) at backtype.storm.provision$_main.doInvoke(provision.clj:147)
2013-02-08 19:22:17,187 ERROR [java.logging] (main) at clojure.lang.RestFn.invoke(RestFn.java:483)
2013-02-08 19:22:17,187 ERROR [java.logging] (main) at clojure.lang.Var.invoke(Var.java:381)
2013-02-08 19:22:17,187 ERROR [java.logging] (main) at user$eval8251.invoke(NO_SOURCE_FILE:1)
2013-02-08 19:22:17,188 ERROR [java.logging] (main) at clojure.lang.Compiler.eval(Compiler.java:5424)
2013-02-08 19:22:17,188 ERROR [java.logging] (main) ... 12 more
2013-02-08 19:22:17,188 ERROR [java.logging] (main) Caused by: org.jclouds.aws.AWSResponseException: request POST https://ec2.us-east-1.amazonaws.com/ HTTP/1.1 failed with code 400, error: AWSError{requestId='31001939-f631-402a-a040-7ce66a25f41a', requestToken='null', code='InvalidGroup.NotFound', message='Unable to find group 'jclouds#supervisor-amplcluster#us-east-1'', context='{Response=, Errors=}'}
2013-02-08 19:22:17,189 ERROR [java.logging] (main) at org.jclouds.aws.handlers.ParseAWSErrorFromXmlContent.handleError(ParseAWSErrorFromXmlContent.java:78)
2013-02-08 19:22:17,189 ERROR [java.logging] (main) at org.jclouds.http.handlers.DelegatingErrorHandler.handleError(DelegatingErrorHandler.java:69)
2013-02-08 19:22:17,189 ERROR [java.logging] (main) at org.jclouds.http.internal.BaseHttpCommandExecutorService$HttpResponseCallable.shouldContinue(BaseHttpCommandExecutorService.java:191)
2013-02-08 19:22:17,189 ERROR [java.logging] (main) at org.jclouds.http.internal.BaseHttpCommandExecutorService$HttpResponseCallable.call(BaseHttpCommandExecutorService.java:161)
2013-02-08 19:22:17,189 ERROR [java.logging] (main) at org.jclouds.http.internal.BaseHttpCommandExecutorService$HttpResponseCallable.call(BaseHttpCommandExecutorService.java:130)
2013-02-08 19:22:17,189 ERROR [java.logging] (main) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
2013-02-08 19:22:17,189 ERROR [java.logging] (main) at java.util.concurrent.FutureTask.run(FutureTask.java:166)
2013-02-08 19:22:17,189 ERROR [java.logging] (main) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
2013-02-08 19:22:17,190 ERROR [java.logging] (main) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
2013-02-08 19:22:17,190 ERROR [java.logging] (main) at java.lang.Thread.run(Thread.java:679)

I've tried through the aws ec2 cli program with the latest version, and happens the same to me.
This fails to me:

ec2-authorize 'jclouds#nimbus-amcluster#us-east-1' -P tcp -p 1-65535 -u e2641eXXXXXXXXXXXXXc0928224c2414fde521bd8XXXXXXXXXf9283 -o 'jclouds#supervisor-amcluster#us-east-1'

This works to me:

ec2-authorize --help 'jclouds#nimbus-amcluster#us-east-1' -P tcp -p 1-65535 -o 'jclouds#supervisor-amcluster#us-east-1'

The same thorugh the php-sdk-api:
This works to me:

<?php

$ingroup = $argv[1];
$outgroup = $argv[2];

include("sdk/sdk.class.php");

    // Instantiate the AmazonEC2 class
    $ec2 = new AmazonEC2();

    $response = $ec2->authorize_security_group_ingress(
        array('GroupName' => $ingroup, 
                'IpPermissions' => array( 
                    array('IpProtocol' => 'tcp', 
                    'FromPort' => '1', 
                    'ToPort' => '65535', 
                    'Groups' => array( 
                        array(
                    'GroupName' => $outgroup,
                 ),
                    )
            )
        )
    )
        );
    var_dump($response->body);

This fails:

<?php

$ingroup = $argv[1];
$outgroup = $argv[2];

include("sdk/sdk.class.php");

    // Instantiate the AmazonEC2 class
    $ec2 = new AmazonEC2();

    $response = $ec2->authorize_security_group_ingress(
        array('GroupName' => $ingroup, 
                'IpPermissions' => array( 
                    array('IpProtocol' => 'tcp', 
                    'FromPort' => '1', 
                    'ToPort' => '65535', 
                    'Groups' => array( 
                        array(
                    'GroupName' => $outgroup,
                    'UserId' => 'AMXNXNXNXNXNXNX',
                 ),
                    )
            )
        )
    )
        );
    var_dump($response->body);
@jasonjckn
Collaborator

thanks for explanation! merging. I didn't test it myself.

@jasonjckn jasonjckn merged commit 09a96e5 into nathanmarz:master Mar 23, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment