Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using DescribeSecurityGroup IpPermissions in RevokeSecurityGroupIngress Raises Error #1716

Closed
phene opened this issue Feb 20, 2018 · 21 comments
Labels
documentation This is a problem with documentation. service-api General API label for AWS Services.

Comments

@phene
Copy link

phene commented Feb 20, 2018

Issue description

Passing the ip_permissions property of the result of describe_security_groups to revoke_security_group_ingress results in a server error Aws::EC2::Errors::InvalidParameterValue: missing mandatory parameter: exactly one of remote-security-group, remote-ip-range, remote-ipv6-range, or prefix-list-id must be present This is due to the definition of rules that reference other security groups that take the format:

{
  :from_port => 0, 
  :ip_protocol => "tcp", 
  :ip_ranges => [], 
  :ipv_6_ranges => [], 
  :prefix_list_ids => [], 
  :to_port => 65535, 
  :user_id_group_pairs => [
    {:group_id => "[elided]", :user_id => "[elided]"}, 
    {:group_id => "[elided]", :user_id => "[elided]"}
  ]
}

There appears to be a mismatch in expectations between the client code and server code since adding dry_run: true to my request results in: Aws::EC2::Errors::DryRunOperation: Request would have succeeded, but DryRun flag is set.

Background

Because of a cyclical dependency issue between EMR-managed security groups, I'm attempting to clear the rules from them before letting CloudFormation delete them. Before my CloudFormation stack delete code runs, I attempt to run DataPipeline#clear_security_group_rules defined below.

Gem name

aws-sdk-ec2 @ 1.27.0 and 1.25.0

Version of Ruby, OS environment

MacOS Sierra, ruby-2.4.2

Code snippets / steps to reproduce

class DataPipeline
  # EMR-managed groups create security groups with cross-dependency rules.
  # The rules need to be removed so CloudFormation can delete the security groups
  def clear_security_group_rules
    security_groups.each do |security_group|
      puts security_group.ip_permissions.map(&:to_hash)
      ec2_client.revoke_security_group_ingress(
        group_id: security_group.group_id,
        ip_permissions: security_group.ip_permissions
      )
    end
  end

  def security_groups
    ec2_client.describe_security_groups(
      group_ids: ['sg-af0264d0', 'sg-bd0f69c2'] # Hard-coded here to reduce irrelevant code
    ).security_groups
  end

  def ec2_client
    @ec2_client ||= Aws::EC2::Client.new
  end
end


pipeline = DataPipeline.new
pipeline.clear_security_group_rules
# Prints:
#   {:from_port=>0, :ip_protocol=>"tcp", :ip_ranges=>[], :ipv_6_ranges=>[], :prefix_list_ids=>[], :to_port=>65535, :user_id_group_pairs=>[{:group_id=>"[elided]", :user_id=>"[elided]"}, {:group_id=>"[elided]", :user_id=>"[elided]"}]}
#   {:from_port=>0, :ip_protocol=>"udp", :ip_ranges=>[], :ipv_6_ranges=>[], :prefix_list_ids=>[], :to_port=>65535, :user_id_group_pairs=>[{:group_id=>"[elided]", :user_id=>"[elided]"}, {:group_id=>"[elided]", :user_id=>"[elided]"}]}
#   {:from_port=>-1, :ip_protocol=>"icmp", :ip_ranges=>[], :ipv_6_ranges=>[], :prefix_list_ids=>[], :to_port=>-1, :user_id_group_pairs=>[{:group_id=>"[elided]", :user_id=>"[elided]"}, {:group_id=>"[elided]", :user_id=>"[elided]"}]}
# Raises:
#   Aws::EC2::Errors::InvalidParameterValue: missing mandatory parameter: exactly one of remote-security-group, remote-ip-range, remote-ipv6-range, or prefix-list-id must be present
@cjyclaire cjyclaire added the service-api General API label for AWS Services. label Feb 21, 2018
@cjyclaire
Copy link
Contributor

Thanks for the info! I've passed this issue to service team, will update.
Meanwhile, feel free to contact [AWS Support] directly for this issue as well.

@phene
Copy link
Author

phene commented Feb 22, 2018

AWS Support thinks it's the fault of the API library.

Thanks for contacting AWS Support. My name is Gurdeep and I will assist you with this case.

From your case correspondence, I understand while using the AWS Ruby SDK for making the API call “revoke_security_group_ingress” you are getting the following error: 

“Aws::EC2::Errors::InvalidParameterValue: missing mandatory parameter: exactly one of remote-security-group, remote-ip-range, remote-ipv6-range, or prefix-list-id must be present”.

You have passed the security group id and the IP permissions in the request. But the AWS server is returning the error that required parameters were not passed in the query. Please, correct me if I’m wrong.

I did deep dive and replicated the scenario you want to achieve with the provided code. I got the same error during the replication. For the further analysis that what is causing the issue, I have taken the packet capture and in the capture, it seems that the API request “revoke_security_group_ingress” is not sending the required parameters to the AWS Server. That's why AWS Server is returning the “InvalidParameterValue” error.

Packet Capture showing the API call without required parameters:

08:13:23.721861 IP ip-172-31-6-219.ap-northeast-1.compute.internal.37336 > 54.239.96.159.http: Flags [P.], seq 2638:2971, ack 1727, win 486, length 333 
0x0000: 4500 0175 5162 4000 ff06 de97 ac1f 06db E..uQb@......... 
0x0010: 36ef 609f 91d8 0050 9154 1fd4 4bdf bcdf 6.`....P.T..K... 
0x0020: 5018 01e6 4bf0 0000 4163 7469 6f6e 3d52 P...K...Action=R 
0x0030: 6576 6f6b 6553 6563 7572 6974 7947 726f evokeSecurityGro 
0x0040: 7570 496e 6772 6573 7326 4772 6f75 7049 upIngress&GroupI 
0x0050: 643d 7367 2d39 3865 3161 3065 3126 4970 d=sg-98e1a0e1&Ip 
0x0060: 5065 726d 6973 7369 6f6e 732e 312e 4672 Permissions.1.Fr 
0x0070: 6f6d 506f 7274 3d31 3233 2649 7050 6572 omPort=123&IpPer 
0x0080: 6d69 7373 696f 6e73 2e31 2e47 726f 7570 missions.1.Group 
0x0090: 733d 2649 7050 6572 6d69 7373 696f 6e73 s=&IpPermissions 
0x00a0: 2e31 2e49 7050 726f 746f 636f 6c3d 7463 .1.IpProtocol=tc 
0x00b0: 7026 4970 5065 726d 6973 7369 6f6e 732e p&IpPermissions. 
0x00c0: 312e 4970 5261 6e67 6573 2e31 2e43 6964 1.IpRanges.1.Cid 
0x00d0: 7249 703d 342e 342e 342e 3425 3246 3332 rIp=4.4.4.4%2F32 
0x00e0: 2649 7050 6572 6d69 7373 696f 6e73 2e31 &IpPermissions.1 
0x00f0: 2e49 7052 616e 6765 732e 312e 4465 7363 .IpRanges.1.Desc 
0x0100: 7269 7074 696f 6e3d 7465 7374 2649 7050 ription=test&IpP 
0x0110: 6572 6d69 7373 696f 6e73 2e31 2e49 7076 ermissions.1.Ipv 
0x0120: 3652 616e 6765 733d 2649 7050 6572 6d69 6Ranges=&IpPermi 
0x0130: 7373 696f 6e73 2e31 2e50 7265 6669 784c ssions.1.PrefixL 
0x0140: 6973 7449 6473 3d26 4970 5065 726d 6973 istIds=&IpPermis 
0x0150: 7369 6f6e 732e 312e 546f 506f 7274 3d31 sions.1.ToPort=1 
0x0160: 3233 2656 6572 7369 6f6e 3d32 3031 362d 23&Version=2016- 
0x0170: 3131 2d31 35 11-15 

I have raised the issue with the internal SDK team. The internal team is looking into the matter and will fix it. I will update you with the finding of the internal team. Issue is also get acknowledged by the AWS on the Github.

For the time being if possible please use other SDKs available on the AWS. For checking more available SDKs please refer to the below given link:

https://aws.amazon.com/tools/

Hope this information is useful for you. Meanwhile, If you have a further query feel free to write us back. We will be more than happy to help you.

Best regards,

GURDEEP S.
Amazon Web Services

@phene
Copy link
Author

phene commented Feb 22, 2018

When I throw some debugging into the aws client, I see that my request params properly includes the Groups.1.GroupId, etc that AWS Support does not include.

"Action=RevokeSecurityGroupIngress
&GroupId=sg-bd0f69c2
&IpPermissions.1.FromPort=0
&IpPermissions.1.Groups.1.GroupId=sg-af0264d0
&IpPermissions.1.Groups.1.UserId=elided
&IpPermissions.1.Groups.2.GroupId=sg-bd0f69c2
&IpPermissions.1.Groups.2.UserId=elided
&IpPermissions.1.IpProtocol=tcp
&IpPermissions.1.IpRanges=
&IpPermissions.1.Ipv6Ranges=
&IpPermissions.1.PrefixListIds=
&IpPermissions.1.ToPort=65535
&IpPermissions.2.FromPort=0
&IpPermissions.2.Groups.1.GroupId=sg-af0264d0
&IpPermissions.2.Groups.1.UserId=elided
&IpPermissions.2.Groups.2.GroupId=sg-bd0f69c2
&IpPermissions.2.Groups.2.UserId=elided
&IpPermissions.2.IpProtocol=udp
&IpPermissions.2.IpRanges=
&IpPermissions.2.Ipv6Ranges=
&IpPermissions.2.PrefixListIds=
&IpPermissions.2.ToPort=65535
&IpPermissions.3.FromPort=-1
&IpPermissions.3.Groups.1.GroupId=sg-af0264d0
&IpPermissions.3.Groups.1.UserId=elided
&IpPermissions.3.Groups.2.GroupId=sg-bd0f69c2
&IpPermissions.3.Groups.2.UserId=elided
&IpPermissions.3.IpProtocol=icmp
&IpPermissions.3.IpRanges=
&IpPermissions.3.Ipv6Ranges=
&IpPermissions.3.PrefixListIds=
&IpPermissions.3.ToPort=-1
&Version=2016-11-15"

@cjyclaire
Copy link
Contributor

cjyclaire commented Feb 22, 2018

I revisit your issue, below snippet works for me:

sg = ec2.describe_security_groups(group_ids: ids)
sg.security_groups do |g|
  ec2.revoke_security_group_ingress(group_id: g.group_id, ip_permissions: g.ip_permissions)
end

Can you tell me which operation cause the dryrun mismatch for you?

I was working with support, the error he provided to me is different from what you have mentioned earlier, I couldn't reproduce a scenario where SDK doesn't use the parameter provided. I'll work with him to see what issue he ran into.

Also, to help with debugging, you can set :http_wire_trace to true from client

@phene
Copy link
Author

phene commented Feb 22, 2018

Are you testing with security group rules that refer to other security groups? I'm performing this on the security groups that EMR generates ingress rules onto.

2018-02-22 22:38:33 UTC 492 test.rb: info: Clearing security group ingress rules for geofftest-Pipeline-MAU-Hourly-EMRSlaveSecurityGroup-1HAQF7I7DYHQQ
opening connection to ec2.us-west-2.amazonaws.com:443...
opened
starting SSL for ec2.us-west-2.amazonaws.com:443...
SSL established
<- "POST / HTTP/1.1\r\nContent-Type: application/x-www-form-urlencoded; charset=utf-8\r\nAccept-Encoding: \r\nUser-Agent: aws-sdk-ruby3/3.16.0 ruby/2.4.2 x86_64-darwin16 aws-sdk-ec2/1.27.0\r\nHost: ec2.us-west-2.amazonaws.com\r\nX-Amz-Date: 20180222T223833Z\r\nX-Amz-Security-Token: [elided]\r\nX-Amz-Content-Sha256: 2a023cace68c591c82bbc6e763856e26b24d9be4ff7e80ab7e6b64f2f0edcc55\r\nAuthorization: AWS4-HMAC-SHA256 Credential=ASIAJXJFIVK4U7KZNVBA/20180222/us-west-2/ec2/aws4_request, SignedHeaders=content-type;host;x-amz-content-sha256;x-amz-date;x-amz-security-token, Signature=265decb5522a4fc03b347351de0d972566caa20f90a0a00518380b1363880f64\r\nContent-Length: 1127\r\nAccept: */*\r\n\r\n"
-> "HTTP/1.1 400 Bad Request\r\n"
-> "Transfer-Encoding: chunked\r\n"
-> "Date: Thu, 22 Feb 2018 22:38:33 GMT\r\n"
-> "Connection: close\r\n"
-> "Server: AmazonEC2\r\n"
-> "\r\n"
-> "154\r\n"
reading 340 bytes...
-> "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Response><Errors><Error><Code>InvalidParameterValue</Code><Message>missing mandatory parameter: exactly one of remote-security-group, remote-ip-range, remote-ipv6-range, or prefix-list-id must be present</Message></Error></Errors><RequestID>632215ad-8a0d-4b6a-a345-d7132e914ec5</RequestID></Response>"
read 340 bytes
reading 2 bytes...
-> "\r\n"
read 2 bytes
-> "0\r\n"
-> "\r\n"
Conn close
/Users/geoff/.rvm/gems/ruby-2.4.2@analytics-cluster/gems/aws-sdk-core-3.16.0/lib/seahorse/client/plugins/raise_response_errors.rb:15:in `call': missing mandatory parameter: exactly one of remote-security-group, remote-ip-range, remote-ipv6-range, or prefix-list-id must be present (Aws::EC2::Errors::InvalidParameterValue)
	from /Users/geoff/.rvm/gems/ruby-2.4.2@analytics-cluster/gems/aws-sdk-core-3.16.0/lib/aws-sdk-core/plugins/jsonvalue_converter.rb:20:in `call'
	from /Users/geoff/.rvm/gems/ruby-2.4.2@analytics-cluster/gems/aws-sdk-core-3.16.0/lib/aws-sdk-core/plugins/idempotency_token.rb:17:in `call'
	from /Users/geoff/.rvm/gems/ruby-2.4.2@analytics-cluster/gems/aws-sdk-core-3.16.0/lib/aws-sdk-core/plugins/param_converter.rb:24:in `call'
	from /Users/geoff/.rvm/gems/ruby-2.4.2@analytics-cluster/gems/aws-sdk-core-3.16.0/lib/aws-sdk-core/plugins/response_paging.rb:10:in `call'
	from /Users/geoff/.rvm/gems/ruby-2.4.2@analytics-cluster/gems/aws-sdk-core-3.16.0/lib/seahorse/client/plugins/response_target.rb:23:in `call'
	from /Users/geoff/.rvm/gems/ruby-2.4.2@analytics-cluster/gems/aws-sdk-core-3.16.0/lib/seahorse/client/request.rb:70:in `send_request'
	from /Users/geoff/.rvm/gems/ruby-2.4.2@analytics-cluster/gems/aws-sdk-ec2-1.27.0/lib/aws-sdk-ec2/client.rb:22760:in `revoke_security_group_ingress'

@cjyclaire
Copy link
Contributor

taking a look, I finally can reproduce this, will update

@cjyclaire
Copy link
Contributor

cjyclaire commented Feb 22, 2018

Okay, I finally realize what going on there @phene

missing mandatory parameter: exactly one of remote-security-group, remote-ip-range, remote-ipv6-range, or prefix-list-id must be present

It means EXACTLY ONE of those ip-range ipv6-range or prefix-list-id is require per one API call. The "missing mandatory parameter" is quite misleading.

It doesn't mean that you are missing parameters, it's suggesting that you can only provide one per one API call.

I'll contact service team to update their doc and error message on this, it's quite misleading.

@cjyclaire cjyclaire added closing-soon This issue will automatically close in 4 days unless further comments are made. and removed information requested labels Feb 22, 2018
@phene
Copy link
Author

phene commented Feb 22, 2018

That doesn't make sense -- doesn't your earlier test pass both ip-range and ipv6-range given they are properties of IpPermission? You also left out remote-security-group from your list of mandatory parameters, which is the key here. How would you modify my code such that it succeeds? My suspicion is that the IpPermissions.1.Groups.1.GroupId parameter is not be interpreted as the remote-security-group.

@cjyclaire
Copy link
Contributor

@phene My bad, It's my mistake in my earlier code snippet, I do miss the each at first, so it appear "succeed" first and actually nothing is called. After I add the each, I can reproduce same error you got, after I hard code ip_permissions with single required field, request succeed.

@phene
Copy link
Author

phene commented Feb 23, 2018

So is the conclusion that revoking via IpPermissions is not supported? This goes against SDK docs.

@cjyclaire
Copy link
Contributor

cjyclaire commented Feb 23, 2018

The point is you might need to do some tweaks instead of passing through the IpPermissions directly. the update can only process one of field per time.

@phene
Copy link
Author

phene commented Feb 23, 2018

OK, then the SDK docs and parameter validation should be updated to reflect that. It's rather inconvenient to do this since an IpPermission object can actually reflect multiple rules since there can be multiple user_id_group_pairs entries. This also seems like a major naming kludge and should probably be rectified.

@awood45
Copy link
Member

awood45 commented Feb 23, 2018

Sorry about the confusion, I can totally see where you're following the way the docs are written, and how the exception name you got back from the service is not helpful (upon re-reading it does seem to say the key "exactly one" provision, but I read it the way you did the first time).

We've passed this feedback on to the service team and will press for a change to the shared documentation for this method.

@awood45 awood45 added documentation This is a problem with documentation. and removed closing-soon This issue will automatically close in 4 days unless further comments are made. labels Feb 23, 2018
@awood45
Copy link
Member

awood45 commented Feb 23, 2018

That aside, have you been able to get your calls working with this clarification?

@phene
Copy link
Author

phene commented Feb 23, 2018

Not yet... I'm not even sure what to do with this error message or the VPC is mentions. There's no VPC ID parameter in the revoke_security_group_ingress method.

ec2 management console 2018-02-22 17-50-57

      def clear_security_group_rules
        security_groups.each do |security_group|
          Helpers.info "Clearing security group ingress rules for #{security_group.group_name}"
          puts security_group.to_hash
          security_group.ip_permissions.each do |ip_permission|
            revocations(ip_permission).each do |revocation|
              revocation.merge!(group_id: security_group.group_id)
              puts revocation
              Helpers.call_aws(
                ec2_client, :revoke_security_group_ingress,
                revocation
              )
            end
          end
        end
      end

      def revocations(ip_permission)
        [].tap do |rules|
          if ip_permission.user_id_group_pairs.any?
            ip_permission.user_id_group_pairs.each do |user_id_group_pair|
              rules << {
                ip_protocol: ip_permission.ip_protocol,
                to_port: ip_permission.to_port,
                source_security_group_name: user_id_group_pair.group_id,
                source_security_group_owner_id: user_id_group_pair.user_id,
              }
            end
          end
        end
      end
$ ruby test.rb
2018-02-23 01:48:39 UTC 18898 test.rb: info: Clearing security group ingress rules for geofftest-Pipeline-MAU-Hourly-EMRSlaveSecurityGroup-1HAQF7I7DYHQQ
{:description=>"geofftest-Pipeline-MAU-Hourly EMR Slave", :group_name=>"geofftest-Pipeline-MAU-Hourly-EMRSlaveSecurityGroup-1HAQF7I7DYHQQ", :ip_permissions=>[{:from_port=>0, :ip_protocol=>"tcp", :ip_ranges=>[], :ipv_6_ranges=>[], :prefix_list_ids=>[], :to_port=>65535, :user_id_group_pairs=>[{:group_id=>"sg-af0264d0", :user_id=>"585460489129"}, {:group_id=>"sg-bd0f69c2", :user_id=>"585460489129"}]}, {:from_port=>0, :ip_protocol=>"udp", :ip_ranges=>[], :ipv_6_ranges=>[], :prefix_list_ids=>[], :to_port=>65535, :user_id_group_pairs=>[{:group_id=>"sg-af0264d0", :user_id=>"585460489129"}, {:group_id=>"sg-bd0f69c2", :user_id=>"585460489129"}]}, {:from_port=>-1, :ip_protocol=>"icmp", :ip_ranges=>[], :ipv_6_ranges=>[], :prefix_list_ids=>[], :to_port=>-1, :user_id_group_pairs=>[{:group_id=>"sg-af0264d0", :user_id=>"585460489129"}, {:group_id=>"sg-bd0f69c2", :user_id=>"585460489129"}]}], :owner_id=>"585460489129", :group_id=>"sg-bd0f69c2", :ip_permissions_egress=>[{:ip_protocol=>"-1", :ip_ranges=>[{:cidr_ip=>"0.0.0.0/0"}], :ipv_6_ranges=>[], :prefix_list_ids=>[], :user_id_group_pairs=>[]}], :tags=>[{:key=>"region", :value=>"us-west-2"}, {:key=>"input_file", :value=>"config/parameters/stack-input-files/data-pipeline/default.yaml"}, {:key=>"system_name", :value=>"geofftest"}, {:key=>"Name", :value=>"geofftest-Pipeline-MAU-Hourly-EMR-Slave"}, {:key=>"aws:cloudformation:logical-id", :value=>"EMRSlaveSecurityGroup"}, {:key=>"aws:cloudformation:stack-name", :value=>"geofftest-Pipeline-MAU-Hourly"}, {:key=>"branch", :value=>"task/INTEL-4552-emr-managed-security-groups"}, {:key=>"user_id", :value=>"Jenkins"}, {:key=>"aws:cloudformation:stack-id", :value=>"arn:aws:cloudformation:us-west-2:585460489129:stack/geofftest-Pipeline-MAU-Hourly/d6b03ee0-168b-11e8-926b-503aca41a099"}, {:key=>"main_stack", :value=>"geofftest-Pipeline-MAU-Hourly"}, {:key=>"build_number", :value=>"latest"}, {:key=>"job_name", :value=>"MAUHourlyPipeline"}], :vpc_id=>"vpc-16f0646f"}
{:ip_protocol=>"tcp", :to_port=>65535, :source_security_group_name=>"sg-af0264d0", :source_security_group_owner_id=>"585460489129", :group_id=>"sg-bd0f69c2"}
/Users/geoff/.rvm/gems/ruby-2.4.2@analytics-cluster/gems/aws-sdk-core-3.16.0/lib/seahorse/client/plugins/raise_response_errors.rb:15:in `call': The security group 'sg-af0264d0' does not exist in default VPC 'vpc-ba8abbde' (Aws::EC2::Errors::InvalidGroupNotFound)
	from /Users/geoff/.rvm/gems/ruby-2.4.2@analytics-cluster/gems/aws-sdk-core-3.16.0/lib/aws-sdk-core/plugins/jsonvalue_converter.rb:20:in `call'
	from /Users/geoff/.rvm/gems/ruby-2.4.2@analytics-cluster/gems/aws-sdk-core-3.16.0/lib/aws-sdk-core/plugins/idempotency_token.rb:17:in `call'
	from /Users/geoff/.rvm/gems/ruby-2.4.2@analytics-cluster/gems/aws-sdk-core-3.16.0/lib/aws-sdk-core/plugins/param_converter.rb:24:in `call'
	from /Users/geoff/.rvm/gems/ruby-2.4.2@analytics-cluster/gems/aws-sdk-core-3.16.0/lib/aws-sdk-core/plugins/response_paging.rb:10:in `call'
	from /Users/geoff/.rvm/gems/ruby-2.4.2@analytics-cluster/gems/aws-sdk-core-3.16.0/lib/seahorse/client/plugins/response_target.rb:23:in `call'
	from /Users/geoff/.rvm/gems/ruby-2.4.2@analytics-cluster/gems/aws-sdk-core-3.16.0/lib/seahorse/client/request.rb:70:in `send_request'
	from /Users/geoff/.rvm/gems/ruby-2.4.2@analytics-cluster/gems/aws-sdk-ec2-1.27.0/lib/aws-sdk-ec2/client.rb:22760:in `revoke_security_group_ingress'
	from /Users/geoff/projects/ops.aws.analytics-cluster/lib/configurator/helpers.rb:255:in `call_aws'
	from /Users/geoff/projects/ops.aws.analytics-cluster/lib/configurator/stack/data_pipeline.rb:25:in `block (3 levels) in clear_security_group_rules'

@phene
Copy link
Author

phene commented Feb 26, 2018

I see that source_security_group_name requires it belongs to the default VPC. How can I remove a rule for a source security group that does not exist on the default VPC?

@phene
Copy link
Author

phene commented Feb 26, 2018

Somehow, this works via the CLI:
aws --profile [elided] --region us-west-2 ec2 revoke-security-group-ingress --group-id sg-c56410ba --source-group sg-577f0b28 --protocol tcp --port 0-65535

@phene
Copy link
Author

phene commented Feb 26, 2018

Sending one ip_permission entry at a time (like the aws cli) yields the same original error:

      def clear_security_group_rules
        security_groups.each do |security_group|
          Helpers.info "Clearing security group ingress rules for #{security_group.group_name}"
          security_group.ip_permissions.each do |ip_permission|
            next unless ip_permission.user_id_group_pairs.any?
            ip_permission.user_id_group_pairs.each do |user_id_group_pair|
              ip_perm = ip_permission.to_hash
              ip_perm[:user_id_group_pairs] = [user_id_group_pair.to_hash]

              ec2_client.revoke_security_group_ingress(
                group_id: security_group.group_id,
                ip_permissions: [ip_perm]
              )
             end
          end
        end
      end

@phene
Copy link
Author

phene commented Feb 26, 2018

It appears that removing the "empty" key/value pairs fixes the issue. Reconstructing a new hash (rather than using to_hash, which includes :ip_ranges=>[], :ipv_6_ranges=>[], :prefix_list_ids=>[],) succeeds. The error "missing parameter" is doubly infuriating since I included too many of them.

@phene
Copy link
Author

phene commented Feb 26, 2018

To summarize this issue. I cannot send:

{
  group_id: [elided],
  ip_permissions: [{
    :from_port => 0, 
    :ip_protocol => "tcp", 
    :ip_ranges => [], 
    :ipv_6_ranges => [], 
    :prefix_list_ids => [], 
    :to_port => 65535, 
    :user_id_group_pairs => [
      {:group_id => "[elided1]", :user_id => "[elided1]"}, 
      {:group_id => "[elided2]", :user_id => "[elided2]"}
    ]
  },
  { 
    :from_port => -1,
    :ip_protocol => "icmp", 
    :ip_ranges => [], 
    :ipv_6_ranges => [], 
    :prefix_list_ids => [], 
    :to_port => -1, 
    :user_id_group_pairs => [
      {:group_id => "[elided1]", :user_id => "[elided1]"}, 
      {:group_id => "[elided2]", :user_id => "[elided2]"}
    ]
  },
  ]
}

But I can send:

{
  group_id: [elided],
  ip_permissions: [{
    :from_port => 0, 
    :ip_protocol => "tcp", 
    :to_port => 65535, 
    :user_id_group_pairs => [
      {:group_id => "[elided1]", :user_id => "[elided1]"}
    ]
  }]
}

and

{
  group_id: [elided],
  ip_permissions: [{
    :from_port => 0, 
    :ip_protocol => "tcp", 
    :to_port => 65535, 
    :user_id_group_pairs => [
      {:group_id => "[elided2]", :user_id => "[elided2]"}
    ]
  }]
}

and

{
  group_id: [elided],
  ip_permissions: [{
    :from_port => -1, 
    :ip_protocol => "icmp", 
    :to_port => -1, 
    :user_id_group_pairs => [
      {:group_id => "[elided1]", :user_id => "[elided1]"}
    ]
  }]
}

, etc...

@cjyclaire
Copy link
Contributor

cjyclaire commented Jun 26, 2018

To follow-up, we were following up with service team to make sure error messages would be enhanced in similar situations : ) Appreciate your patience and efforts!
Closing the issue as it has been figured out, free feel to re-open with further comments or questions :D

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation This is a problem with documentation. service-api General API label for AWS Services.
Projects
None yet
Development

No branches or pull requests

3 participants