Failed connecting to SNS: Profile "..." not found!

[ulsa]

I suddenly, after having it working fine for weeks, get problems starting dynamic-dynamodb. Seems that it doesn't find the instance profile:

2014-06-03 12:58:50,194 - dynamic-dynamodb - DEBUG - Authenticating to SNS using EC2 instance profile
2014-06-03 12:58:50,230 - dynamic-dynamodb - ERROR - Failed connecting to SNS: Profile "arn:aws:iam::546683855188:instance-profile/dev-MyInstanceProfile-8Y3JNGL82DFS" not found!
2014-06-03 12:58:50,230 - dynamic-dynamodb - ERROR - Please report an issue at: https://github.com/sebdah/dynamic-dynamodb/issues
Traceback (most recent call last):
  File "/usr/local/bin/dynamic-dynamodb", line 22, in <module>
    import dynamic_dynamodb
  File "/usr/local/lib/python2.7/dist-packages/dynamic_dynamodb/__init__.py", line 29, in <module>
    from dynamic_dynamodb.aws import dynamodb
  File "/usr/local/lib/python2.7/dist-packages/dynamic_dynamodb/aws/dynamodb.py", line 19, in <module>
    from dynamic_dynamodb.aws import sns
  File "/usr/local/lib/python2.7/dist-packages/dynamic_dynamodb/aws/sns.py", line 124, in <module>
    SNS_CONNECTION = __get_connection_SNS()
  File "/usr/local/lib/python2.7/dist-packages/dynamic_dynamodb/aws/sns.py", line 106, in __get_connection_SNS
    profile_name=metadata['iam']['info'][u'InstanceProfileArn'])
  File "/usr/local/lib/python2.7/dist-packages/boto/sns/__init__.py", line 53, in connect_to_region
    return region.connect(**kw_params)
  File "/usr/local/lib/python2.7/dist-packages/boto/regioninfo.py", line 188, in connect
    return self.connection_cls(region=self, **kw_params)
  File "/usr/local/lib/python2.7/dist-packages/boto/sns/connection.py", line 76, in __init__
    profile_name=profile_name)
  File "/usr/local/lib/python2.7/dist-packages/boto/connection.py", line 1073, in __init__
    profile_name=profile_name)
  File "/usr/local/lib/python2.7/dist-packages/boto/connection.py", line 558, in __init__
    profile_name)
  File "/usr/local/lib/python2.7/dist-packages/boto/provider.py", line 197, in __init__
    self.get_credentials(access_key, secret_key, security_token, profile_name)
  File "/usr/local/lib/python2.7/dist-packages/boto/provider.py", line 293, in get_credentials
    profile_name)
boto.provider.ProfileNotFoundError: Profile "arn:aws:iam::546683855188:instance-profile/dev-MyInstanceProfile-8Y3JNGL82DFS" not found!

However, the instance profile is there:

$ curl 169.254.169.254/latest/meta-data/iam/info/
{ 
  "Code" : "Success",
  "LastUpdated" : "2014-06-03T12:38:37Z",
  "InstanceProfileArn" : "arn:aws:iam::546683855188:instance-profile/dev-MyInstanceProfile-8Y3JNGL82DFS",
  "InstanceProfileId" : "AIPAI2F5MGWJMDEXAMPLE"
}

I have tried using 1.13.0, 1.14.0 and 1.15.0. I do pip install --upgrade dynamic-dynamodb.

Can it be a problem with boto?

$ pip list | grep boto
boto (2.29.1)
botocore (0.42.0)

[ulsa]

Or is it simply a missing sns:Publish permission on my instance profile? In that case, I think the error message from boto is misleading.

[ulsa]

Got the same problem on another machine running aws-ec2-assign-elastic-ip. I think it's a bug in boto.

[ulsa]

Yep, boto 2.28.0 works, but 2.29.0 and 2.29.1 both fail.

[clarkenheim]

@ulsa I dont suppose you could tell me how to install boto version 2.28.0 while this bug is fixed in 2.29.0?

Thanks

[sebdah]

I can release a forced version lock in Dynamic DynamoDB later today. Will
update here when it's released.

Br,
Sebastian Dahlgren

Sent from my phone
On Jun 5, 2014 4:11 PM, "Charlie Clark" notifications@github.com wrote:

@ulsa https://github.com/ulsa I dont suppose you could tell me how to
install boto version 2.28.0 while this bug is fixed in 2.29.0?

Thanks

Reply to this email directly or view it on GitHub
#183 (comment)
.

[clarkenheim]

excellent ;)

[ulsa]

Meanwhile, you can install using:

pip install boto==0.28.0

[sebdah]

This is fixed and released in 1.15.1.

[danielgtaylor]

@sebdah, I'm one of the maintainers of Boto.

Boto 2.29.x fixed a bug where invalid AWS credential profile names were silently ignored (specifically boto/boto@af80944 as part of the AWS shared credential support), and I believe that this change has highlighted a bug in Dynamic DynamoDB. The line in the stack trace above that looks suspect to me is:

  File "/usr/local/lib/python2.7/dist-packages/dynamic_dynamodb/aws/sns.py", line 106, in __get_connection_SNS
    profile_name=metadata['iam']['info'][u'InstanceProfileArn'])

You seem to be creating an SNS connection and passing in profile_name as an ARN. The profile_name parameter defines the specific credential profile name to use from your configuration file, for example:

~/.boto

[Credentials]
aws_access_key_id = 'my-default-key'
aws_secret_access_key = 'my-default-secret'

[profile foo]
aws_access_key_id = 'my-foo-key'
aws_secret_access_key = 'my-foo-secret'

You would use it like so:

>>> import boto.sns
>>> sns = boto.sns.connect_to_region('us-east-1', profile='foo')
>>> sns.aws_access_key_id
'my-foo-key'

I would strongly advise against forcing an older version of Boto as a solution to this issue, and instead fix the SNS instantiation in Dynamic DynamoDB to not pass an ARN to the profile_name parameter.

[sebdah]

Thanks for pointing this out @danielgtaylor!

I'll reimplement this so that the connection handlers fetch and use the credentials rather than the profile ARN from the metadata.

[sebdah]

Further discussion and coding on this will be done in #188.