Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AWS IAM MSK V2 authentication failure when session token expired #976

Closed
kikyomits opened this issue Aug 30, 2022 · 1 comment
Closed

AWS IAM MSK V2 authentication failure when session token expired #976

kikyomits opened this issue Aug 30, 2022 · 1 comment
Assignees
Labels

Comments

@kikyomits
Copy link
Contributor

kikyomits commented Aug 30, 2022

Describe the bug

I contributed to msk_iam_v2 module on the other day but our team recently identified a bug with that.

Currently, we give aws.Credentials to msk_iam_v2.Mechanism but looks aws.Credentials doesn't automatically refresh the session secret. While looking at AWS GO SDK V2 code, I should have provided aws.CredentialsProvider to Mechanism.

Kafka Version

  • What version(s) of Kafka are you testing against?
  • What version of kafka-go are you using?
  • github.com/segmentio/kafka-go v0.4.34
  • github.com/segmentio/kafka-go/sasl/aws_msk_iam_v2 v0.0.0-20220809022639-fcb5875e8e6a
  • MSK: 2.8.1

To Reproduce

  1. Run ECS task and consume events from MSK with using msk_iam_v2 module.
  2. Make sure ECS Task Role has proper permission to authenticate with MSK. (probably you can reproduce this by using EC2 with instance profile.)
  3. Initially, successfully consuming events from MSK until AWS session token is expired. 6 hours by default.
  4. Leave it for 6 hours (default session token expiration is 6 hours)
  5. kafka-go failed to get authenticated with this error
    could not successfully authenticate to <prefix>.kafka.ap-southeast-2.amazonaws.com:9098 with SASL: [58] SASL Authentication Failed: SASL Authentication failed

The sample configuration of SASLMechanism with msk_iam_v2 is below

import "github.com/aws/aws-sdk-go-v2/config"
import signer "github.com/aws/aws-sdk-go-v2/aws/signer/v4"

awsCfg, _ := config.LoadDefaultConfig(ctx)
creds, _ := awsCfg.Credentials.Retrieve(ctx)
dialer := &kafka.Dialer{
	Timeout:   10 * time.Second,
	DualStack: true,
	SASLMechanism: &aws_msk_iam_v2.Mechanism{
		Signer: signer.NewSigner(),
		Credentials: creds,
		Region: "ap-southeast-2",
	},
	TLS: &tls.Config{},
}

Expected Behavior

Automatic rotation of session token as Go SDK V2 does.

Observed Behavior

Failed after session token is expired.

The error message is:

could not successfully authenticate to <prefix>.kafka.ap-southeast-2.amazonaws.com:9098 with SASL: [58] SASL Authentication Failed: SASL Authentication failed

Additional Context

I will make another contribution for this bug fix this week.

@kikyomits kikyomits added the bug label Aug 30, 2022
kikyomits added a commit to kikyomits/kafka-go that referenced this issue Aug 30, 2022
Use `CredentialsProvider` instead of static `Credentials`. See segmentio#976
kikyomits added a commit to kikyomits/kafka-go that referenced this issue Aug 30, 2022
Use `CredentialsProvider` instead of static `Credentials`. See segmentio#976
kikyomits added a commit to kikyomits/kafka-go that referenced this issue Aug 30, 2022
Use `CredentialsProvider` instead of static `Credentials`. See segmentio#976
kikyomits added a commit to kikyomits/kafka-go that referenced this issue Aug 30, 2022
Use `CredentialsProvider` instead of static `Credentials`. See segmentio#976
kikyomits added a commit to kikyomits/kafka-go that referenced this issue Aug 30, 2022
Use `CredentialsProvider` instead of static `Credentials`. See segmentio#976
@achille-roussel achille-roussel self-assigned this Sep 2, 2022
kikyomits added a commit to kikyomits/kafka-go that referenced this issue Sep 3, 2022
Use `CredentialsProvider` instead of static `Credentials`. See segmentio#976
kikyomits added a commit to kikyomits/kafka-go that referenced this issue Sep 3, 2022
Use `CredentialsProvider` instead of static `Credentials`. See segmentio#976
kikyomits added a commit to kikyomits/kafka-go that referenced this issue Sep 10, 2022
Use `CredentialsProvider` instead of static `Credentials`. See segmentio#976
achille-roussel pushed a commit that referenced this issue Sep 12, 2022
Use `CredentialsProvider` instead of static `Credentials`. See #976
@kikyomits
Copy link
Contributor Author

kikyomits commented Sep 15, 2022

The resolution for this issue had been merged to main, resolved. #977

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants