Feat 1817 add iam auth to mysql and redis by sgress454 · Pull Request #32488 · fleetdm/fleet

sgress454 · 2025-09-01T15:35:45Z

Details

This PR gives Fleet servers the ability to connect to RDS MySQL and Elasticache Redis via AWS Identity and Access Management (IAM). It is based almost entirely on the work of @titanous, branched from his original pull request. The main differences between his branch and this are:

Removal of auto-detection of AWS region (and cache name for Elasticache) in favor of specifying these values in configuration. The auto-detection is admittedly handy but parsing AWS host URLs is not considered a best practice.
Relying on the existence of these new configs to determine whether or not to connect via IAM. This sidesteps a thorny issue of whether to try an IAM-based Elasticache connection when a password is not supplied, since this is technically a valid setup.

Checklist for submitter

If some of the following don't apply, delete the relevant line.

Changes file added for user-visible changes in changes/, orbit/changes/ or ee/fleetd-chrome/changes.
See Changes files for more information.

Testing

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

…17-add-iam-auth-to-mysql-and-redis

Why? I opened a separate docs PR to merge into the docs branch: https://github.com/fleetdm/fleet/pull/32219/files

codecov · 2025-09-01T15:47:49Z

Codecov Report

❌ Patch coverage is 19.83122% with 190 lines in your changes missing coverage. Please review.
✅ Project coverage is 63.94%. Comparing base (27560c7) to head (abea1f4).
⚠️ Report is 15 commits behind head on main.

Files with missing lines	Patch %	Lines
server/aws_common/iam_auth.go	0.00%	48 Missing ⚠️
...erver/datastore/mysql/common_mysql/aws_iam_auth.go	0.00%	47 Missing ⚠️
server/datastore/redis/aws_iam_auth.go	0.00%	45 Missing ⚠️
server/datastore/mysql/common_mysql/common.go	25.00%	21 Missing and 6 partials ⚠️
server/datastore/redis/redis.go	26.92%	16 Missing and 3 partials ⚠️
cmd/fleet/serve.go	0.00%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #32488      +/-   ##
==========================================
- Coverage   64.02%   63.94%   -0.09%     
==========================================
  Files        1987     1991       +4     
  Lines      195579   195652      +73     
  Branches     6461     6360     -101     
==========================================
- Hits       125221   125100     -121     
- Misses      60560    60746     +186     
- Partials     9798     9806       +8

Flag	Coverage Δ
backend	`65.18% <19.83%> (-0.07%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

titanous · 2025-09-01T15:45:39Z


+	// Auto-detect ElastiCache and use IAM auth if no password is provided
+	useIAMAuth := false
+	if conf.Password == "" && conf.Region != "" && conf.CacheName != "" {


I think the region should be optional, in the vast majority of cases the region will just be whatever the default is from the environment, and here we don't need it because CacheName can be used.

You're right, although we do need a region to generate the auth token. I think if we change this to

region: cfg.Region

it should fall back to the default region if provided.

titanous · 2025-09-01T15:47:40Z

+	// Auto-detect RDS and use IAM auth if no password is provided
+	var iamTokenGen *awsIAMAuthTokenGenerator
+	useIAMAuth := false
+	if conf.Password == "" && conf.PasswordPath == "" && conf.Region != "" {


Not entirely sold on requiring the region, is it required for the other AWS integration features?

It is -- see the Fleet server configuration docs and the various _region configs.

Digging into this deeper, we're doing something similar to what I suggested above where it falls back to the default region if configured (or rather, overrides the default regions in LoadAWSConfig if one is provided).

For MySQL there's a bit of a quandary because, without a flag specifying that you're using RDS, we don't have a good way to determine when to try IAM auth. Your original PR handled this by parsing the server address. This is very handy when it works but introduces an amount of brittleness and edge cases that we'd rather not support. So in the absence of a specific use_iam_auth flag (proposed and rejected due to inconsistency with the rest of the config), the presence of region + absence of password is the best indicator we have that IAM auth is desired.

lucasmrod

LGTM!

Great tooling btw 👌

lucasmrod · 2025-09-03T13:48:04Z

+	m.cacheMu.Lock()
+	defer m.cacheMu.Unlock()
+
+	// Double-check in case another goroutine generated a token while we were waiting


Nit for the future: (no need to change and re-test)
For simplicity, I would have just used a sync.Mutex and just lock the whole GetToken method. Not sure if there are performance gains, and they are negligible next to network times, etc.

lucasmrod · 2025-09-03T14:20:47Z

 		"multiStatements":      []string{"true"},
 	}
-	if conf.TLSConfig != "" {
+	if conf.Password == "" && conf.PasswordPath == "" && conf.Region != "" {


Nit: I would use if useIAMAuth for clarity. Or add a comment (to not invalidate all the testing already done) that we allow clear text passwords because IAM.

def should use useIAMAuth here rather than repeating this logic, will update

Oh wait this is in another method, we haven't set useIAMAuth here. I could make a helper method for it but for the sake of not running the tests again, I'll leave this be.

lucasmrod · 2025-09-03T15:51:30Z

+	}
+
 	if conf.UseTLS {
+		var err error


Nit, not needed AFAICS.

titanous and others added 21 commits July 24, 2025 18:26

Add generic caching IAM token generator for RDS/ElastiCache

0df8b03

Add support for AWS ElastiCache (Redis) IAM auth

0ef576d

Add support for AWS RDS MySQL IAM auth

d31f4e1

Update fleet-server-configuration.md

46e05a1

Update fleet-server-configuration.md

3ac8ab7

Update docs/Configuration/fleet-server-configuration.md

e860a43

Update fleet-server-configuration.md

6e8b900

Update tools/redis-tests/elasticache/cleanup-test-env.sh

523fd5d

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

Update tools/mysql-tests/rds/deploy-test-env.sh

508bda2

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

Update server/datastore/redis/gen_aws_region_map.go

f35e74c

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

Update fleet-server-configuration.md

0c92378

Update docs/Configuration/fleet-server-configuration.md

bf18b15

Merge branch 'aws-iam-auth' of github.com:titanous/fleet into feat-18…

a1ca083

…17-add-iam-auth-to-mysql-and-redis

Revert server config docs back to main (#32221)

3cea469

Why? I opened a separate docs PR to merge into the docs branch: https://github.com/fleetdm/fleet/pull/32219/files

fallback to iam

a3179ea

cleanup

b0f03fe

don't auto-detect region and cache

6b14db1

revert back to simpler logic

6429e82

remove mysql region / proxy discovery

d0c963b

fix config

eab2fa7

changelog

5f2d555

sgress454 requested a review from a team as a code owner September 1, 2025 15:35

sgress454 temporarily deployed to Docker Hub September 1, 2025 15:35 — with GitHub Actions Inactive

titanous reviewed Sep 1, 2025

View reviewed changes

sgress454 added 2 commits September 2, 2025 08:23

add to yaml

065466c

lint

82bc93a

sgress454 had a problem deploying to Docker Hub September 2, 2025 13:36 — with GitHub Actions Error

update tools

f285ce2

sgress454 temporarily deployed to Docker Hub September 2, 2025 13:42 — with GitHub Actions Inactive

sgress454 assigned lucasmrod Sep 2, 2025

sgress454 linked an issue Sep 2, 2025 that may be closed by this pull request

IAM authentication for MySQL and Redis #1817

Closed

30 tasks

lucasmrod approved these changes Sep 3, 2025

View reviewed changes

Merge branch 'main' into feat-1817-add-iam-auth-to-mysql-and-redis

abea1f4

sgress454 temporarily deployed to Docker Hub September 3, 2025 18:17 — with GitHub Actions Inactive

sgress454 merged commit 602f5a4 into main Sep 4, 2025
40 checks passed

sgress454 deleted the feat-1817-add-iam-auth-to-mysql-and-redis branch September 4, 2025 15:08

getvictor mentioned this pull request Dec 7, 2025

AWS IAM auth code in common_mysql increases complexity #36846

Closed

1 task

Conversation

sgress454 commented Sep 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Details

Checklist for submitter

Testing

Uh oh!

codecov Bot commented Sep 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lucasmrod left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

sgress454 commented Sep 1, 2025 •

edited

Loading

codecov Bot commented Sep 1, 2025 •

edited

Loading