Skip to content

aws: update support for additional regions#11733

Open
ShelbyZ wants to merge 2 commits intofluent:masterfrom
ShelbyZ:aws-regions
Open

aws: update support for additional regions#11733
ShelbyZ wants to merge 2 commits intofluent:masterfrom
ShelbyZ:aws-regions

Conversation

@ShelbyZ
Copy link
Copy Markdown
Contributor

@ShelbyZ ShelbyZ commented Apr 21, 2026

Summary

AWS region mapping handles existing regions, China, GovCloud, and recently was updated to support EU sovereign cloud, but lacks dedicated cloud. This set of changes aims to update the existing regions supported to cover a wider selection by adding missing mappings. Customers can currently override the ENDPOINT parameter to work around this limitation, but it would be nicer to just handle it based on the supplied REGION parameter.

Testing
Before we can approve your change; please submit the following in a comment:

  • [N/A] Example configuration file for the change
  • [N/A] Debug log output from testing the change
  • [N/A] Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

  • [N/A] Run local packaging test showing all targets (including any new ones) build.
  • [N/A] Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

  • Documentation required for this feature

fluent/fluent-bit-docs#2552

Backporting

  • [N/A] Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

Summary by CodeRabbit

  • Bug Fixes

    • Improved handling when AWS service or region inputs are missing, preventing invalid endpoint generation
  • New Features

    • Broadened endpoint resolution to support China regions, EU Sovereign Cloud, and multiple isolated/sovereign regional endpoints
  • Tests

    • Expanded tests covering many region formats and domain suffix mappings, with added null-input assertions

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 21, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: ae029ce3-0805-4197-a6f1-b88f4b0bd4f0

📥 Commits

Reviewing files that changed from the base of the PR and between bb4c047 and 29fe77e.

📒 Files selected for processing (2)
  • src/aws/flb_aws_util.c
  • tests/internal/aws_util.c
🚧 Files skipped from review as they are similar to previous changes (2)
  • tests/internal/aws_util.c
  • src/aws/flb_aws_util.c

📝 Walkthrough

Walkthrough

Replaced bespoke region checks with a static prefix→suffix lookup table to determine AWS endpoint domain suffixes; flb_aws_endpoint() now returns NULL for NULL inputs. Tests expanded to cover NULL inputs and multiple non-commercial/isolated AWS partitions and regional patterns.

Changes

Cohort / File(s) Summary
AWS Endpoint Selection Logic
src/aws/flb_aws_util.c
Replaced hardcoded region checks with a static prefix→suffix table (endpoint_suffixes[]) and prefix match resolution (first-match). flb_aws_endpoint() returns NULL when service or region is NULL. Removed previous bespoke CN/eusc logic and consolidated error-path resource handling.
AWS Endpoint Tests
tests/internal/aws_util.c
Added assertions that flb_aws_endpoint() returns NULL for NULL inputs; expanded expected endpoint assertions for commercial, China (.amazonaws.com.cn), EU Sovereign (.amazonaws.eu), and isolated S3 partitions (C2S, SC2S, CSP, ADC-E). Ensures allocated strings are freed after each assertion.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

Suggested reviewers

  • edsiper
  • patrick-stephens
  • niedbalski
  • sparrc
  • cosmo0920
  • celalettin1286

Poem

🐰
I hopped through regions, sniffed each name,
Found CN, EU, and clouds of fame.
A table maps the lands I see,
Endpoints now fit tidy and free—
Hooray for tidy domain acclaim!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The pull request title 'aws: update support for additional regions' directly reflects the main change: expanding AWS region mapping support with new endpoint suffixes and domain-suffix handling.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@ShelbyZ ShelbyZ changed the title Aws regions aws: update support for additional regions Apr 21, 2026
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bb4c04712e

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/aws/flb_aws_util.c
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/aws/flb_aws_util.c (1)

573-590: ⚠️ Potential issue | 🟠 Major

Detach or destroy the HTTP client before releasing its connection.

Line 578 releases u_conn while returning c without first detaching it, and the error path releases u_conn before destroying c. Since flb_http_client_detach_connection() restores the connection’s original network setup, keep that cleanup while the request still owns the connection.

Proposed lifecycle-ordering fix
     if (ret != 0 && c != NULL) {
         flb_http_client_destroy(c);
         c = NULL;
     }
 
+    if (c != NULL) {
+        flb_http_client_detach_connection(c);
+    }
+
     flb_upstream_conn_release(u_conn);
     flb_sds_destroy(signature);
     return c;
 
 error:
-    if (u_conn) {
-        flb_upstream_conn_release(u_conn);
-    }
     if (signature) {
         flb_sds_destroy(signature);
     }
     if (c) {
         flb_http_client_destroy(c);
     }
+    if (u_conn) {
+        flb_upstream_conn_release(u_conn);
+    }
     return NULL;
 }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/aws/flb_aws_util.c` around lines 573 - 590, Return and error paths
currently release the upstream connection (u_conn) before detaching or
destroying the HTTP client (c); call flb_http_client_detach_connection(c) if
available or flb_http_client_destroy(c) while the client still owns the
connection, then flb_upstream_conn_release(u_conn), and finally clean up
signature (flb_sds_destroy(signature)) before returning; ensure both the
normal-return branch (when ret != 0) and the error: branch follow this order and
null out c/u_conn after release to avoid double-free/use-after-free.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Outside diff comments:
In `@src/aws/flb_aws_util.c`:
- Around line 573-590: Return and error paths currently release the upstream
connection (u_conn) before detaching or destroying the HTTP client (c); call
flb_http_client_detach_connection(c) if available or flb_http_client_destroy(c)
while the client still owns the connection, then
flb_upstream_conn_release(u_conn), and finally clean up signature
(flb_sds_destroy(signature)) before returning; ensure both the normal-return
branch (when ret != 0) and the error: branch follow this order and null out
c/u_conn after release to avoid double-free/use-after-free.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 0978ee13-fb38-4b71-b541-2a7b5efabe01

📥 Commits

Reviewing files that changed from the base of the PR and between 29deec9 and bb4c047.

📒 Files selected for processing (2)
  • src/aws/flb_aws_util.c
  • tests/internal/aws_util.c

ShelbyZ added 2 commits April 21, 2026 17:30
- add new endpoint suffixes
- use mapping between region prefix to endpoint suffix
- update flb_aws_endpoint to use new mapping

Signed-off-by: Shelby Hagman <shelbyzh@amazon.com>
- update test_flb_aws_endpoint to test additional regions

Signed-off-by: Shelby Hagman <shelbyzh@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant