Skip to content

Add quack-on-ec2, and add cloudformation list improvements#167

Merged
carlopi merged 3 commits into
duckdb:mainfrom
carlopi:cloudformation_list_improvements
Jul 15, 2026
Merged

Add quack-on-ec2, and add cloudformation list improvements#167
carlopi merged 3 commits into
duckdb:mainfrom
carlopi:cloudformation_list_improvements

Conversation

@carlopi

@carlopi carlopi commented Jul 15, 2026

Copy link
Copy Markdown
Member

Two pieces:

  1. add external resource aws:cloudformation:quack-on-ec2 (based on machinery at Add external resource type registry + SQL level (and C++ level) create_external_resource duckdb#23731)
  2. add cloudformation_list_stacks() that perform parallel cross region listing. That combined with filtering allows to list external resources

First piece allows to create resources of type aws:cloudformation:quack-on-ec2, poll for completion, and then

Second piece allows to list cloudformation's stacks, it can be paired with previous features, and could be both of general utility, example:

SELECT region, stack_name, status, creation_time, region_error
      FROM cloudformation_describe_stacks();
     ┌───────────┬───────────────┬─────────────────┬──────────────────────┬──────────────┐
     │  region   │  stack_name   │     status      │    creation_time     │ region_error │
     ├───────────┼───────────────┼─────────────────┼──────────────────────┼──────────────┤
     │ us-east-1 │ quack-mailbox │ CREATE_COMPLETE │ 2026-06-25T15:21:59Z │ NULL         │
     │ us-east-1 │ quack-setup   │ UPDATE_COMPLETE │ 2026-06-26T09:06:13Z │ NULL         │
     └───────────┴───────────────┴─────────────────┴──────────────────────┴──────────────┘

(and indeed, I have at the moment only us-east-1 stacks)

Note that there are now 3 versions of listing:

  • cloudformation_describe_stacks() → all 17 default regions, parallel, skip-on-error (sentinel rows).
  • cloudformation_describe_stacks('us-east-1') → one region, throws on error.
  • cloudformation_describe_stacks(['us-east-1','eu-west-1']) → those regions, parallel, skip-on-error.

@carlopi
carlopi force-pushed the cloudformation_list_improvements branch from b952a08 to 7f7c094 Compare July 15, 2026 07:50
@carlopi
carlopi merged commit 7d6b5be into duckdb:main Jul 15, 2026
11 checks passed
@carlopi

carlopi commented Jul 15, 2026

Copy link
Copy Markdown
Member Author

I think this can go in, still pending is figuring out testing against live credentials (locally and automated on CI)

@carlopi
carlopi deleted the cloudformation_list_improvements branch July 15, 2026 08:39

@Tmonster Tmonster left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to avoid polluting src/. Would be nice if some of this stuff was under src/cloudformation or something

// Default-enabled commercial regions swept by the no-argument overload. Hardcoded for now; live enumeration
// (ec2:DescribeRegions / account:ListRegions) is the accuracy follow-up so newly-enabled opt-in regions are
// not silently missed.
static const char *const AWS_DEFAULT_REGIONS[] = {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should go in an aws utils file.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also there are china regions that are missed here. Maybe they are not "default" ?

Value outputs;
// Empty for a real stack; the AWS error message for a failed-region sentinel row (region set, all stack
// columns NULL). Discriminates the two row kinds: real stacks have region_error IS NULL.
string region_error;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could this just be a generic error I imagine there could be other errors with stacks that are not region related

};

// Fetch all stacks in one region (paginated), appending to `out`. Throws on AWS error.
static void DescribeRegionStacks(const string &region, vector<CloudFormationDescribeStacksRow> &out) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I don't think Region needs to be in the name. Code documentation makes it clear a region must be present

carlopi added a commit to carlopi/duckdb_aws that referenced this pull request Jul 15, 2026
Addresses review feedback on duckdb#167: disentangle the error slot from the
'region' column and leave room for non-region error conditions later. The
sentinel-row semantics are unchanged; only the column/field name differs.
Tmonster pushed a commit that referenced this pull request Jul 16, 2026
Addresses review feedback on #167: the hardcoded region sweep set no longer
lives inline in cloudformation_functions.cpp. A // FIXME records the plan to
replace it with live ec2:DescribeRegions/account:ListRegions enumeration.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants