Merge with upstream v20231116 (#30)

* Update CHANGELOG.md for v20230703 AMI release (awslabs#1337) * Update CHANGELOG.md for v20230703 AMI release * Update CHANGELOG.md Co-authored-by: Carter <mckdev@amazon.com> * Update CHANGELOG.md --------- Co-authored-by: Carter <mckdev@amazon.com> * Update CHANGELOG.md (awslabs#1338) * Add logging for aws managed csi drivers (awslabs#1336) * Update CHANGELOG.md latest AMI release notes to highlight this was last 1.22 AMI (awslabs#1342) * Removing 1.22 from Makefile (awslabs#1343) * Generate version info for cached images only when is active (awslabs#1341) * Remove region names from us-iso/us-isob credential provider config (awslabs#1344) * Amazon Linux 2023 proof-of-concept (awslabs#1340) * Remove hardcoded pull_cni_from_github var (awslabs#1346) * Remove sonobuoy_e2e_registry (awslabs#1249) * Revert "avoid hard coding provisioner index array" (awslabs#1347) This reverts commit 6c16765. Signed-off-by: Davanum Srinivas <davanum@gmail.com> * Update sync-eni-max-pods.yaml role ARN (awslabs#1350) * Add CodeCommit sync action (awslabs#1351) * update core CNI plugins version (awslabs#1308) * Update internal build config (awslabs#1353) * Update binary references (awslabs#1355) * Update CHANGELOG.md for 20230711 AMI release (awslabs#1357) * Enable discard_unpacked_layers by default (awslabs#1360) * Mount bpffs on all supported Kubernetes versions (awslabs#1349) * Cleanup /var/log/audit (awslabs#1363) * Use GitHub bot user as committer/author (awslabs#1366) * Update eni-max-pods.txt (awslabs#1365) * Update CHANGELOG.md for 20230728 AMI release (awslabs#1371) * Update eni-max-pods.txt (awslabs#1373) Co-authored-by: GitHub <noreply@github.com> * Install latest amazon-ssm-agent from S3 (awslabs#1370) * Do not set KubeletCredentialProviders feature flag for 1.28+ (awslabs#1375) * Fix bug in var doc gen (awslabs#1378) * Generate docs for GitHub Pages (awslabs#1379) * Add write permissions to deploy-docs workflow (awslabs#1381) * Force-push docs to gh-pages (awslabs#1382) * Cache IMDS tokens per-user (awslabs#1386) * Install latest runc 1.1.* (awslabs#1384) * Update eni-max-pods.txt (awslabs#1388) * Update binary build dates (awslabs#1390) * Fetch new IMDS token for every request (awslabs#1395) * Update CHANGELOG for v20230816 (awslabs#1396) * Update eni-max-pods.txt (awslabs#1397) * Update Makefile with latest binaries (awslabs#1403) * Add CI bot (awslabs#1402) * Disable janitor in forks (awslabs#1407) * Add note about bot authorization (awslabs#1406) * noproxy for direct communication to apiserver and timeouts of 3 seconds (awslabs#1393) * Update CHANGELOG.md for 20230825 AMI release (awslabs#1408) * Update CHANGELOG.md for 20230825 AMI release --------- Co-authored-by: Vela WU <50354807+FerrelWallis@users.noreply.github.com> * Allow --reserved-cpus kubelet arg to be used (awslabs#1405) * Install kernel-headers, kernel-devel (awslabs#1302) * Handle eventually-consistent PrivateDnsName (awslabs#1383) * Add .git-commit to archivebuild (awslabs#1411) * Use archivebuild-wrapper system (awslabs#1413) * Discover .git-commit from environment (awslabs#1418) * Update eni-max-pods.txt (awslabs#1423) Co-authored-by: GitHub <noreply@github.com> * Update eni-max-pods.txt (awslabs#1424) Co-authored-by: GitHub <noreply@github.com> * Require builder instance to use IMDSv2 (awslabs#1422) * Add release note config (awslabs#1426) * Update eni-max-pods.txt (awslabs#1429) Co-authored-by: GitHub <noreply@github.com> * Use 2023-09-14 binaries, add 1.28 target (awslabs#1431) * Update eni-max-pods.txt (awslabs#1432) Co-authored-by: GitHub <noreply@github.com> * Set pid_max to 4194304 (awslabs#1434) * Install nerdctl (awslabs#1321) * Update CHANGELOG.md for 20230919 AMI release (awslabs#1439) * Update CHANGELOG.md for 20230919 AMI release Co-authored-by: Carter <cartermckinnon@gmail.com> --------- Co-authored-by: Carter <cartermckinnon@gmail.com> * bump latest Kubernetes build target version (awslabs#1440) * fix: Tag cached image with the ECR URI for the target region (awslabs#1442) * Add H100 into gpu clock (awslabs#1447) * bug: incorrect region variable name (awslabs#1449) Co-authored-by: ljosyula <ljosyula@amazon.com> * Update eni-max-pods.txt (awslabs#1452) Co-authored-by: GitHub <noreply@github.com> * Update CHANGELOG.md for 20231002 AMI release (awslabs#1456) Co-authored-by: ljosyula <ljosyula@amazon.com> * Build with latest binaries by default (awslabs#1391) * Fix region in cached image names (awslabs#1461) * Add 1.28 to CI (awslabs#1464) * Add optional FIPS support (awslabs#1458) * Set remote_folder on all shell provisioners (awslabs#1462) * Pull eksctl supported versions for CI (awslabs#1465) * remove kubernetes versions file and use eksctl supported version list * recognize compression Co-authored-by: Carter <cartermckinnon@gmail.com> --------- Co-authored-by: Carter <cartermckinnon@gmail.com> * Add CHANGELOG entry placeholder (awslabs#1466) * Add named arguments to bot commands (awslabs#1463) * get-ecr-uri.sh falls back to use another region in partition if region unconfigured (awslabs#1468) * Force delete CI clusters, don't wait for pod eviction (awslabs#1472) * Add CHANGELOG workflow for new releases (awslabs#1467) * Allow more flexible kernel_version (awslabs#1469) * Add r7i to eni-max-pods.txt (awslabs#1473) Co-authored-by: GitHub <noreply@github.com> * Fix containerd slice configuration (awslabs#1437) * Correctly tag cached images for us-gov-west-1 FIPS endpoint (awslabs#1476) * Lint space errors (awslabs#1121) * Ignore commit to address space errors (awslabs#1478) * Collect more info about Amazon VPC CNI (awslabs#1245) * Update eni-max-pods.txt (awslabs#1485) Co-authored-by: GitHub <noreply@github.com> * Fail fast if we cannot determine kubelet version (awslabs#1484) kubelet is likely to fail when there is a mismatch with GLIBC that is in the image vs the one golang uses to build the kubelet. So fail the image right away when this happens as this specific kubelet binary will NOT work in any instance started with this image. ``` 2023-10-25T10:11:38-04:00: amazon-ebs: kubelet: /lib64/libc.so.6: version `GLIBC_2.32' not found (required by kubelet) 2023-10-25T10:11:38-04:00: amazon-ebs: kubelet: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by kubelet) ``` Signed-off-by: Davanum Srinivas <davanum@gmail.com> * Persist CI version-info.json as artifact (awslabs#1493) * Add new i4i sizes to eni-max-pods.txt (awslabs#1495) Co-authored-by: GitHub <noreply@github.com> * Update eni-max-pods.txt (awslabs#1497) Co-authored-by: GitHub <noreply@github.com> * Drop the FIPS related provisioners for al2023 (awslabs#1499) Signed-off-by: Davanum Srinivas <davanum@gmail.com> * Set nerdctl default namespace to k8s.io (awslabs#1488) * Update CHANGELOG.md for release v20231027 (awslabs#1502) Co-authored-by: GitHub <noreply@github.com> * Skip installing amazon-ssm-agent if already present (awslabs#1501) * Exclude automated eni-max-pods.txt PR's from release notes (awslabs#1498) * Remove extraneous space character (awslabs#1505) * Update CHANGELOG.md (awslabs#1507) * Update CHANGELOG.md to fix docker version (awslabs#1511) * Update docker to the latest 20.10 version (awslabs#1510) * Changelog entry format tweaks (awslabs#1508) * Document how to collect UserData (awslabs#1504) * Update Fluence changelog * Update what kubernetes ami will be build --------- Signed-off-by: Davanum Srinivas <davanum@gmail.com> Co-authored-by: Xavier Ryan <108886506+xr1776@users.noreply.github.com> Co-authored-by: Carter <mckdev@amazon.com> Co-authored-by: jacobwolfaws <113703057+jacobwolfaws@users.noreply.github.com> Co-authored-by: Prasad Shende <prasad0896@users.noreply.github.com> Co-authored-by: camrakin <113552683+camrakin@users.noreply.github.com> Co-authored-by: Davanum Srinivas <davanum@gmail.com> Co-authored-by: Jeffrey Nelson <jdnelson@amazon.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Sichaow <sichaow@amazon.com> Co-authored-by: GitHub <noreply@github.com> Co-authored-by: Vincent Marguerie <24724195+vincentmrg@users.noreply.github.com> Co-authored-by: Andrew Johnstone <andrew@ajohnstone.com> Co-authored-by: Vela WU <50354807+wwvela@users.noreply.github.com> Co-authored-by: Vela WU <50354807+FerrelWallis@users.noreply.github.com> Co-authored-by: Raghvendra Singh <90425886+raghs-aws@users.noreply.github.com> Co-authored-by: Matthew Wong <mattwon@amazon.com> Co-authored-by: Nick Baker <ndbaker1@outlook.com> Co-authored-by: ddl-retornam <56278673+ddl-retornam@users.noreply.github.com> Co-authored-by: Carter <cartermckinnon@gmail.com> Co-authored-by: Bryant Biggs <bryantbiggs@gmail.com> Co-authored-by: Laxmi Soumya Josyula <42261978+ljosyula@users.noreply.github.com> Co-authored-by: ljosyula <ljosyula@amazon.com> Co-authored-by: Alex Schultz <aschultz@clumio.com> Co-authored-by: Julien Baladier <julienbaladier@users.noreply.github.com> Co-authored-by: Matt <merkes@amazon.com> Co-authored-by: Zoltán Reegn <zoltan.reegn@gmail.com> Co-authored-by: donovanrost <donovan.rost@gmail.com> Co-authored-by: guessi <guessi@gmail.com> Co-authored-by: pjaudiomv <34245618+pjaudiomv@users.noreply.github.com> Co-authored-by: Edmond Ceausu <eceausu@amazon.com>
AdvMicrogrid · Dec 6, 2023 · 3da49ea · 3da49ea
1 parent 0fa3d92
commit 3da49ea
Show file tree

Hide file tree

Showing 77 changed files with 3,189 additions and 396 deletions.
diff --git a/.circleci/config.yml b/.circleci/config.yml
@@ -31,7 +31,7 @@ jobs:
           # By default, Circle CI have 10 minutes timeout without response,
           # so we must extend that timeout limit to be sure that ami build can pass
           no_output_timeout: 60m
-          command: cd aws_ecr_ami && PACKER_VARIABLE_FILE=fluence-eks-worker-al2-variables.json make -e -j2 1.23
+          command: cd aws_ecr_ami && PACKER_VARIABLE_FILE=fluence-eks-worker-al2-variables.json make -e -j2 1.24
 workflows:
   aws_eks_ami:
     jobs:

diff --git a/.git-blame-ignore-revs b/.git-blame-ignore-revs
@@ -1,3 +1,4 @@
 # Applied code style rules to shell files
 6014c4e6872a23f82ca295afa93b033207042876
-
+# Addressed space errors
+bde408b340d992aad39e13de1aaf929f358f4338
diff --git a/.github/actions/bot/.gitignore b/.github/actions/bot/.gitignore
@@ -0,0 +1 @@
+node_modules/
diff --git a/.github/actions/bot/README.md b/.github/actions/bot/README.md
@@ -0,0 +1,21 @@
+# bot
+
+This GitHub Action parses commands from pull request comments and executes them.
+
+Only authorized users (members and owners of this repository) are able to execute commands.
+
+Commands look like `/COMMAND ARGS`, for example:
+```
+/echo hello world
+```
+
+Multiple commands can be included in a comment, one per line; but each command must be unique.
+
+Some commands accept additional, named arguments specified on subsequent lines.
+Named arguments look like `+NAME ARGS`, for example:
+```
+/ci launch
++build cache_container_images=true
+```
+
+Multiple named arguments can be specified.
diff --git a/.github/actions/bot/action.yaml b/.github/actions/bot/action.yaml
@@ -0,0 +1,13 @@
+name: "Bot"
+description: "🤖 beep boop"
+runs:
+  using: "composite"
+  steps:
+    - uses: "actions/checkout@v3"
+    - uses: "actions/github-script@v6"
+      with:
+        script: |
+          const crypto = require('crypto');
+          const uuid = crypto.randomUUID();
+          const bot = require('./.github/actions/bot/index.js');
+          await bot(core, github, context, uuid);
diff --git a/.github/actions/bot/index.js b/.github/actions/bot/index.js
@@ -0,0 +1,213 @@
+// this script cannot require/import, because it's called by actions/github-script.
+// any dependencies must be passed in the inline script in action.yaml
+
+async function bot(core, github, context, uuid) {
+    const payload = context.payload;
+
+    if (!payload.comment) {
+        console.log("No comment found in payload");
+        return;
+    }
+    console.log("Comment found in payload");
+
+    // user's org membership must be public for the author_association to be MEMBER
+    // go to the org's member page, find yourself, and set the visibility to public
+    const author = payload.comment.user.login;
+    const authorized = ["OWNER", "MEMBER"].includes(payload.comment.author_association);
+    if (!authorized) {
+        console.log(`Comment author is not authorized: ${author}`);
+        return;
+    }
+    console.log(`Comment author is authorized: ${author}`);
+
+    let commands;
+    try {
+        commands = parseCommands(uuid, payload, payload.comment.body);
+    } catch (error) {
+        console.log(error);
+        const reply = `@${author} I didn't understand [that](${payload.comment.html_url})! 🤔\n\nTake a look at my [logs](${getBotWorkflowURL(payload, context)}).`
+        replyToCommand(github, payload, reply);
+        return;
+    }
+    if (commands.length === 0) {
+        console.log("No commands found in comment body");
+        return;
+    }
+    const uniqueCommands = [...new Set(commands.map(command => typeof command))];
+    if (uniqueCommands.length != commands.length) {
+        replyToCommand(github, payload, `@${author} you can't use the same command more than once! 🙅`);
+        return;
+    }
+    console.log(commands.length + " command(s) found in comment body");
+
+    for (const command of commands) {
+        const reply = await command.run(author, github);
+        if (typeof reply === 'string') {
+            replyToCommand(github, payload, reply);
+        } else if (reply) {
+            console.log(`Command returned: ${reply}`);
+        } else {
+            console.log("Command did not return a reply");
+        }
+    }
+}
+
+// replyToCommand creates a comment on the same PR that triggered this workflow
+function replyToCommand(github, payload, reply) {
+    github.rest.issues.createComment({
+        owner: payload.repository.owner.login,
+        repo: payload.repository.name,
+        issue_number: payload.issue.number,
+        body: reply
+    });
+}
+
+// getBotWorkflowURL returns an HTML URL for this workflow execution of the bot
+function getBotWorkflowURL(payload, context) {
+    return `https://github.com/${payload.repository.owner.login}/${payload.repository.name}/actions/runs/${context.runId}`;
+}
+
+// parseCommands splits the comment body into lines and parses each line as a command or named arguments to the previous command.
+function parseCommands(uuid, payload, commentBody) {
+    const commands = [];
+    if (!commentBody) {
+        return commands;
+    }
+    const lines = commentBody.split(/\r?\n/);
+    for (const line of lines) {
+        console.log(`Parsing line: ${line}`);
+        const command = parseCommand(uuid, payload, line);
+        if (command) {
+            commands.push(command);
+        } else {
+            const namedArguments = parseNamedArguments(line);
+            if (namedArguments) {
+                const previousCommand = commands.at(-1);
+                if (previousCommand) {
+                    if (typeof previousCommand.addNamedArguments === 'function') {
+                        previousCommand.addNamedArguments(namedArguments.name, namedArguments.args);
+                    } else {
+                        throw new Error(`Parsed named arguments but previous command (${previousCommand.constructor.name}) does not support arguments: ${JSON.stringify(namedArguments)}`);
+                    }
+                } else {
+                    // don't treat this as an error, because the named argument syntax might just be someone '+1'-ing.
+                    console.log(`Parsed named arguments with no previous command: ${JSON.stringify(namedArguments)}`);
+                }
+            }
+        }
+    }
+    return commands
+}
+
+// parseCommand parses a line as a command.
+// The format of a command is `/NAME ARGS...`.
+// Leading and trailing spaces are ignored.
+function parseCommand(uuid, payload, line) {
+    const command = line.trim().match(/^\/([a-z\-]+)(?:\s+(.+))?$/);
+    if (command) {
+        return buildCommand(uuid, payload, command[1], command[2]);
+    }
+    return null;
+}
+
+// buildCommand builds a command from a name and arguments.
+function buildCommand(uuid, payload, name, args) {
+    switch (name) {
+        case "echo":
+            return new EchoCommand(uuid, payload, args);
+        case "ci":
+            return new CICommand(uuid, payload, args);
+        default:
+            console.log(`Unknown command: ${name}`);
+            return null;
+    }
+}
+
+// parseNamedArgument parses a line as named arguments.
+// The format of a command is `+NAME ARGS...`.
+// Leading and trailing spaces are ignored.
+function parseNamedArguments(line) {
+    const parsed = line.trim().match(/^\+([a-z\-]+)(?:\s+(.+))?$/);
+    if (parsed) {
+        return {
+            name: parsed[1],
+            args: parsed[2]
+        }
+    }
+    return null;
+}
+
+class EchoCommand {
+    constructor(uuid, payload, args) {
+        this.phrase = args ? args : "echo";
+    }
+
+    run(author) {
+        return `@${author} *${this.phrase}*`;
+    }
+}
+
+class CICommand {
+    constructor(uuid, payload, args) {
+        this.repository_owner = payload.repository.owner.login;
+        this.repository_name = payload.repository.name;
+        this.pr_number = payload.issue.number;
+        this.comment_url = payload.comment.html_url;
+        this.uuid = uuid;
+        this.goal = "test";
+        // "test" goal, which executes all CI stages, is the default when no goal is specified
+        if (args != null && args != "") {
+            this.goal = args;
+        }
+        this.goal_args = {};
+    }
+
+    addNamedArguments(goal, args) {
+        this.goal_args[goal] = args;
+    }
+
+    async run(author, github) {
+        const pr = await github.rest.pulls.get({
+            owner: this.repository_owner,
+            repo: this.repository_name,
+            pull_number: this.pr_number
+        });
+        const mergeable = pr.data.mergeable;
+        switch (mergeable) {
+            case true:
+                break;
+            case false:
+            case null:
+                return `@${author} this PR is not currently mergeable, you'll need to rebase it first.`;
+            default:
+                throw new Error(`Unknown mergeable value: ${mergeable}`);
+        }
+        const inputs = {
+            uuid: this.uuid,
+            pr_number: this.pr_number.toString(),
+            git_sha: pr.data.merge_commit_sha,
+            goal: this.goal,
+            requester: author,
+            comment_url: this.comment_url
+        };
+        for (const [goal, args] of Object.entries(this.goal_args)) {
+            inputs[`${goal}_arguments`] = args;
+        }
+        console.log(`Dispatching workflow with inputs: ${JSON.stringify(inputs)}`);
+        await github.rest.actions.createWorkflowDispatch({
+            owner: this.repository_owner,
+            repo: this.repository_name,
+            workflow_id: 'ci-manual.yaml',
+            ref: 'master',
+            inputs: inputs
+        });
+        return null;
+    }
+}
+
+
+module.exports = async (core, github, context, uuid) => {
+    bot(core, github, context, uuid).catch((error) => {
+        core.setFailed(error);
+    });
+}