Skip to content

v2.4.0

Choose a tag to compare

@jeffyaw jeffyaw released this 20 Sep 21:29
· 44 commits to main since this release
v2.4.0
f18b99b

Security

  • The display command no longer runs anything when pasted into PowerShell. Every result carries a command string built for a reader to paste, and a value containing a single quote made it executable. shellQuoteArg emitted the POSIX close-escape-reopen idiom; PowerShell reads '...' as literal text the way POSIX does but has no backslash escape inside it, so that form tokenises into separate words -- a ; in the value ends the statement, whatever follows it RUNS, and a # comments out the dangling quote that would otherwise be a parse error. Measured on win32/arm64 with PowerShell 5.1, driving the real call against a loopback stub: a --query value of x'; echo PWNED # printed PWNED. The other 24 character-class probes were delivered byte-identically in PowerShell and in Git Bash, so a single quote in the value is the whole trigger -- and a --query such as Buckets[?Name=='prod'].Name is an ordinary way to get one. Quoting is now chosen by the platform the reader is on: Windows emits PowerShell's own escape, a doubled quote, which is also inert in bash ('a''b' is concatenation), so a Git Bash paste yields a wrong value rather than a running command. command is still a display string, not a universal one: it is correct in a POSIX shell and in PowerShell, and NOT in cmd.exe, where &, | and a newline are live whatever the quoting (measured 0 of 24 probes correct, and unchanged by this fix). A %VAR% is now quoted rather than emitted bare, so a batch file no longer expands it away. Re-quote the string yourself before pasting it into a shell other than the server's own.
  • The 0600 on the two files that can hold credentials is verified now rather than assumed, and a filesystem that cannot honour it fails the call instead of leaking. The --cli-input-json params file (which can carry a SecureString) and ~/.aws/credentials (which carries an access key, a secret key and a session token in plaintext) were each opened at 0600 and then fchmod-ed at 0600 -- open's mode because Node honours it, the fchmod because oam drops it. Neither is sufficient: a filesystem may accept a chmod, report success, and change nothing. Measured on WSL Ubuntu (linux/arm64, Node 22.23.2) against a Windows drive under /mnt (v9fs/DrvFs): mkdtemp returns 0777, the exclusive create returns 0777, and fchmodSync(fd, 0o600) succeeds while changing nothing -- so both files landed world-readable and world-writable, with nothing able to detect it. World-writable is the worse half: the CLI opens the params path after this server has closed it, so another local user can substitute the payload in between. Neither configuration is exotic -- TMPDIR into a Windows drive is how you share one scratch directory between a WSL distro and its Windows host, and AWS_SHARED_CREDENTIALS_FILE (botocore's own variable, named in aws_assume_role's description) pointed at the Windows-side ~/.aws is how you share one credentials file and SSO cache. The mode is now checked with one fstat, and a file that cannot be made private is not written at all: the call fails naming the variable to change (TMPDIR/TMP/TEMP, or AWS_SHARED_CREDENTIALS_FILE) before any secret byte reaches disk. This is a failure you can newly hit: with either path on such a mount, calls that used to appear to succeed now fail with that message. That is the intended trade, because the previous behaviour was a silent exposure. CIFS/SMB, FAT-family and NTFS mounts under fmask/dmask, and Docker Desktop bind mounts, are expected to behave the same way -- reasoned, not measured. No POSIX measurement of oam exists anywhere, since oam publishes no linux-arm64 build and this work had no Mac, so every oam filesystem row in this release was measured on Windows.
  • The AWS CLI is resolved to an absolute path before every call, and never from the working directory, on any platform. 2.3.3 closed the Windows case by setting NoDefaultCurrentDirectoryInExePath on the server process, which left the protection resting on the server's own environment and did nothing for a PATH that names the working directory itself: on macOS and Linux an empty PATH entry or . means the current directory to execvp. The server now walks the absolute directories on the child environment's PATH itself -- aws.exe only on Windows, a regular executable file elsewhere -- and spawns the path it found. Every entry that would resolve against a directory the server does not control is skipped rather than searched: empty, ., a relative bin, a drive-relative C:foo, a root-relative \tools, an unexpanded %X%\bin. Each Windows aws child also gets NoDefaultCurrentDirectoryInExePath=1 explicitly, so the CLI's own helpers do not resolve there either -- aws ssm start-session hands off to session-manager-plugin by bare name, and a credential_process can name a bare command too. There is no bare-name fallback: when no CLI is found nothing runs, because falling back is exactly the planted-checkout case on a machine without the CLI. Resolving the path also settles a runtime difference -- oam searches its own install directory before the parent PATH while Node searches the working directory -- so Node and oam now run the same binary. The walk is not cached and does not need to be: 0.47-1.16 ms over this host's 58-entry PATH (20 runs, median 0.56) against a CLI start of hundreds of milliseconds, and a cache would go stale after an install or an aws update.

Fixed

  • aws_logs_tail returns structured events against a real AWS CLI -- for the first time since it shipped. It wrapped aws logs tail --format json on the premise that the output is NDJSON. It never was: that flag selects the CLI's pretty-print formatter, which prints <iso-timestamp> <stream> <message> lines and re-indents any message that is entirely JSON across several lines (awscli/customizations/logs/tail.py, byte-identical from 2.34.3 to the v2 head). The first line never parsed, so every non-empty call took the raw-string fallback: events was one text blob, eventCount and totalEvents were null, truncated was false, and the maxEvents bound added in 2.2.0 never applied -- on aws-cli 2.34.3 against a loopback endpoint a six-event window came back as a 637-character string, and a 2,000-event window with maxEvents: 5 as 329,998 characters. Only an empty window was right. Nor can that text be parsed back into events: a message beginning with a timestamp reads as a new event, a stream name with a space cannot be told from the message after it, a traceback's continuation lines carry no prefix, and a JSON message has already been re-serialized. The tool now calls FilterLogEvents through aws logs filter-log-events -- the same API tail calls -- whose JSON output is a single document, and returns events as {timestamp, logStreamName, message}: timestamp in ISO 8601 UTC with milliseconds, message verbatim rather than re-indented and right-stripped. The parser is strict now, which is the pin that keeps this from recurring: stdout that is not the document the fixed --query asked for is ok: false naming what arrived, where the old parser turned text into ok: true. That removes an arm the Stability section documented -- events as the raw blob with eventCount and totalEvents both null -- so events is always an array now. It is not held for a major because every non-empty call against a real CLI landed there: the shape being removed is the bug itself, not an output any caller could have gotten a real event out of. The tail scenarios in the fake are replaced by payloads captured from the real CLI, and a new suite drives the handler through the INSTALLED aws binary against an in-process endpoint on every npm test -- ten cases, four of which also assert the fake returns the identical envelope -- so a fake that drifts from the real format fails instead of hiding the next one.
  • A log-group ARN now addresses the group it names. The ARN used to be cut down to its bare name, so a source-account ARN silently read the same-named group in the caller's own account, and an ARN for another region read the call's region instead. It is now sent as FilterLogEvents' logGroupIdentifier (a trailing :* removed), so a cross-account read works from a CloudWatch cross-account monitoring account; an ARN whose region differs from the call's region is refused before anything spawns, with both regions named, because FilterLogEvents is regional; and an AWS CLI older than 2.9.15, which cannot address a group by ARN, gets an error saying so -- naming the ARN's own account and the bare-name workaround -- instead of a silent retarget.
  • Every aws call pins the CLI settings that change what this server reads. AWS CLI 2.34.0 added cli_error_format; set to json, yaml, text or table, it removes the An error occurred (Code) when calling ... line the error classifier reads, so an expired or rejected credential came back as nonzero_exit with no remedy instead of expired_creds / invalid_creds (measured on 2.34.3 against a loopback stub answering 403). cli_auto_prompt = on failed every call with exit 255 before a connection was opened -- aws sso login included, so aws_login_start could not work at all. Both are now pinned through the environment (AWS_CLI_ERROR_FORMAT=enhanced, the CLI's own default, and AWS_CLI_AUTO_PROMPT=off), not with flags: AWS CLI 2.22.0 answers --cli-error-format with Unknown options and exit 252, while an environment variable it does not know is ignored. The pins go on top of whatever environment the caller built rather than instead of it, so aws_multi_account's per-account credentials and the AWS_MAX_ATTEMPTS=1 that keeps aws_lambda_invoke to one attempt both survive them.
  • Non-ASCII output no longer breaks calls on Windows. Writing to a pipe, the Windows CLI uses the ANSI code page. A response with any character outside it -- an emoji or a check mark in a log line, a CJK S3 key -- failed the whole call with 'charmap' codec can't encode characters (exit 255), aws_logs_tail included, and characters inside it came back as U+FFFD with ok: true: for an e-acute the CLI wrote a lone cp1252 0xE9, which this server's UTF-8 decode cannot keep. The server now sets AWS_CLI_OUTPUT_ENCODING=utf-8 (which exists from 2.24.14) and PYTHONUTF8=1 (which is what covers older CLIs) on every aws process, and a real-CLI case round-trips an e-acute, CJK and an astral-plane character exactly on 2.34.3 and on 2.22.0. The test suite's fake CLI always wrote UTF-8, which is why this never showed up there.
  • Request params over ~32 KB work on Windows. They travelled as one command-line argument, and past Windows' 32,767-character limit the spawn failed with spawn ENAMETOOLONG. Is the AWS CLI installed and on PATH? -- sending you to debug a PATH that was fine. A CloudFormation template body (up to 51,200 bytes), a Step Functions definition, an SSM document and a dynamodb batch-write-item all reach that size legitimately. Params over 8,192 characters now go through a private temp file (--cli-input-json file://..., mode 0600 -- verified, not assumed, see the Security note below -- in a directory made for that one call and removed when the call ends), written as ASCII-escaped JSON: the CLI reads a file:// param as text in the locale's preferred encoding, so a plain UTF-8 file holding caf + e-acute reached the endpoint mojibaked, with exit 0 -- silent corruption of the request. If the temp directory itself holds a $ or %, or starts with ~, the call fails as bad_input naming TMP/TEMP/TMPDIR, because the CLI expands those in a file:// path. A single value that is still too long (a very large Cloud Control desired state, say) now fails as bad_input naming the flag and its length, on Windows (ENAMETOOLONG) and Linux (E2BIG) alike, rather than as a spawn_failure blaming PATH.
  • A timeoutMs above 2,147,483,647 (about 24.8 days) no longer times the call out at once: node stores a timer delay in a signed 32-bit int and fires a longer one after 1 ms, so runAwsCall now clamps it to the maximum rather than passing it through.

Changed

  • On arm64 Linux, AWS_MCP_RUNTIME=oam no longer advises installing something that does not exist. oam publishes darwin arm64/x64, windows arm64/x64 and linux x64 -- there is no linux-arm64 asset -- so on a Pi, an arm64 cloud instance or WSL on an ARM Windows host, discovery can never succeed and "Install or update from oamjs.org" sent the reader to a releases page with nothing on it for their machine. Those hosts are now told there is no build for their platform, and pointed at AWS_MCP_RUNTIME=node or OAM_BIN. Every other platform's message is unchanged. The fallback itself was already right: measured on linux/arm64 against a real POSIX build, a bare launch serves 28 tools on Node, a bad OAM_BIN prints its diagnostic and then serves, and AWS_MCP_RUNTIME=oam exits without serving.
  • On AWS CLI 2.35.8+, aws_logs_tail stops reading once it has the newest maxEvents -- and a truncated result reports totalEvents: null. This is a behavior change, not merely a fix: aws logs tail read every page in the window before the cap applied, so a busy hour ended in output_too_large or the 60s timeout after paying for every FilterLogEvents call. CLI 2.35.8 added FilterLogEvents' startFromHead; with it the tool reads newest-first and asks for maxEvents + 1 events -- one sentinel, which makes truncated exact without a NextToken that FilterLogEvents documents as over-reporting -- so a busy group costs a page or two instead of the whole window (a 1,200-event window measured one request). It therefore no longer knows the window's full size: totalEvents is null whenever truncated is true on that path. No existing caller can have seen a number there, because against a real CLI totalEvents was always null for a non-empty window. Older CLIs reject startFromHead before sending anything (exit 252, zero requests, measured on 2.34.3 and 2.22.0), and the tool reads the rejection itself rather than guessing at a version -- in 2.34.3's wording, in 2.22.0's, which prints the validation lines with no ParamValidation header, and in the CLI's JSON error format. It remembers that answer for the life of the process, so a current CLI never pays for the detection, and reads the whole window as before -- exact totalEvents -- but asks the CLI to print only the newest maxEvents plus a count, so even there a busy window no longer dies on the 5 MB output cap (780 KB unprojected against 130 KB projected, for 2,000 events). An endpoint that ACCEPTS startFromHead and ignores it is caught too: moto's FilterLogEvents never reads the member, so LocalStack does not either, and an ascending page that was also truncated triggers a whole-window re-read instead of presenting the window's oldest events as its newest.
  • aws_logs_tail sends every caller-supplied value inside --cli-input-json, where the CLI never expands file://. 2.3.3 refused a filterPattern starting with file:// or fileb:// rather than let the CLI replace it with a local file's contents; the payload is passed through literally, so such a pattern now reaches CloudWatch as the text it is. The reject on a leading - stays, so nothing else about the accepted input set changes.
  • Blob-typed params are always base64. This is a behavior change for one configuration. With cli_binary_format = raw-in-base64-out in your AWS config -- common advice, AWS's own docs suggest it for aws lambda invoke --payload -- a base64 value such as KMS Plaintext or Kinesis Data was encoded a second time and AWS stored the base64 text instead of the bytes, silently: measured on 2.34.3 and 2.22.0, a dynamodb put-item with B: "aGVsbG8=" put YUdWc2JHOD0= on the wire. Calls that carry params now pass --cli-binary-format base64, the CLI's own default and the one such setting with no environment variable to pin instead. If you were passing raw text to a blob parameter under that config, it now fails with Invalid base64 instead of succeeding; send base64. The flag is scoped to calls that carry params, so a command string without them is unchanged -- and AWS CLI v1, which this server does not support, has no such global option at all: those calls now fail with Unknown options, and the error says it is v1 rather than leaving you to read the flag name.
  • On AWS CLIs older than 2.25.0 ON WINDOWS, ~/.aws/config and ~/.aws/credentials are now read as UTF-8. A side effect of PYTHONUTF8=1 (above): those CLIs are frozen with a PyInstaller build that honors it for the whole interpreter, not for output alone. It costs nothing on macOS or Linux, because there is no ANSI code page to switch away from: measured on linux/arm64 with aws-cli 2.36.49, a cp1252 byte in ~/.aws/config fails to parse identically with PYTHONUTF8 unset, =0 and =1, and under LC_ALL=C and LC_ALL=POSIX, while the same character encoded as UTF-8 parses in all of them -- so the pin changes nothing there, and such a file was already unreadable on those platforms before this server existed. A non-ASCII character saved in a legacy Windows code page there now fails with Unable to parse config file and exit 255 -- re-save the file as UTF-8, or update the CLI -- while a UTF-8 file holding a character the code page lacks, which those CLIs rejected, now parses; both directions are asserted against 2.22.0 in the real-CLI suite. A Python-based credential_process the CLI starts inherits the variable too. The trade was taken deliberately: without the pin, every Windows install fails or corrupts on any non-ASCII output, while this side effect needs both an old CLI and a legacy-encoded byte in a config file.

Added

  • aws_logs_tail's success envelope carries logGroupIdentifier -- the ARN that was sent, with the trailing :* removed, or null for a bare-name call -- because command redacts the --cli-input-json payload and would otherwise hide whether an ARN input was honored.
  • AWS_MCP_AWS_CLI: point the server at a specific aws executable. For MCP hosts started from a GUI that do not inherit your shell's PATH, now more common since AWS's recommended installer puts the CLI in ~/.local/bin on macOS and Linux. It applies to every call and to aws sso login, which the old test-only override never reached. It must be absolute, and on Windows must name an .exe: node refuses to start a .cmd or .bat by path without a shell, while oam runs the same path through cmd.exe. An unusable value fails every call with a message naming the variable rather than quietly running a different CLI than you configured -- this setting picks which binary handles your credentials. When no CLI can be found at all, the error now says how many PATH directories were checked, that the working directory is never searched, how to find the path (where.exe aws / command -v aws), and names an aws.cmd or aws.bat shim if that is what it found, since a pip-installed AWS CLI v1 leaves one.