feat(cloud): TOON re-encoder for JSON-array outputs#4
Closed
jhonatanjunio wants to merge 1 commit into
Closed
Conversation
…, gcloud, az) Adds `strategies::toon` — a zero-dep encoder that re-emits JSON arrays of uniform flat objects as TOON (Token-Oriented Object Notation, https://github.com/toon-format/toon). TOON declares the schema once in a header and lists rows CSV-style, typically saving 40-60% on tokens for this shape. `CloudHandler` calls the encoder when the command requests JSON output (`--json`, `-o json`, `--output json`, `--format=json`, etc.) and pipes through to the standard filter pipeline otherwise. The encoder is strictly lossless-or-reject: it returns None whenever the input deviates from a top-level array of flat uniform objects (mismatched keys, key ordering, nested values, empty array, non-array root, or when the TOON output would not actually be smaller). Closes claudioemmanuel#128 Signed-off-by: Jhonatan Junio <jhonatanjuniocp@gmail.com>
02f4b83 to
5202d61
Compare
Owner
Author
|
Superseded by upstream PR — moved to claudioemmanuel/squeez. See README of this fork for the upstream link. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3
Summary
Adds a Token-Oriented Object Notation (TOON, https://github.com/toon-format/toon) re-encoder applied to JSON-array outputs from
gh,kubectl,aws,gcloud, andaz. Reduces tokens on this shape by 40-60% by declaring the schema once in a header and listing rows CSV-style.Why
gh pr list --json …,kubectl get pods -o json,aws ec2 describe-instances --output json, etc. all return arrays of uniform objects — exactly the shape TOON was designed for. TodayCloudHandleronly strips noise; the heavy redundant field-name repetition stays. Free 40-60%.What changed
src/strategies/toon.rs(~340 LoC incl. tests).try_to_toon(text: &str) -> Option<String>— hand-rolled JSON sub-parser + TOON emitter, zero-dep.Nonefor any non-uniform / nested / out-of-order / unsmaller input.CloudHandlercalls it when the command requests JSON (--json,-o json,-ojson,--output json,--output=json,--format json,--format=json,-f json); otherwise pipes through unchanged.commands::cloud::tests::detects_json_request_flagsverifies the detector across CLI conventions.Worked example
Input (raw
gh pr list --json number,title,state):[ {"number": 1, "title": "Add feature", "state": "OPEN"}, {"number": 2, "title": "Fix bug", "state": "MERGED"}, {"number": 3, "title": "Refactor", "state": "CLOSED"} ]Output (TOON):
Constraints honoured
Cargo.tomlunchanged).Noneif TOON would not be smaller than the input — never makes things worse.Tests
13 new unit tests:
TOON encoder (12):
encodes_uniform_gh_pr_listencodes_kubectl_pods_shapequotes_values_with_commasrejects_heterogeneous_keysrejects_keys_in_different_orderrejects_nested_objectsrejects_nested_arraysrejects_empty_arrayrejects_non_array_rootrejects_when_toon_not_smallerhandles_nulls_and_booleans_and_floatsdecodes_json_string_escapesCloudHandler dispatch (1):
detects_json_request_flagsFull lib suite: 172/172 passing (160 prior + 12 + 1).
Out of scope
docker ps --format '{{json .}}'— follow-up.cargo metadata --format-version 1— nested by design, not worth TOON.