fix(dynamodb): BETWEEN-aware + whitespace-tolerant AND/OR tokenizer#670
Merged
vieiralucas merged 1 commit intomainfrom Apr 22, 2026
Merged
fix(dynamodb): BETWEEN-aware + whitespace-tolerant AND/OR tokenizer#670vieiralucas merged 1 commit intomainfrom
vieiralucas merged 1 commit intomainfrom
Conversation
- Rewrite split_on_top_level_keyword around a keyword-boundary helper
(match_keyword). Alphanumeric keywords (AND, OR, BETWEEN) require
ASCII-whitespace word boundaries so ':s\tAND\t:o' and ':s\nAND\n:o'
split like ':s AND :o'. Punctuation keywords (',') keep matching
literally.
- Track a top-level BETWEEN skip counter so 'x BETWEEN :lo AND :hi'
no longer has its inner AND mistaken for a top-level separator.
- Unignore 3 tests: key_condition_between,
key_condition_whitespace_variations, filter_between.
- Add unit tests for split_on_top_level_keyword covering BETWEEN skip,
whitespace variants, case-insensitivity, and identifier substrings.
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
This was referenced Apr 22, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
" AND "/" OR "match. That broke forBETWEEN :lo AND :hi(the inner AND got treated as a top-level separator) and for any non-space whitespace between tokens (tabs, newlines, multi-space).split_on_top_level_keywordaround a keyword-boundary helper (match_keyword). Alphanumeric keywords (AND/OR/BETWEEN) now require ASCII-whitespace word boundaries; punctuation keywords (,) still match literally.BETWEENskip counter sox BETWEEN :lo AND :hino longer has its inner AND consumed as a top-level separator.key_condition_betweenkey_condition_whitespace_variationsfilter_betweensplit_on_top_level_keywordcovering BETWEEN skip, whitespace variants, case-insensitive keyword matching, and identifier substrings (e.g.landmust not matchAND).Test plan
cargo test -p fakecloud-dynamodb --lib(all 213 pass, 3 ignored remaining for follow-up PRs)cargo test -p fakecloud-e2e --test dynamodb(all 46 pass)cargo clippy -p fakecloud-dynamodb --all-targets -- -D warningscargo fmt --allSummary by cubic
Fixes DynamoDB expression splitting so top-level AND/OR no longer break on BETWEEN’s inner AND and tolerate tabs/newlines/multi-space. Re-enables impacted tests and adds targeted unit tests for the tokenizer.
x BETWEEN :lo AND :hidoesn’t split on the inner AND.land), and respect parentheses.match_keywordhelper and refactorsplit_on_top_level_keyword.key_condition_between,key_condition_whitespace_variations,filter_between; add new tokenizer unit tests.Written for commit 6fbf67a. Summary will update on new commits.