feat: add initial partial json stream parser code - #1
Conversation
|
Note Currently processing new changes in this PR. This may take a few minutes, please wait... 📒 Files selected for processing (11)
Tip You can disable poems in the walkthrough.Disable the ✨ Finishing Touches🧪 Generate unit tests✅ Unit Test PR creation complete.
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
|
Note Unit test generation is an Early Access feature. Expect some limitations and changes as we gather feedback and continue to improve it. Generating unit tests... This may take up to 20 minutes. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
🧹 Nitpick comments (6)
.gitignore (1)
1-7: Consider expanding ignores to cover other common artefactsBesides
.dart_tool/andpubspec.lock, most Dart packages also ignore thebuild/,.DS_Store,.idea/, and.packagesfiles/directories. This prevents accidental inclusion of platform-specific or generated artefacts in the repo and plays nicely with CI/CD pipelines.analysis_options.yaml (1)
14-14: Enable stricter lints early to avoid technical debtSticking with the default
recommendedset is fine for a PoC, but a streaming-parser library will quickly benefit from additional safety/robustness rules (e.g.unawaited_futures,use_string_buffers,prefer_final_locals). Consider uncommenting thelinter:section and opting-in to rules that catch concurrency & performance issues before shipping 1.0.0.CHANGELOG.md (1)
1-4: Add release date for traceabilityIncluding the YYYY-MM-DD next to each version header greatly helps consumers track package history.
lib/partial_json_stream_parser.dart (1)
1-10: Name the library for clearer import prefixesThe directive
library;defines an anonymous library. Giving it an explicit name (e.g.library partial_json_stream_parser;) improves tooling support, avoids potential conflicts, and lets users import with a predictable prefix.-library; +library partial_json_stream_parser;lib/src/parse_result.dart (1)
2-13: Implement equality & hashCode for testability
ParseResultwill appear heavily in unit tests; overriding==/hashCode(and possibly providingcopyWith) makes equality assertions concise and less error-prone.class ParseResult { ... const ParseResult(this.value, this.remaining); + @override + bool operator ==(Object other) => + identical(this, other) || + other is ParseResult && + runtimeType == other.runtimeType && + value == other.value && + remaining == other.remaining; + + @override + int get hashCode => Object.hash(value, remaining);README.md (1)
233-233: Consider more direct phrasing.The static analysis tool flagged "Feel free to" as potentially unprofessional. Consider a more direct alternative:
-Contributions are welcome! Feel free to submit a Pull Request. +Contributions are welcome! Please submit a Pull Request.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (11)
.gitignore(1 hunks)CHANGELOG.md(1 hunks)README.md(1 hunks)analysis_options.yaml(1 hunks)example/partial_json_stream_parser_example.dart(1 hunks)lib/partial_json_stream_parser.dart(1 hunks)lib/src/parse_result.dart(1 hunks)lib/src/parser_exception.dart(1 hunks)lib/src/partial_json_parser.dart(1 hunks)pubspec.yaml(1 hunks)test/partial_json_stream_parser_test.dart(1 hunks)
🧰 Additional context used
🪛 LanguageTool
README.md
[style] ~233-~233: Consider using a less common alternative to make your writing sound more unique and professional.
Context: ...ontributing Contributions are welcome! Feel free to submit a Pull Request. ## License Thi...
(FEEL_FREE_TO_STYLE_ME)
🔇 Additional comments (20)
lib/partial_json_stream_parser.dart (1)
8-10: Re-export public surface onlyIf
src/partial_json_parser.dartcontains implementation details not meant for public API, expose a facade instead of re-exporting the entire file. Otherwise future internal refactors become breaking changes.pubspec.yaml (3)
1-12: LGTM! Well-structured package metadata.The package metadata is comprehensive and follows Dart package conventions. The description clearly states the purpose and target use cases, and the topics are relevant for discoverability.
17-21: Excellent dependency management.Zero runtime dependencies is perfect for a core parsing library, reducing potential conflicts. The dev dependencies (lints and test) are appropriate for maintaining code quality and testing.
14-15: No changes needed for the Dart SDK constraintThe caret constraint
^3.6.2permits any Dart 3.x release ≥ 3.6.2 (and < 4.0.0), including all current and upcoming minor versions. It only excludes patch releases prior to 3.6.2, which can be upgraded easily. Given that your package targets modern streaming/LLM use cases and likely depends on features introduced in 3.6.2 or later, this SDK constraint is appropriate as-is.lib/src/parser_exception.dart (1)
1-20: LGTM! Well-designed exception class.The
PartialJsonExceptionclass follows Dart best practices with:
- Clear field separation (message, input, position)
- Efficient
constconstructor- Comprehensive
toString()method for debugging- Proper use of
StringBufferfor string concatenationThe design provides excellent context for debugging parsing errors.
test/partial_json_stream_parser_test.dart (8)
1-42: LGTM! Well-organized test structure.Excellent test organization with:
- Proper imports and setup
- Clear grouping of related tests
- Good use of
setUp()for parser initialization- Comprehensive baseline tests for complete JSON
The test structure follows Dart testing best practices.
43-73: Comprehensive incomplete object testing.The tests thoroughly cover incomplete object scenarios with sensible expectations:
- Missing values default to
null- Incomplete keys are handled gracefully
- Edge cases like empty objects are tested
The fallback behavior is intuitive and user-friendly.
101-137: Thorough string handling tests.Excellent coverage of string parsing edge cases:
- Both strict and non-strict mode testing
- Proper handling of escape sequences and newlines
- Incomplete escape sequence handling
- Edge cases with escaped quotes
Critical for robust streaming JSON parsing.
170-174: Verify the incomplete negative number behavior.The test expects
{"value": -to parse as{'value': '-'}(string). This behavior might be unexpected - users might expectnullor a number instead of a string containing just the minus sign.Consider if this is the intended behavior or if it should return
nulllike other incomplete values.
295-317: LGTM! Excellent callback testing.The extra token callback test is well-structured:
- Properly captures callback parameters
- Verifies correct data is passed to callback
- Tests important functionality for handling trailing content
This ensures the callback feature works as documented.
319-349: Valuable real-world scenario testing.These tests excellently simulate actual LLM streaming scenarios:
- Progressive chunk parsing
- Complex response structures
- Realistic JSON formats from chat completions
Perfect for validating the primary use case of this library.
266-293: Comprehensive edge case coverage.Excellent testing of boundary conditions:
- Empty inputs and minimal JSON structures
- Mixed complete/incomplete nested elements
- Critical cases that often break parsers
This coverage significantly improves parser robustness.
246-264: Important whitespace tolerance testing.Good coverage of whitespace handling:
- Leading/trailing whitespace tolerance
- Whitespace between tokens
- Newlines and tabs in formatting
Essential for handling real-world JSON with varied formatting.
example/partial_json_stream_parser_example.dart (4)
1-44: Excellent basic usage examples.The examples effectively demonstrate:
- Core incomplete JSON parsing functionality
- Streaming simulation with progressive chunks
- Clear expected outputs for each case
Perfect for getting users started quickly.
46-116: Comprehensive advanced examples.Excellent demonstration of advanced features:
- Complex nested structures
- Various incomplete data types
- Callback mechanism usage
- Realistic LLM tool call scenarios
These examples show the library's full capabilities.
117-134: Valuable progressive parsing demonstration.This example excellently shows real-world usage:
- Realistic incremental data reception
- Proper error handling
- Practical application of the streaming capability
Great for understanding how to integrate with streaming APIs.
136-178: Well-implemented helper functions.The utility functions enhance example clarity:
- Proper recursive pretty printing
- Clean value formatting
- Concise result summarization
Good separation of concerns and reusable utility code.
README.md (2)
1-167: Excellent comprehensive documentation.The README provides outstanding coverage:
- Clear feature benefits with practical examples
- Complete installation and usage instructions
- Relevant real-world use cases (LLMs, WebSocket, progressive UI)
- Examples that align with the actual API
Perfect for helping users understand and adopt the library.
169-205: Clear and complete API reference.The API documentation effectively covers:
- Constructor parameters and their purposes
- Method signatures and return types
- Result object structure
- Concise but complete information
Well-structured reference material for developers.
lib/src/partial_json_parser.dart (1)
25-43: Well-structured parser initializationThe character-based dispatch map is an efficient approach for determining which parser to use. Good use of late final initialization.
| if (s.isEmpty) { | ||
| return {}; | ||
| } |
There was a problem hiding this comment.
Reconsider empty string handling
Returning an empty object {} for empty input might be unexpected. Consider returning null or throwing an exception to distinguish between empty input and an actual empty object.
if (s.isEmpty) {
- return {};
+ return null; // or throw an appropriate exception
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if (s.isEmpty) { | |
| return {}; | |
| } | |
| if (s.isEmpty) { | |
| return null; // or throw an appropriate exception | |
| } |
🤖 Prompt for AI Agents
In lib/src/partial_json_parser.dart around lines 49 to 51, the current code
returns an empty object {} when the input string is empty, which can be
misleading. Modify the code to either return null or throw an exception for
empty input to clearly differentiate between empty input and a valid empty
object. Choose one approach and update the return statement or add error
handling accordingly.
| final keyResult = _parseAny(s, e); | ||
| final key = keyResult.value.toString(); | ||
| s = keyResult.remaining.trim(); |
There was a problem hiding this comment.
Validate that object keys are strings
JSON specification requires object keys to be strings. Converting arbitrary values to strings using toString() could mask errors.
// Parse key
final keyResult = _parseAny(s, e);
-final key = keyResult.value.toString();
+if (keyResult.value is! String) {
+ throw FormatException('Object key must be a string');
+}
+final key = keyResult.value as String;📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| final keyResult = _parseAny(s, e); | |
| final key = keyResult.value.toString(); | |
| s = keyResult.remaining.trim(); | |
| // Parse key | |
| final keyResult = _parseAny(s, e); | |
| if (keyResult.value is! String) { | |
| throw FormatException('Object key must be a string'); | |
| } | |
| final key = keyResult.value as String; | |
| s = keyResult.remaining.trim(); |
🤖 Prompt for AI Agents
In lib/src/partial_json_parser.dart around lines 125 to 127, the code converts
the parsed key to a string using toString() without validating its type, which
violates the JSON specification requiring object keys to be strings. Modify the
code to check if keyResult.value is actually a string before assigning it to
key. If it is not a string, throw a descriptive error or handle the invalid key
appropriately instead of converting it implicitly.
| // Non-strict mode: return raw content | ||
| if (incompleteEscapeRegex.hasMatch(content)) { | ||
| return ParseResult(content, ''); | ||
| } | ||
| return ParseResult(content, ''); | ||
| } else { |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Remove duplicate return statement
The condition check for incomplete escape is redundant as both branches return the same value.
if (!strict) {
// Non-strict mode: return raw content
- if (incompleteEscapeRegex.hasMatch(content)) {
- return ParseResult(content, '');
- }
return ParseResult(content, '');
}🤖 Prompt for AI Agents
In lib/src/partial_json_parser.dart around lines 194 to 199, there is a
redundant condition check for incompleteEscapeRegex.hasMatch(content) because
both the if and else branches return the same ParseResult(content, ''). Remove
the if condition and the duplicate return statement, keeping only a single
return ParseResult(content, '') to simplify the code.
| if (numStr.isEmpty || numStr == '-' || numStr == '.') { | ||
| return ParseResult(numStr, ''); | ||
| } |
There was a problem hiding this comment.
Type inconsistency in return value
Returning a string for incomplete numbers is inconsistent with the expected numeric return type. Consider returning null or 0 for incomplete numbers.
// Handle incomplete numbers
if (numStr.isEmpty || numStr == '-' || numStr == '.') {
- return ParseResult(numStr, '');
+ return ParseResult(null, ''); // or ParseResult(0, '')
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if (numStr.isEmpty || numStr == '-' || numStr == '.') { | |
| return ParseResult(numStr, ''); | |
| } | |
| // Handle incomplete numbers | |
| if (numStr.isEmpty || numStr == '-' || numStr == '.') { | |
| return ParseResult(null, ''); // or ParseResult(0, '') | |
| } |
🤖 Prompt for AI Agents
In lib/src/partial_json_parser.dart around lines 273 to 275, the function
returns a string when the number string is incomplete, which conflicts with the
expected numeric return type. Modify the return statement to return null or 0
instead of the string numStr for these cases to maintain type consistency.
| if (s.toLowerCase().startsWith('t')) { | ||
| // Handle incomplete 'true' | ||
| if (s.length < 4) { | ||
| return ParseResult(true, ''); | ||
| } | ||
| return ParseResult(true, s.substring(4)); | ||
| } | ||
| throw e; |
There was a problem hiding this comment.
JSON literals are case-sensitive
JSON specification requires true to be lowercase. Using toLowerCase() accepts invalid JSON.
ParseResult _parseTrue(String s, Exception e) {
- if (s.toLowerCase().startsWith('t')) {
+ if (s.startsWith('t')) {
// Handle incomplete 'true'
if (s.length < 4) {
return ParseResult(true, '');
}
return ParseResult(true, s.substring(4));
}
throw e;
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if (s.toLowerCase().startsWith('t')) { | |
| // Handle incomplete 'true' | |
| if (s.length < 4) { | |
| return ParseResult(true, ''); | |
| } | |
| return ParseResult(true, s.substring(4)); | |
| } | |
| throw e; | |
| ParseResult _parseTrue(String s, Exception e) { | |
| if (s.startsWith('t')) { | |
| // Handle incomplete 'true' | |
| if (s.length < 4) { | |
| return ParseResult(true, ''); | |
| } | |
| return ParseResult(true, s.substring(4)); | |
| } | |
| throw e; | |
| } |
🤖 Prompt for AI Agents
In lib/src/partial_json_parser.dart around lines 297 to 304, the code
incorrectly uses toLowerCase() to check for the 'true' literal, which violates
JSON's case sensitivity. Replace the toLowerCase() check with a strict
comparison to the exact lowercase string 'true'. Ensure that the parser only
accepts 'true' in lowercase and rejects any other casing to comply with the JSON
specification.
| if (s.toLowerCase().startsWith('f')) { | ||
| // Handle incomplete 'false' | ||
| if (s.length < 5) { | ||
| return ParseResult(false, ''); | ||
| } | ||
| return ParseResult(false, s.substring(5)); | ||
| } | ||
| throw e; |
There was a problem hiding this comment.
JSON literals are case-sensitive
JSON specification requires false to be lowercase. Using toLowerCase() accepts invalid JSON.
ParseResult _parseFalse(String s, Exception e) {
- if (s.toLowerCase().startsWith('f')) {
+ if (s.startsWith('f')) {
// Handle incomplete 'false'
if (s.length < 5) {
return ParseResult(false, '');
}
return ParseResult(false, s.substring(5));
}
throw e;
}🤖 Prompt for AI Agents
In lib/src/partial_json_parser.dart around lines 309 to 316, the code
incorrectly uses toLowerCase() to check for the 'false' literal, which violates
JSON's case sensitivity. Remove the toLowerCase() call and instead check if the
string starts exactly with 'false' in lowercase. Adjust the length check and
substring extraction accordingly to only accept the valid lowercase literal.
|
✅ UTG Post-Process Complete No new issues were detected in the generated code and all check runs have completed. The unit test generation process has completed successfully. |
|
Creating a PR to put the unit tests in... The changes have been created in this pull request: View PR |
1 participant
Summary by CodeRabbit
New Features
Tests