-
Notifications
You must be signed in to change notification settings - Fork 2
Initial implementation #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Changes from all commits
Commits
Show all changes
27 commits
Select commit
Hold shift + click to select a range
7fc8294
Initial implementation
kaidokert 6964625
Address easy feedback
kaidokert 30b6e50
Bitstack fix
kaidokert 480285b
Missing case handling for truncated floats
kaidokert c06cd7a
Fix misleading error type
kaidokert 80d4490
Fix up errors
kaidokert 629b96c
More error returns
kaidokert 49379ab
Some fixes
kaidokert c503369
Make float default
kaidokert 54313c9
A lot of pending fixes
kaidokert 2161020
Updates
kaidokert 94e2233
No more recursion
kaidokert a6adcac
More followups
kaidokert da8c604
Fix actions
kaidokert a600fbf
More actions fixing
kaidokert b19b9f8
Update README, build
kaidokert 70905a4
Matrix fix
kaidokert 1897274
Float cleanups
kaidokert e1e96f7
Catch the last float config
kaidokert 03ad246
One more float fix
kaidokert 414a252
Fix potential out of bounds access
kaidokert eae1257
More validation range stuff
kaidokert 005210f
More floaty fixy
kaidokert af87a3a
LAst fix ever ?
kaidokert 9b7617f
Cool fix
kaidokert 9a19962
Fix infinite recursion
kaidokert 3eee892
Elide a couple lifetimes
kaidokert File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
* text=auto eol=lf |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,72 @@ | ||
# SPDX-License-Identifier: Apache-2.0 | ||
|
||
on: [push, pull_request] | ||
|
||
name: Build and test | ||
|
||
jobs: | ||
check: | ||
name: Check | ||
runs-on: ubuntu-latest | ||
steps: | ||
- name: Checkout sources | ||
uses: actions/checkout@v4 | ||
|
||
- name: Install stable toolchain | ||
uses: actions-rs/toolchain@v1 | ||
with: | ||
profile: minimal | ||
toolchain: stable | ||
override: true | ||
|
||
- name: Run cargo check | ||
run: cargo check | ||
- name: Run cargo check --no-default-features | ||
run: cargo check --no-default-features | ||
|
||
test: | ||
name: Tests | ||
runs-on: ubuntu-latest | ||
strategy: | ||
fail-fast: false | ||
matrix: | ||
include: | ||
# Default configuration (int64 + float) | ||
- name: Default (int64 + float) | ||
features: "" | ||
|
||
# No features baseline | ||
- name: No features | ||
features: --no-default-features | ||
|
||
# int32 configurations | ||
- name: int32 + float | ||
features: --no-default-features --features "int32,float" | ||
- name: int32 + float-skip | ||
features: --no-default-features --features "int32,float-skip" | ||
- name: int32 + float-error | ||
features: --no-default-features --features "int32,float-error" | ||
- name: int32 + float-truncate | ||
features: --no-default-features --features "int32,float-truncate" | ||
|
||
# int64 configurations (beyond default) | ||
- name: int64 + float-skip | ||
features: --no-default-features --features "int64,float-skip" | ||
- name: int64 + float-error | ||
features: --no-default-features --features "int64,float-error" | ||
- name: int64 + float-truncate | ||
features: --no-default-features --features "int64,float-truncate" | ||
steps: | ||
- name: Checkout sources | ||
uses: actions/checkout@v4 | ||
|
||
- name: Install stable toolchain | ||
uses: actions-rs/toolchain@v1 | ||
with: | ||
profile: minimal | ||
toolchain: stable | ||
override: true | ||
|
||
- name: Run cargo test | ||
working-directory: stax | ||
run: cargo test ${{ matrix.features }} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
repos: | ||
- repo: https://github.com/pre-commit/pre-commit-hooks | ||
rev: v5.0.0 | ||
hooks: | ||
- id: trailing-whitespace | ||
- id: end-of-file-fixer | ||
- id: check-added-large-files | ||
- id: check-merge-conflict | ||
- id: check-json | ||
exclude: | | ||
(?x)^( | ||
.vscode/.*.json | ||
)$ | ||
- id: check-yaml | ||
- id: check-toml | ||
- id: mixed-line-ending | ||
args: ["--fix=lf"] | ||
|
||
- repo: https://github.com/doublify/pre-commit-rust | ||
rev: v1.0 | ||
hooks: | ||
- id: fmt | ||
name: Fmt |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,141 @@ | ||
# Configurable Number Handling | ||
|
||
The stax JSON parser provides comprehensive configurability for number handling, making it suitable for both full-featured and embedded environments. | ||
|
||
## Feature Flags | ||
|
||
### Integer Width | ||
Choose the integer type to avoid pulling in unnecessary math routines: | ||
|
||
- **`int64`** (default): Use `i64` for full range integer support | ||
- **`int32`**: Use `i32` for embedded targets (no 64-bit math routines) | ||
|
||
### Float Support | ||
Control float parsing behavior: | ||
|
||
- **`float`**: Enable full f64 parsing support | ||
- **No float feature**: Disable float parsing (multiple behavior options available) | ||
|
||
### Float Behavior (when `float` feature is disabled) | ||
Choose what happens when floats are encountered: | ||
|
||
- **Default**: Return `FloatDisabled` with raw string preserved for manual parsing | ||
- **`float-error`**: Fail parsing when floats are encountered (embedded fail-fast) | ||
- **`float-truncate`**: Truncate simple decimals to integers (1.7 → 1, errors on scientific notation) | ||
- **`float-skip`**: Skip float values during parsing (continue with next token) [TODO] | ||
|
||
## Configuration Examples | ||
|
||
### Full Featured (Default) | ||
```toml | ||
[dependencies] | ||
stax = { path = "../stax", features = ["int64", "float"] } | ||
``` | ||
- 64-bit integers, full float support | ||
- Best for desktop/server applications | ||
|
||
### Embedded Friendly | ||
```toml | ||
[dependencies] | ||
stax = { path = "../stax", features = ["int32", "float-error"] } | ||
``` | ||
- 32-bit integers (no 64-bit math) | ||
- Error on floats (fail fast) | ||
- Minimal code size for embedded systems | ||
|
||
### Embedded with Float Tolerance | ||
```toml | ||
[dependencies] | ||
stax = { path = "../stax", features = ["int32", "float-truncate"] } | ||
``` | ||
- 32-bit integers | ||
- Truncate simple decimals to integers (1.7 → 1) | ||
- Error on scientific notation (avoids float math) | ||
|
||
### Legacy Float Disabled | ||
```toml | ||
[dependencies] | ||
stax = { path = "../stax", features = ["int64"] } | ||
``` | ||
- 64-bit integers | ||
- Floats return `FloatDisabled` with raw string preserved | ||
- Manual parsing available via `JsonNumber::parse()` | ||
|
||
## API Usage | ||
|
||
All configurations preserve the exact raw string while providing different parsed representations: | ||
|
||
```rust | ||
match event { | ||
Event::Number(num) => { | ||
// Raw string always available (exact precision) | ||
println!("Raw: {}", num.as_str()); | ||
|
||
// Parsed value depends on configuration | ||
match num.parsed { | ||
NumberResult::Integer(i) => println!("Integer: {}", i), | ||
NumberResult::IntegerOverflow => println!("Overflow: {}", num.as_str()), | ||
|
||
#[cfg(feature = "float")] | ||
NumberResult::Float(f) => println!("Float: {}", f), | ||
|
||
#[cfg(all(not(feature = "float"), feature = "float-truncate"))] | ||
NumberResult::FloatTruncated(i) => println!("Truncated: {}", i), | ||
|
||
#[cfg(not(feature = "float"))] | ||
NumberResult::FloatDisabled => { | ||
// Manual parsing still available | ||
let manual: f64 = num.parse().unwrap(); | ||
} | ||
} | ||
|
||
// Convenience methods adapt to configuration | ||
if let Some(int_val) = num.as_int() { | ||
println!("As configured int: {}", int_val); | ||
} | ||
} | ||
} | ||
``` | ||
|
||
## Testing Different Configurations | ||
|
||
Run the demo with different configurations. The truncate mode shows both error and success paths: | ||
|
||
```bash | ||
# Basic no-float (raw strings preserved) | ||
cargo run --example no_float_demo --no-default-features | ||
|
||
# Embedded-friendly with error on floats | ||
cargo run --example no_float_demo --features int32,float-error | ||
|
||
# Embedded with float truncation (demonstrates both error and success scenarios) | ||
cargo run --example no_float_demo --features int32,float-truncate | ||
|
||
# Full featured | ||
cargo run --example no_float_demo --features int64,float | ||
``` | ||
|
||
**Note**: The `float-truncate` configuration demonstrates both successful truncation (with simple decimals) and error handling (with scientific notation) by testing two different JSON inputs. | ||
|
||
## Scientific Notation Handling | ||
|
||
Different configurations handle scientific notation (`1e3`, `2.5e-1`, `1.23e+2`) differently: | ||
|
||
| Configuration | Behavior | Rationale | | ||
|---------------|----------|-----------| | ||
| `float` enabled | Full evaluation: `1e3` → 1000.0 | Complete f64 math available | | ||
| `float-error` | Error: `FloatNotAllowed` | Fail fast on any float syntax | | ||
| `float-truncate` | Error: `InvalidNumber` | Avoid float math entirely | | ||
| Default (disabled) | Raw string: `"1e3"` preserved | Manual parsing available | | ||
|
||
**Why truncate mode errors on scientific notation?** | ||
Properly evaluating `1e3` to `1000` requires floating-point arithmetic, which defeats the purpose of embedded no-float configurations. The truncate mode is designed for simple cases like `1.7` → `1` where no exponentiation is needed. | ||
|
||
## Benefits | ||
|
||
- **Zero runtime overhead**: Behavior configured at compile time | ||
- **Exact precision**: Raw strings always preserved | ||
- **Embedded friendly**: Avoid 64-bit math and float routines when not needed | ||
- **Flexible**: Choose the right tradeoffs for your use case | ||
- **no_std compatible**: No heap allocations | ||
- **Fail fast**: Error configurations catch incompatible data early |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
[workspace] | ||
resolver = "2" | ||
members = [ "stax","tokenizer", "demos"] |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.