Parent: #22
Goal
Implement the CSS tokenizer per CSS Syntax Module Level 3. Iterator-based, yields CssToken values.
File Changes
- crates/ie-css/src/tokenizer.rs — new file
- crates/ie-css/src/lib.rs — update modules
- crates/ie-css/Cargo.toml — if needed
Implementation
- CssToken enum: Ident, Function, AtKeyword, Hash, String, Url, Delim, Number, Percentage, Dimension, Whitespace, Colon, Semicolon, Comma, brackets, Cdo, Cdc, Eof
- Handle escape sequences, string tokens, URL tokens, comments
- Numbers: integer vs number distinction
- Iterator-based
Tests
- Simple rule tokens, strings with escapes, URL tokens, dimensions, percentages, functions
- ~10 unit tests
Acceptance Criteria
- cargo test -p ie-css passes
- Clippy clean
Parent: #22
Goal
Implement the CSS tokenizer per CSS Syntax Module Level 3. Iterator-based, yields CssToken values.
File Changes
Implementation
Tests
Acceptance Criteria