Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

lexer: TOML: Support more integer and floating formats #1832

Merged
merged 12 commits into from Jul 2, 2022
Merged

lexer: TOML: Support more integer and floating formats #1832

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
7340209
Fix floating point format in TOML lexer
ToruNiina Jun 14, 2022
f1eec39
Support sign of number in TOML lexer
ToruNiina Jun 14, 2022
bb0ebc7
Support special float (nan/inf) in TOML lexer
ToruNiina Jun 14, 2022
284acde
Support hex, oct, bin integers in TOML lexer
ToruNiina Jun 14, 2022
e36833f
Support underscores between digits in TOML lexer
ToruNiina Jun 14, 2022
47bfed6
Add more int/float examples to TOML visual sample
ToruNiina Jun 14, 2022
97eab34
Use Num::Oct and non-captured group for oct int
ToruNiina Jul 1, 2022
b1b7f8d
Use Num::Bin and non-captured group for bin int
ToruNiina Jul 1, 2022
75d11fc
Use Num::Hex and non-captured group for hex int
ToruNiina Jul 1, 2022
34fbc10
Use non-captured group if capture is not needed
ToruNiina Jul 1, 2022
1cff1bf
Use non-captured group if capture is not needed
ToruNiina Jul 1, 2022
8a9c645
Use non-captured group if capture is not needed
ToruNiina Jul 1, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
11 changes: 8 additions & 3 deletions lib/rouge/lexers/toml.rb
View file
Open in desktop
Expand Up @@ -28,9 +28,14 @@ class TOML < RegexLexer

rule %r/\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z/, Literal::Date

rule %r/(\d+\.\d*|\d*\.\d+)([eE][+-]?[0-9]+)?j?/, Num::Float
rule %r/\d+[eE][+-]?[0-9]+j?/, Num::Float
rule %r/\-?\d+/, Num::Integer
rule %r/[+-]?\d+(?:_\d+)*\.\d+(?:_\d+)*(?:[eE][+-]?\d+(?:_\d+)*)?/, Num::Float
rule %r/[+-]?\d+(?:_\d+)*[eE][+-]?\d+(?:_\d+)*/, Num::Float
rule %r/[+-]?(?:nan|inf)/, Num::Float

rule %r/0x\h+(?:_\h+)*/, Num::Hex
rule %r/0o[0-7]+(?:_[0-7]+)*/, Num::Oct
rule %r/0b[01]+(?:_[01]+)*/, Num::Bin
rule %r/[+-]?\d+(?:_\d+)*/, Num::Integer
end

state :root do
Expand Down
20 changes: 20 additions & 0 deletions spec/visual/samples/toml
View file
Open in desktop
Expand Up @@ -118,3 +118,23 @@ point = { x = 1, y = 2 }
physical.color = "orange"
physical.shape = "round"
site."google.com" = true

# floating point numbers
float_space1 = 3.141_592_653
float_space2 = 123_456.789_012e+1_000
just_inf = inf
pos_inf = +inf
neg_inf = -inf
just_nan = nan
pos_nan = +nan
neg_nan = -nan

# integer formats
dec = 123456
dec_spacing = 123_456
hex = 0xdeadBEEF
hex_spacing = 0xdead_BEEF
oct = 0o755
oct_spacing = 0o755_777
bin = 0b1100
bin_spacing = 0b0101_1100