Skip to content

Conversation

@byroot
Copy link
Member

@byroot byroot commented Nov 1, 2025

We can use ctz builtin to get the number of consecutive digits.

== Parsing float parsing (2251051 bytes)
ruby 3.4.6 (2025-09-16 revision dbd83256b1) +YJIT +PRISM [arm64-darwin24]
Warming up --------------------------------------
               after    25.000 i/100ms
Calculating -------------------------------------
               after    231.963 (± 0.0%) i/s    (4.31 ms/i) -      1.175k in   5.065467s

Comparison:
              before:      215.3 i/s
               after:      232.0 i/s - 1.08x  faster

@byroot byroot force-pushed the improve-consecutive-digits branch from aeb0e0e to 53991f0 Compare November 1, 2025 12:05
@byroot
Copy link
Member Author

byroot commented Nov 1, 2025

Interestingly, this seem to fail on ubuntu/GCC, but it passes on clang. I'm not yet clear on why.

// From: https://github.com/simdjson/simdjson/blob/32b301893c13d058095a07d9868edaaa42ee07aa/include/simdjson/generic/numberparsing.h#L333
// Branchless version of: http://0x80.pl/articles/swar-digits-validate.html
uint64_t match = (next_8bytes & 0xF0F0F0F0F0F0F0F0) | (((next_8bytes + 0x0606060606060606) & 0xF0F0F0F0F0F0F0F0) >> 4);
size_t consecutive_digits = trailing_zeros64(match ^ 0x3333333333333333) / CHAR_BIT;
Copy link
Member

@tompng tompng Nov 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From a google search result, __builtin_ctzll(0) seems to be an undefined behavior. I'm not sure it's really undefined though.
The trailing_zeros64 fallbacks code when __builtin_ctzll is not defined, actually returns 0 instead of 64 in this case. So at least trailing_zeros64 fallback code should be fixed.

# When this part is commented out:
# #if HAVE_BUILTIN_CTZLL
#     return __builtin_ctzll(input);
# #else

JSON.parse "[12341234]" #=> JSON::ParserError

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes thank you I just figured this out:

int __builtin_clz (unsigned int x)

Returns the number of leading 0-bits in x, starting at the most significant bit position. If x is 0, the result is undefined.

https://gcc.gnu.org/onlinedocs/gcc/Bit-Operation-Builtins.html

I think GCC assume this case never happens so it removes that branch.

@byroot byroot force-pushed the improve-consecutive-digits branch 2 times, most recently from 947902e to 7b1a77d Compare November 1, 2025 13:26
@byroot byroot force-pushed the improve-consecutive-digits branch 3 times, most recently from 563a55e to 5bcc397 Compare November 1, 2025 13:54
@byroot
Copy link
Member Author

byroot commented Nov 1, 2025

Now I need to review usage of trailing_zeros and trailing_zeros64...

@byroot byroot force-pushed the improve-consecutive-digits branch from 5bcc397 to 82b030f Compare November 1, 2025 14:18
static inline uint32_t trailing_zeros64(uint64_t input)
{
#ifdef JSON_DEBUG
assert(input > 0); // __builtin_ctz(0) is undefined behavior
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm doubting these asserts can actually fail. Not too sure what the compilation flags are on CI, but seems like they are optimized out even with JSON_DEBUG.

I'm unsure what's the best way to add assertions in ruby extensions. Perhaps I should just use RUBY_ASSERT and add a debug ruby in the CI matrix?

@byroot byroot merged commit f7cd41c into ruby:master Nov 1, 2025
37 checks passed
@byroot byroot deleted the improve-consecutive-digits branch November 1, 2025 14:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants