fix(parser): error on source larger than 4 GiB

[overlookmotel]

Token and Span both represent start and end as u32.

This limits size of source which can be parsed to u32::MAX.

oxc/crates/oxc_span/src/span.rs

Lines 14 to 20 in 1957770

	/// NOTE: `u32` is sufficient for "all" reasonable programs. Larger than u32 is a 4GB JS file.
	#[derive(Debug, Default, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]
	#[cfg_attr(feature = "serde", derive(Serialize))]
	pub struct Span {
	pub start: u32,
	pub end: u32,
	}

However, this constraint is currently not enforced.

In a release build, code will not panic on arithmetic overflow, so start/end could wrap around back to zero if source is 4 GiB or more.

That'd produce nonsense spans. But worse, the lexer relies in some places on self.current.token.start being correct, so if the value wrapped around, possibly it'd keep rewinding to the start of the source and lexing it again, causing an infinite loop.

In worst case, if for some reason an application's public API used OXC's parser with user-supplied source code (parser-as-a-service!), this could be exploited for denial of service.

This PR adds an assertion to catch this at the start of parsing instead.

This does add an extra instruction, but I imagine the effect will be negligible compared to the work required to parse the code.

[overlookmotel]

Actually maybe panicking isn't ideal. Should parser instead return an empty AST and an error (i.e. treat too-long code as a syntax error)?

[overlookmotel]

Last commit changes behavior to return an error instead of panic if source size exceeds limit. I assume that's better.

2 problems remain with the tests:

1. I think it's ideal to have a debug_assert!() for source length in Lexer::new so that the size constraint is encapsulated in the lexer. But that makes the overlong_source test fail in a debug build (passes in release mode). That's the reason CI is failing. Any idea how to tackle that?
2. legal_length_source test takes about 5 secs to run in release mode, and 1 minute in debug mode. Is 5s too long for a test?

Could those tests be set to only run in release mode? Does CI runs tests against a release build? Or can we just skip these tests?

[Boshen]

I would just let it crash and change the public API

oxc/crates/oxc_parser/src/lib.rs

Lines 136 to 137 in 0feeac5

	/// Create a new parser
	pub fn new(allocator: &'a Allocator, source_text: &'a str, source_type: SourceType) -> Self {

to something like

/// # Panics
/// * When source length is 4GB (u32 max)
pub fn new(...) {
  assert!(source_text.len() < 4GB, "source text is too long ...");
}

Or play nice and return an empty AST and a nice error.

As for the tests, I'll just omit them because it's not that interesting to unit test a 4GB program.

[codspeed-hq]

CodSpeed Performance Report

Merging #1860 will not alter performance

_{Comparing overlookmotel:parser-max-len (f4c4839) with main (2ac2630)}

Summary

✅ 14 untouched benchmarks

[overlookmotel]

I found solution to the above problems. Now squashed to a single commit and tests pass.

I went with returning empty AST and error approach. Currently I don't think parser has any code paths which lead to panic, and I worried introducing one might hurt performance.

Feel free to remove the 2nd test. That's the one that takes 5 secs to run, but it won't run on CI anyway.

However, I think it'd be ideal to keep the 1st test. Atom is going to rely on the invariant that a string which references source text cannot exceed 4 GiB, and if that invariant was violated, it could produce UB.

Note: To allow the debug_assert! for size in Lexer::new, there are 2 places in parser where the length is checked. However, the 2nd check only occurs on code path where parsing has failed, so won't affect performance of successful parsing, which I imagine is what matters.

Sorry for all the noise on this PR!

[overlookmotel]

Shall I squash commits and rebase on main? Or will that happen automatically when merging?

[Boshen]

Thank you for making Oxc safe!