feat(semantic): Add index mapping from span to reference id

[YangchenYe323]

As a temporary workaround, save mappings from Span to ResolvedReferenceId to enable searching for semantic information given an IdentifierReference

[YangchenYe323]

Benchmark is taking forever 😓 Maybe we'll fall back to a linear search instead

[Boshen]

@YangchenYe323 I found where your algorithm complexity blew up:

https://github.com/Boshen/oxc/blob/024f1a1552516243ab97e60e5a404e14874ca91e/crates/oxc_semantic/src/scope/builder.rs#L69-L85

You don't need to walk up the scope tree here. Pushing the unresolved references to the parent scope is enough. because the parent scope exit will pick up this routine again and carry the unresolved references upwards. Any IdentifierReference is an unresolved reference when the Top scope's unresolved_references contains this IdentifierReference.

I think we should move scope.unresolved_references to ScopeBuilder, and then move it to ScopeTree?

Note: For you need to actively check for Top scope in order to find the correct scope to add the unresolved_references.

[YangchenYe323]

You don't need to walk up the scope tree here. Pushing the unresolved references to the parent scope is enough. because the parent scope exit will pick up this routine again and carry the unresolved references upwards. Any IdentifierReference is an unresolved reference when the Top scope's unresolved_references contains this IdentifierReference.

I think we should move scope.unresolved_references to ScopeBuilder, and then move it to ScopeTree?

Note: For you need to actively check for Top scope in order to find the correct scope to add the unresolved_references.

I've made a new commit. Is it what you're suggesting? I experimented with it and found that it didn't give much boost in performace:

The overall complexity would remain the same either way because the total number of references we're resolving is the same. The depth of the scope tree is not a dominating factor.

[YangchenYe323]

The overall complexity would remain the same either way because the total number of references we're resolving is the same. The depth of the scope tree is not a dominating factor.

For babylon.max.js, the index's capacity grows to as many as 114688 entries, if each entry is a (Span, ResolvedReferenceId) pair, the index is 2MB.

[Boshen]

I wonder if it's due to hashing two u32 resulting too many hash collisions, FxHash is horrible in this regard ...

Maybe we'll fall back to a linear search instead

Shall we try with a Vec, sort it in the final build phase, and do binary search for lookup instead?

[YangchenYe323]

I wonder if it's due to hashing two u32 resulting too many hash collisions, FxHash is horrible in this regard ...

Maybe we'll fall back to a linear search instead

Shall we try with a Vec, sort it in the final build phase, and do binary search for lookup instead?

Unfortunately we can't. Currently ResolvedReferenceId is the index into the resolved_reference vector. If we sort it the ids all point to wrong index, and the references in the Symbol are invalidated as well.

But I found a workaround: use BTreeMap. The overhead seems negligible.

[YangchenYe323]

I wonder if it's due to hashing two u32 resulting too many hash collisions, FxHash is horrible in this regard ...

I couldn't reproduce this issue in isolated micro benchmarks (FxHashMap is a lot faster than BTreeMaps). But the memory footprint of FxHashMap is a lot larger than BtreeMap in our case:

[Boshen]

The benchmark is finally back to normal. We can finally focus our energy on the HIR.