use raw references instead of addr_of! / addr_of_mut!

[Icxolu]

Closes #5846

[codspeed-hq]

Merging this PR will improve performance by 28.2%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 2 improved benchmarks
✅ 103 untouched benchmarks
⏩ 1 skipped benchmark¹

Performance Changes

	Benchmark	BASE	HEAD	Efficiency
⚡	format_simple	397.8 ns	310.3 ns	+28.2%
⚡	critical_section_creation	703.1 ns	615.6 ns	+14.21%

---

_{Comparing Icxolu:raw-references (571ec6a) with main (7aa6163)}

Footnotes

1. 1 benchmark was skipped, so the baseline result was used instead. If it was deleted from the codebase, click here and archive it to remove it from the performance reports. ↩