Skip to content

V3EmitC: Use a reference instead of a pointer for VlSelf - #5254

Merged
wsnyder merged 1 commit into
verilator:masterfrom
cyyself:tmp_use_ref_for_vlself
Jul 15, 2024
Merged

wsnyder merged 1 commit into
verilator:masterfrom
cyyself:tmp_use_ref_for_vlself

Conversation

@cyyself

@cyyself cyyself commented Jul 12, 2024 •

Copy link
Copy Markdown
Contributor

Currently, we use pointers for VlSelf and dereference it every time we process the RTL. However, when the C++ compiler sees a pointer instead of a reference, it will lose a dereferenceable attribute, so the compiler can not speculatively load the value of the pointer and must generate more branches to prevent the unnecessary deference of the pointer. This will cause more branches to be generated in the final binary, which increases the burden on CPU branch prediction. Even worse, the branch footprint in Verilator generated code on large RTL designs such as XiangShan already suppresses the size of the branch target buffer of CPUs we use now, which will cause more branch misprediction and performance degradation.

Thus, we should use a reference instead of a pointer for VlSelf to tell the C++ compiler that the value of VlSelf is always dereferenceable, and the compiler can speculatively load the value of VlSelf without generating more branches.

A simple evaluation performed on XiangShan commit 136f64975e with LLVM16 packaged by Debian (version 16.0.6 (27+b1)) based on Verilator commit 6882f8c with 1 thread EMU running 2 iter coremark workload on OpenXiangShan/ready-to-run commit 24ab7d76e5 on Intel 13900K shows sampled branch instructions reduced from 132746731584 to 121130263138 (-8.75%), the running time reduced from 838.95s to 753.43s (-10.19%).

This increases l1-dcache-loads from 414383597179 to 426881071898 (+3.02%) and slightly increases l1-dcache-load-misses from 33706712758 to 33967571525 (+0.77%), but the LLC load-misses are reduced from 203290290 to 139019005 (-31.62%), maybe because the branch misprediction of wrong path is reduced. I also tested this on 8 threads EMU with Clang PGO as our routine to use Verilated XiangShan on 7950X3D + LLVM 14 and 13900K + LLVM 16. All show a slight improvement in performance even though we reduced the branches with PGO on and slightly increased the load instructions. And I haven't found any performance degradation on Verilated XiangShan RTL so far.

This change also allows more compiler optimization to be performed in verilator-generated code. I am currently working on it.

@cyyself
cyyself force-pushed the tmp_use_ref_for_vlself branch 2 times, most recently from 1797ae6 to c0c52ad Compare July 12, 2024 09:22
@cyyself
cyyself marked this pull request as draft July 12, 2024 09:59

@wsnyder wsnyder left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great idea, thanks for doing this! Using pointers dates back to when there was only C ;)

Please cleanup the tests etc and I'll review again when you mark non-draft.

Comment thread src/V3EmitCBase.h Resolved
@cyyself

cyyself commented Jul 14, 2024

Copy link
Copy Markdown
Contributor Author

I think I may need some help to create a dummy function that still preserves the original function using * vlSelf and wrapper function like void Vxxxx(xxxx *vlSelf) { return Vxxxxx(*vlSelf); } to fix some errors in the CI. I'm not familiar with the V3Emit and the AST code in the verilator.

@wsnyder

wsnyder commented Jul 14, 2024

Copy link
Copy Markdown
Member

I'm not sure your exact question, but picking one of the failures:

./Vt_timing_wait3___024root__DepSet_h12cc0b2a__0.cpp:25:75: error: no member named '__VnoInFunc_await' in 'VlClassRef<Vt_timing_wait3_std__03a__03aprocess>'; did you mean to use '->' instead of '.'?
    co_await VL_NULL_CHECK(vlSelf.t__DOT__p[1U], "t/t_timing_wait3.v", 15).__VnoInFunc_await(vlProcess, vlSymsp);

This is trying to reference a AstNodeClass, so it still needs -> not ., you need to tune where you changed to print ..

@cyyself

cyyself commented Jul 14, 2024 •

Copy link
Copy Markdown
Contributor Author

I'm not sure your exact question, but picking one of the failures:

./Vt_timing_wait3___024root__DepSet_h12cc0b2a__0.cpp:25:75: error: no member named '__VnoInFunc_await' in 'VlClassRef<Vt_timing_wait3_std__03a__03aprocess>'; did you mean to use '->' instead of '.'?
    co_await VL_NULL_CHECK(vlSelf.t__DOT__p[1U], "t/t_timing_wait3.v", 15).__VnoInFunc_await(vlProcess, vlSymsp);

This is trying to reference a AstNodeClass, so it still needs -> not ., you need to tune where you changed to print ..

Yes. For this, I have an idea to check if the string will be vlSelf and then use -> else use .. But for the wrapper, they are independent issues.

@cyyself
cyyself force-pushed the tmp_use_ref_for_vlself branch 3 times, most recently from 5a478ec to 775d168 Compare July 14, 2024 18:26
@cyyself
cyyself marked this pull request as ready for review July 14, 2024 18:26
@cyyself

cyyself commented Jul 14, 2024

Copy link
Copy Markdown
Contributor Author

Now, I found a much easier way to do so. Use std::ref(__).get() to prevent direct use of the reference being optimized out as a pointer by the compiler.

@wsnyder

wsnyder commented Jul 14, 2024

Copy link
Copy Markdown
Member

Can you provide a web link to that ref concept?

@cyyself

cyyself commented Jul 14, 2024

Copy link
Copy Markdown
Contributor Author

Can you provide a web link to that ref concept?

If you want the std::ref, we can found here: https://en.cppreference.com/w/cpp/utility/functional/ref

Using std::ref(___).get() is likely to be a hack, but it does work for the compilers we use now. At least LLVM 14 and LLVM main branch have been tested.

@cyyself
cyyself force-pushed the tmp_use_ref_for_vlself branch 2 times, most recently from b1dda25 to 92db227 Compare July 15, 2024 10:17
@cyyself
cyyself requested a review from wsnyder July 15, 2024 11:06
Comment thread src/V3EmitCFunc.h Resolved
Currently, we use pointers for VlSelf and dereference it every time we
process the RTL. However, when the C++ compiler sees a pointer instead
of a reference, it will lose a dereferenceable attribute, so the
compiler can not speculatively load the value of the pointer and must
generate more branches to prevent the unnecessary deference of the
pointer. This will cause more branches to be generated in the final
binary, which increases the burden on CPU branch prediction. Even worse,
the branch footprint in Verilator generated code on large RTL designs
such as XiangShan already suppresses the size of the branch target
buffer of CPUs we use now, which will cause more branch misprediction
and performance degradation.

Thus, we should use a reference instead of a pointer for VlSelf to tell
the C++ compiler that the value of VlSelf is always dereferenceable, and
the compiler can speculatively load the value of VlSelf without
generating more branches.

A simple evaluation performed on XiangShan commit 136f64975e with LLVM16
packaged by Debian (version 16.0.6 (27+b1)) based on Verilator commit
6882f8c with 1 thread EMU running 2 iter coremark workload on
OpenXiangShan/ready-to-run commit 24ab7d76e5 on Intel 13900K shows
sampled branch instructions reduced from 132746731584 to 121130263138
(-8.75%), the running time reduced from 838.95s to 753.43s (-10.19%).

This increases l1-dcache-loads from 414383597179 to 426881071898
(+3.02%) and slightly increases l1-dcache-load-misses from 33706712758
to 33967571525 (+0.77%), but the LLC load-misses are reduced from
203290290 to 139019005 (-31.62%), maybe because the branch misprediction
of wrong path is reduced. I also tested this on 8 threads EMU with Clang
PGO as our routine to use Verilated XiangShan on 7950X3D + LLVM 14 and
13900K + LLVM 16. All show a slight improvement in performance even
though we reduced the branches with PGO on and slightly increased the
load instructions. And I haven't found any performance degradation on
Verilated XiangShan RTL so far.

This change also allows more compiler optimization to be performed in
verilator-generated code. I am currently working on it.
@cyyself
cyyself force-pushed the tmp_use_ref_for_vlself branch from 92db227 to 35d7d24 Compare July 15, 2024 12:26
@cyyself

cyyself commented Jul 15, 2024

Copy link
Copy Markdown
Contributor Author

@wsnyder
wsnyder merged commit 164a7a7 into verilator:master Jul 15, 2024
RRozak pushed a commit to antmicro/verilator that referenced this pull request Aug 9, 2024
cyyself added a commit to cyyself/verilator that referenced this pull request Sep 13, 2026
Since verilator#5254 generated functions read model state through vlSelfRef, obtained
with `auto& vlSelfRef = std::ref(*vlSelf).get();`. That only works because
reference_wrapper::get() returns T&, so its call result carries LLVM's
`dereferenceable` attribute, and only the SimplifyCFG run before the inliner
can use it. LLVM 23.1 (llvm/llvm-project#204795) trusts `dereferenceable`
only at the point where it is stated unless the object provably cannot be
freed before the use; for a call result that proof needs an uninterrupted
walk back to the call, which the long generated functions cannot provide,
so with LLVM 23.1 and later the branches removed by verilator#5254 come back.

When the model object is the function argument instead, LLVM only needs the
function to be `nofree`, which it infers for the generated functions, and the
fact then holds for the whole function. Loose functions therefore now take
`Model& vlSelfRef` instead of `Model* vlSelf`, with a `vlSelf` pointer
derived from it for code that needs one.

Parts of this change were prepared with the help of a generative AI tool
(Claude Fable 5.1).

Signed-off-by: Yangyu Chen <cyy@cyyself.name>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants