Return values up to 128 bits in registers

[jonas-schievink]

This fixes #26494 (comment) by making Rust's default ABI pass return values up to 128 bits in size in registers, just like the System V ABI.

The result is that these methods from the comment linked above now generate the same code, making the Rust ABI as efficient as the "C" ABI:

pub struct Stats { x: u32, y: u32, z: u32, }

pub extern "C" fn sum_c(a: &Stats, b: &Stats) -> Stats {
    return Stats {x: a.x + b.x, y: a.y + b.y, z: a.z + b.z };
}

pub fn sum_rust(a: &Stats, b: &Stats) -> Stats {
    return Stats {x: a.x + b.x, y: a.y + b.y, z: a.z + b.z };
}

sum_rust:
	movl	(%rsi), %eax
	addl	(%rdi), %eax
	movl	4(%rsi), %ecx
	addl	4(%rdi), %ecx
	movl	8(%rsi), %edx
	addl	8(%rdi), %edx
	shlq	$32, %rcx
	orq	%rcx, %rax
	retq

[rust-highfive]

r? @estebank

(rust_highfive has picked a reviewer for you, use r? to override)

[jonas-schievink]

r? @nagisa perhaps? cc @eddyb

[Mark-Simulacrum]

Is there a reason not to do this on all x86_64 architectures?

@bors try @rust-timer queue

[rust-timer]

Awaiting bors try build completion

[bors]

⌛ Trying commit 1be289d9c76441135b72186b133e949411657024 with merge 90ed0d04e2f640ecd65d3b39ffd37f7fd9b3cd25...

[bors]

☀️ Try build successful - checks-actions, checks-azure
Build commit: 90ed0d04e2f640ecd65d3b39ffd37f7fd9b3cd25 (90ed0d04e2f640ecd65d3b39ffd37f7fd9b3cd25)

[rust-timer]

Queued 90ed0d04e2f640ecd65d3b39ffd37f7fd9b3cd25 with parent 1fd5b9d, future comparison URL.

[rust-timer]

Finished benchmarking try commit (90ed0d04e2f640ecd65d3b39ffd37f7fd9b3cd25): comparison url.

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. Please note that if the perf results are neutral, you should likely undo the rollup=never given below by specifying rollup- to bors.

Importantly, though, if the results of this run are non-neutral do not roll this PR up -- it will mask other regressions or improvements in the roll up.

@bors rollup=never

[jonas-schievink]

Hey that's neat, the improvements are in the compiler itself, so this seems to have a practical impact! It looks like most regressions are in LLVM, which is somewhat expected, since it might deal differently with the code now.

[calebsander]

pub struct Stats { x: u32, y: u32, z: u32, }

pub extern "C" fn sum_c(a: &Stats, b: &Stats) -> Stats {
    return Stats {x: a.x + b.x, y: a.y + b.y, z: a.z + b.z };
}

example::sum_c:
        mov     rax, qword ptr [rsi]
        add     rax, qword ptr [rdi]
        mov     edx, dword ptr [rsi + 8]
        add     edx, dword ptr [rdi + 8]
        mov     cl, byte ptr [rsi + 12]
        add     cl, byte ptr [rdi + 12]
        movzx   ecx, cl
        shl     rcx, 32
        or      rdx, rcx
        ret

The generated assembly here doesn't seem to match the code. It looks like you've used struct Stats { x: u64, y: u32, z: u8 } instead? The assembly I get (using the posted code) is:

example::sum_c:
        mov     eax, dword ptr [rsi]
        add     eax, dword ptr [rdi]
        mov     ecx, dword ptr [rsi + 4]
        add     ecx, dword ptr [rdi + 4]
        mov     edx, dword ptr [rsi + 8]
        add     edx, dword ptr [rdi + 8]
        shl     rcx, 32
        or      rax, rcx
        ret

[jonas-schievink]

@calebsander ah, sorry, my mistake. Updated the snippet.

[pftbest]

Why not make this change for all 64bit platforms? All major ones do this in C: https://godbolt.org/z/ds1ezh

[jonas-schievink]

Because I only know x86_64

[nagisa]

Can you add a codegen test for this that verifies we don't materialize the return value into stack/memory?

r=me after that.

[jonas-schievink]

@bors r=nagisa

[bors]

📌 Commit cc2ba3b has been approved by nagisa

[bors]

⌛ Testing commit cc2ba3b with merge 62fe055...

[bors]

☀️ Test successful - checks-actions, checks-azure
Approved by: nagisa
Pushing 62fe055 to master...

[ecstatic-morse]

The final perf results for this PR are in. Instruction counts have increased on most benchmarks, and task-clock shows no improvement.

This is a bit disappointing, as I expected this PR to be a pretty clear win. The try run also showed small losses across the board (never trust the emoji), although stress tests of the trait resolution code (keccak and inflate) seemed to fair better than they did in the final run. Possibly the change in #77041 caused the discrepancy? @jonas-schievink Am I missing anything? If not, I think we should consider reverting this unless there are benefits outside of perf.

[jonas-schievink]

Oh, that's disappointing. But this PR was mostly aimed at improving the generated code, not speeding up rustc.

Seems likely that #77041 has resulted in the same improvements I saw here.

[pftbest]

It looks to me like the only heavy operation here is a string comparison for spec.target_spec().arch.as_str()

Can it be removed somehow? For example replace it with a check for pointer size == 64

[jonas-schievink]

The regressions are in LLVM from what I can tell, not the code I added

[pftbest]

Oh I see, that's too bad.

[ecstatic-morse]

Ah that's true. It seems like rustc just doesn't benefit from this very much at runtime. Here's the task-clock graph starting from the parent of this commit. Obviously there's a lot of noise here, but check builds remained about the same while codegen-ed builds regressed slightly. That said, I would guess there's code in the ecosystem that would benefit from this, and the compile-time regressions are pretty small.

[eddyb]

Is there anything wrong with 2 * pointer_size? IIRC we already return pairs in two registers.

[eddyb]

That is, we already do 2 * pointer_size on all architectures, just not for arbitrary data, only pairs of scalars.

[jonas-schievink]

Yeah, that makes sense. Opened #77434.