i128 and u128 support #37900

est31 · Nov 20, 2016

Adds support for 128-bit integers. Rebased version of #35954 , with additional improvements.

Thanks @nagisa for mentoring this PR!

Some of the problems that were resolved:

Confirm that intrinsics work on 32 bit platforms and fix them if needed
Wait for #37906 to get fixed, so that work on intrinsics can be completed (worked around by merging the PR on my local setup)
Investigate and fix linkcheck failure

[plugin-breaking-change]

cc #35118

r? @aturon

(rust_highfive has picked a reviewer for you, use r? to override)

rust-highfive · Nov 20, 2016

r? @aturon

(rust_highfive has picked a reviewer for you, use r? to override)

frewsxcv · Nov 20, 2016

frewsxcv reviewed Nov 20, 2016

View changes

frewsxcv · Nov 20, 2016

src/libcore/num/i128.rs

+
+//! The 128-bit signed integer type.
+//!
+//! *[See also the `i128` primitive type](../../std/primitive.i128.html).*


This should maybe be:

[See also the `i128` primitive type](../primitive.i128.html)

@frewsxcv I think this wouldn't work. The other integer primitive types link via ../../std/primitive.$name.html as well, and I can't see generated doc files for them in libcore.

I think the problem is that the primitive.i128.html isn't being generated at all so changing this won't help.

@ollie27 yes, that is consistent with my observations, I can find primitive.i64.html at the linked path, but none for i128.

Investigate and fix linkcheck failure:

By the looks of it the files won't be generated. You'll need to add entries to src/libstd/primitive_docs.rs and update rustdoc a bit more. At least here and here will need to be updated.

ollie27 · Nov 20, 2016

Investigate and fix linkcheck failure:

By the looks of it the files won't be generated. You'll need to add entries to src/libstd/primitive_docs.rs and update rustdoc a bit more. At least here and here will need to be updated.

☔️ The latest upstream changes (presumably #37824) made this pull request unmergeable. Please resolve the merge conflicts.

bors · Nov 21, 2016

☔️ The latest upstream changes (presumably #37824) made this pull request unmergeable. Please resolve the merge conflicts.

cc @rust-lang/compiler

aturon · Nov 22, 2016

cc @rust-lang/compiler

Okay, running on 32 bit into a linker error of the form:

rust/src/librustc_const_math/int.rs:545: Nicht definierter Verweis auf `core::panicking::panic::hdb3be1030a0e450d'
rust/build/i686-unknown-linux-gnu/stage1-rustc/i686-unknown-linux-gnu/release/deps/rustc_const_math-fb45b0b7ff405aab.0.o: In Funktion `rustc_const_math::int::{{impl}}::rem':
rust/src/librustc_const_math/int.rs:577: Nicht definierter Verweis auf `core::panicking::panic::hdb3be1030a0e450d'
/tmp/rustc.8pXinEKP9SgF/libcompiler_builtins-83f89429e0eb597e.rlib(compiler_builtins-83f89429e0eb597e.0.o): In Funktion `compiler_builtins::reimpls::u128_div_mod':
rust/src/libcompiler_builtins/lib.rs:132: Nicht definierter Verweis auf `core::panicking::panic::hdb3be1030a0e450d'
collect2: error: ld returned 1 exit status

Apparently the __udivmodti4 intrinsic may panic. According to readelf, its the only one in the libcompiler_builtin library that calls panic. I've localized the error to the ashl and ashr macros inside the src/libcompiler_builtins/lib.rs file, and the error messages of the panics are "attempt to calculate the remainder with a divisor of zero" and "attempt to divide by zero" (found out with a very reliable and robust method involving a strings invocation and grepping for "rustc version" xD)...

I'm not sure whether or not I should panic here, and how to fix the linker issue if I should indeed panic.

Setting panic=abort for the libcompiler_builtin crate does not remove the panic invocations for some reason, nor the linker failure...

est31 · Nov 22, 2016

Okay, running on 32 bit into a linker error of the form:

rust/src/librustc_const_math/int.rs:545: Nicht definierter Verweis auf `core::panicking::panic::hdb3be1030a0e450d'
rust/build/i686-unknown-linux-gnu/stage1-rustc/i686-unknown-linux-gnu/release/deps/rustc_const_math-fb45b0b7ff405aab.0.o: In Funktion `rustc_const_math::int::{{impl}}::rem':
rust/src/librustc_const_math/int.rs:577: Nicht definierter Verweis auf `core::panicking::panic::hdb3be1030a0e450d'
/tmp/rustc.8pXinEKP9SgF/libcompiler_builtins-83f89429e0eb597e.rlib(compiler_builtins-83f89429e0eb597e.0.o): In Funktion `compiler_builtins::reimpls::u128_div_mod':
rust/src/libcompiler_builtins/lib.rs:132: Nicht definierter Verweis auf `core::panicking::panic::hdb3be1030a0e450d'
collect2: error: ld returned 1 exit status

Apparently the __udivmodti4 intrinsic may panic. According to readelf, its the only one in the libcompiler_builtin library that calls panic. I've localized the error to the ashl and ashr macros inside the src/libcompiler_builtins/lib.rs file, and the error messages of the panics are "attempt to calculate the remainder with a divisor of zero" and "attempt to divide by zero" (found out with a very reliable and robust method involving a strings invocation and grepping for "rustc version" xD)...

I'm not sure whether or not I should panic here, and how to fix the linker issue if I should indeed panic.

Setting panic=abort for the libcompiler_builtin crate does not remove the panic invocations for some reason, nor the linker failure...

@est31 -C panic=abort is a red herring, forget about it, it changes the runtime, not code generation. Sounds like all we have to do is either always compile libcompiler_builtins with -Z force-overflow-checks=off or explicitly use Wrapping and/or wrapping_add etc.

eddyb · Nov 22, 2016

@est31 -C panic=abort is a red herring, forget about it, it changes the runtime, not code generation. Sounds like all we have to do is either always compile libcompiler_builtins with -Z force-overflow-checks=off or explicitly use Wrapping and/or wrapping_add etc.

☔️ The latest upstream changes (presumably #37487) made this pull request unmergeable. Please resolve the merge conflicts.

bors · Nov 23, 2016

☔️ The latest upstream changes (presumably #37487) made this pull request unmergeable. Please resolve the merge conflicts.

The approach we've used so far in compiler-builtins is to use explicit wrapping operations everywhere. Given that we're mostly translating C code it may make sense to just use -Z force-overflow-checks=off.

Using the Wrapping type is annoying due to #23545 (see https://github.com/rust-lang/rust/blob/master/src/libcore/num/wrapping.rs#L103-L118), so if you want to use Wrapping everywhere you have to do things like x << y.0 as usize where x and y are both Wrapping(_).

cc @japaric

mattico · Nov 23, 2016

The approach we've used so far in compiler-builtins is to use explicit wrapping operations everywhere. Given that we're mostly translating C code it may make sense to just use -Z force-overflow-checks=off.

Using the Wrapping type is annoying due to #23545 (see https://github.com/rust-lang/rust/blob/master/src/libcore/num/wrapping.rs#L103-L118), so if you want to use Wrapping everywhere you have to do things like x << y.0 as usize where x and y are both Wrapping(_).

cc @japaric

Progress update: I've been able to eliminate the linker errors by substituting the operations that could fail with unchecked_ versions of them. It makes sense as the C intrinsics do the same and we do the checking afterwards.

I've right now arrived at debugging the functionality of the actual intrinsics themselves... Right now the failure is with the i128 unit tests, when parsing the literals for very large integers, apparently it overflows which it shouldn't.

est31 · Nov 23, 2016

Progress update: I've been able to eliminate the linker errors by substituting the operations that could fail with unchecked_ versions of them. It makes sense as the C intrinsics do the same and we do the checking afterwards.

I've right now arrived at debugging the functionality of the actual intrinsics themselves... Right now the failure is with the i128 unit tests, when parsing the literals for very large integers, apparently it overflows which it shouldn't.

☔️ The latest upstream changes (presumably #37890) made this pull request unmergeable. Please resolve the merge conflicts.

bors · Nov 24, 2016

☔️ The latest upstream changes (presumably #37890) made this pull request unmergeable. Please resolve the merge conflicts.

nagisa · Nov 25, 2016

nagisa reviewed Nov 25, 2016

View changes

src/libcompiler_builtins/lib.rs

Okay, the PR is ready for review. I've finished the work on intrinsics and wouldn't know of anything broken or which needs to be done. The tests succeed for both 64 bit and 32 bit platforms.

est31 · Nov 25, 2016

Okay, the PR is ready for review. I've finished the work on intrinsics and wouldn't know of anything broken or which needs to be done. The tests succeed for both 64 bit and 32 bit platforms.

I am quite happy for the future availability of 128 bit numbers in Rust. The "mul_mod" operation is used by "pow_mod" (modular pow), that is sufficiently common:

fn mul_mod(a: u64, b: u64, m: u64) -> (u64, u64);
fn mul_mod(a: u128, b: u128, m: u128) -> (u128, u128);

If you have 128 bits you can perform the 64 bit mul_mod using 128 bit values, but it's more efficient to use the "divq" instruction and work on 64 bit values. And you can probably perform the 128 bit mul_mod more efficiently without 256 bit numbers.

Are you going to support those mul_mod (or something similar), or should this enhancement request be moved elsewhere?

leonardo-m · Nov 25, 2016

I am quite happy for the future availability of 128 bit numbers in Rust. The "mul_mod" operation is used by "pow_mod" (modular pow), that is sufficiently common:

fn mul_mod(a: u64, b: u64, m: u64) -> (u64, u64);
fn mul_mod(a: u128, b: u128, m: u128) -> (u128, u128);

If you have 128 bits you can perform the 64 bit mul_mod using 128 bit values, but it's more efficient to use the "divq" instruction and work on 64 bit values. And you can probably perform the 128 bit mul_mod more efficiently without 256 bit numbers.

Are you going to support those mul_mod (or something similar), or should this enhancement request be moved elsewhere?

@leonardo-m I think providing modular pow is outside of the scope of this PR. Feel free to open an issue or RFC at https://github.com/rust-lang/rfcs . Also, you might be interested in https://github.com/rust-num/num

est31 · Nov 25, 2016

@leonardo-m I think providing modular pow is outside of the scope of this PR. Feel free to open an issue or RFC at https://github.com/rust-lang/rfcs . Also, you might be interested in https://github.com/rust-num/num

My suggestion was about mul_mod (not about pow_mod), and I think it needs a low-level implementation with intrinsics or asm to be efficient. I'll probably open an issue here.

leonardo-m · Nov 25, 2016

My suggestion was about mul_mod (not about pow_mod), and I think it needs a low-level implementation with intrinsics or asm to be efficient. I'll probably open an issue here.

r? @eddyb (feel free to transfer)

alexcrichton · Nov 25, 2016

r? @eddyb (feel free to transfer)

Started crater run.

eddyb · Nov 27, 2016

Started crater run.

eddyb · Nov 28, 2016

eddyb reviewed Nov 28, 2016

View changes

LGTM, only 2 minor nits. I assume FIXMEs are left for later.

src/libcore/fmt/num.rs

eddyb · Nov 28, 2016

src/librustc_trans/mir/block.rs

-                        });
+                        let const_err = common::const_to_opt_u128(len, false)
+                            .and_then(|len| common::const_to_opt_u128(index, false)
+                                .map(|index| ErrKind::IndexOutOfBounds {


IndexOutOfBounds should've been updated to u128.

Currently we do not have any platform that supports isize larger than i64 so its not really important. Would be good to add a FIXME, though.

The index is assured by typecheck to be usize and therefore can't exceed the range of 64 bits, can it? Would it be an idea to re-add const_to_opt_uint for cases like this and the one you mentioned above?

Hmm, the function was removed because it would cause bad behaviour in cross compilation situations...

I guess I'll add a const_to_opt_u64 function instead, as there are no 128 bit targets nor hosts neither in existence nor on the horizon.

Oh, the assumption that pointers are always 64-bit or less is interesting (but not explicit?). The code is fine as-is in that case.

eddyb · Nov 28, 2016

src/librustc_trans/mir/block.rs

@@ -281,7 +281,7 @@ impl<'bcx, 'tcx> MirContext<'bcx, 'tcx> {

            mir::TerminatorKind::Assert { ref cond, expected, ref msg, target, cleanup } => {
                let cond = self.trans_operand(&bcx, cond).immediate();
-                let mut const_cond = common::const_to_opt_uint(cond).map(|c| c == 1);
+                let mut const_cond = common::const_to_opt_u128(cond, false).map(|c| c == 1);


Heh, this only wants one bit (cond has type i1). Not sure if there's a nicer way to do this though.

There is none without re-adding const_to_opt_uint and you agreed on IRC that that shouldn't be done.

rebased. ping @eddyb

est31 · Nov 30, 2016

rebased. ping @eddyb

☔️ The latest upstream changes (presumably #38014) made this pull request unmergeable. Please resolve the merge conflicts.

bors · Nov 30, 2016

☔️ The latest upstream changes (presumably #38014) made this pull request unmergeable. Please resolve the merge conflicts.

Crater's been having a hard time lately. Latest run failed again. I'm trying again.

brson · Nov 30, 2016

Crater's been having a hard time lately. Latest run failed again. I'm trying again.

☔️ The latest upstream changes (presumably #37800) made this pull request unmergeable. Please resolve the merge conflicts.

bors · Nov 30, 2016

☔️ The latest upstream changes (presumably #37800) made this pull request unmergeable. Please resolve the merge conflicts.

Seems I'm running into a problem here...

When attempting to rebase this PR from basing on 5a02480 to base on dc81742 (compare changes), I can't bootstrap it anymore.

The error I'm running into (note its with stage0 still used as compiler so its not fault of this PR!) seems like a genuine zombie issue, a revival of issue #37686. Its fix (commit 84239df) is only present on the master branch, but not on the beta branch (at least not on the beta branch of commit d4aea2d on which the new bootstrap compiler as of #37800 bases on).

I'd like to hear your opinions on how to resolve this issue. I'm almost certain that other PRs in the future will run into this problem as well.

est31 · Dec 1, 2016

Seems I'm running into a problem here...

When attempting to rebase this PR from basing on 5a02480 to base on dc81742 (compare changes), I can't bootstrap it anymore.

The error I'm running into (note its with stage0 still used as compiler so its not fault of this PR!) seems like a genuine zombie issue, a revival of issue #37686. Its fix (commit 84239df) is only present on the master branch, but not on the beta branch (at least not on the beta branch of commit d4aea2d on which the new bootstrap compiler as of #37800 bases on).

I'd like to hear your opinions on how to resolve this issue. I'm almost certain that other PRs in the future will run into this problem as well.

Note: you can reproduce the issue on current master by doing ./x.py test --stage 0 src/test/run-pass --test-args 37686.

est31 · Dec 1, 2016

Note: you can reproduce the issue on current master by doing ./x.py test --stage 0 src/test/run-pass --test-args 37686.

Okay, figured out how to workaround the issue, see the last commit. So its not required anymore for this PR to get stage0 updated.

est31 · Dec 1, 2016

Okay, figured out how to workaround the issue, see the last commit. So its not required anymore for this PR to get stage0 updated.

Still working on crater.

brson · Dec 1, 2016

Still working on crater.

My crater run failed because nightly cargo is broken. I think we have to wait until nightly cargo works again.

brson · Dec 2, 2016

My crater run failed because nightly cargo is broken. I think we have to wait until nightly cargo works again.

☔️ The latest upstream changes (presumably #38055) made this pull request unmergeable. Please resolve the merge conflicts.

bors · Dec 3, 2016

☔️ The latest upstream changes (presumably #38055) made this pull request unmergeable. Please resolve the merge conflicts.

☔️ The latest upstream changes (presumably #38079) made this pull request unmergeable. Please resolve the merge conflicts.

bors · Dec 3, 2016

☔️ The latest upstream changes (presumably #38079) made this pull request unmergeable. Please resolve the merge conflicts.

Crater report shows a single regression (from changing the AST).

@bors r+

eddyb · Dec 4, 2016

Crater report shows a single regression (from changing the AST).

@bors r+

📌 Commit f60a7c2 has been approved by eddyb

bors · Dec 4, 2016

📌 Commit f60a7c2 has been approved by eddyb

⌛️ Testing commit f60a7c2 with merge e618fe0...

bors · Dec 4, 2016

⌛️ Testing commit f60a7c2 with merge e618fe0...

Rebased. r? @eddyb

est31 · Dec 15, 2016

Rebased. r? @eddyb

Keep going! :-)

leonardo-m · Dec 15, 2016

Keep going! :-)

Release is close, and this may cause regressions. It'd be better to have it at the start of a release period, not that it gets reverted before release because of the regressions it caused. Gonna reopen once the release is over.

est31 · Dec 15, 2016

Release is close, and this may cause regressions. It'd be better to have it at the start of a release period, not that it gets reverted before release because of the regressions it caused. Gonna reopen once the release is over.

See #38482 for the continuation

est31 · Dec 20, 2016

See #38482 for the continuation

And #37900 was such a rememberable number too 😢

DirkyJerky · Dec 20, 2016

And #37900 was such a rememberable number too 😢

rust-highfive assigned aturon Nov 20, 2016

est31 referenced this pull request Nov 21, 2016
Closed
rustbuild: linker error during 32 bit compilation #37906

aturon removed their assignment Nov 22, 2016

est31 changed the title from [WIP] i128 and u128 support to i128 and u128 support Nov 25, 2016

rust-highfive assigned eddyb Nov 25, 2016

eddyb added the S-waiting-on-crater label Nov 27, 2016

eddyb removed the S-waiting-on-crater label Dec 4, 2016

est31 closed this Dec 15, 2016

est31 referenced this pull request Dec 20, 2016
Merged
i128 and u128 support #38482

joewalker referenced this pull request Jan 9, 2017
Closed
Add a basic EDN parser #149

rust-lang/rust

Join GitHub today

i128 and u128 support #37900

Conversation

est31 commented Nov 20, 2016 • edited Edited 1 time est31 edited Nov 25, 2016 (most recent)

rust-highfive assigned aturon Nov 20, 2016

This comment has been minimized.

rust-highfive commented Nov 20, 2016

frewsxcv reviewed Nov 20, 2016 View changes

This comment has been minimized.

This comment has been minimized.

est31 Nov 20, 2016 • edited Edited 1 time est31 edited Nov 20, 2016 (most recent)

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

ollie27 commented Nov 20, 2016

est31 referenced this pull request Nov 21, 2016

This comment has been minimized.

bors commented Nov 21, 2016

This comment has been minimized.

aturon commented Nov 22, 2016

aturon removed their assignment Nov 22, 2016

This comment has been minimized.

est31 commented Nov 22, 2016

This comment has been minimized.

eddyb commented Nov 22, 2016

This comment has been minimized.

bors commented Nov 23, 2016

This comment has been minimized.

mattico commented Nov 23, 2016

This comment has been minimized.

est31 commented Nov 23, 2016

This comment has been minimized.

bors commented Nov 24, 2016

nagisa reviewed Nov 25, 2016 View changes

est31 changed the title from [WIP] i128 and u128 support to i128 and u128 support Nov 25, 2016

This comment has been minimized.

est31 commented Nov 25, 2016 • edited Edited 1 time est31 edited Nov 25, 2016 (most recent)

This comment has been minimized.

leonardo-m commented Nov 25, 2016 • edited Edited 1 time leonardo-m edited Nov 25, 2016 (most recent)

This comment has been minimized.

est31 commented Nov 25, 2016

This comment has been minimized.

leonardo-m commented Nov 25, 2016 • edited Edited 1 time leonardo-m edited Nov 25, 2016 (most recent)

This comment has been minimized.

alexcrichton commented Nov 25, 2016

rust-highfive assigned eddyb Nov 25, 2016

eddyb added the S-waiting-on-crater label Nov 27, 2016

This comment has been minimized.

eddyb commented Nov 27, 2016

eddyb reviewed Nov 28, 2016 View changes

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

est31 Nov 28, 2016 • edited Edited 1 time est31 edited Nov 28, 2016 (most recent)

This comment has been minimized.

est31 commented Nov 30, 2016

This comment has been minimized.

bors commented Nov 30, 2016

This comment has been minimized.

brson commented Nov 30, 2016

This comment has been minimized.

bors commented Nov 30, 2016

This comment has been minimized.

est31 commented Dec 1, 2016 • edited Edited 1 time est31 edited Dec 1, 2016 (most recent)

This comment has been minimized.

est31 commented Dec 1, 2016

This comment has been minimized.

est31 commented Dec 1, 2016 • edited Edited 1 time est31 edited Dec 1, 2016 (most recent)

This comment has been minimized.

brson commented Dec 1, 2016

This comment has been minimized.

brson commented Dec 2, 2016

This comment has been minimized.

bors commented Dec 3, 2016

This comment has been minimized.

est31 commented Nov 20, 2016 •

edited

Edited 1 time

est31 edited Nov 25, 2016 (most recent)

frewsxcv reviewed Nov 20, 2016

View changes

est31 Nov 20, 2016 •

edited

Edited 1 time

est31 edited Nov 20, 2016 (most recent)

nagisa reviewed Nov 25, 2016

View changes

est31 commented Nov 25, 2016 •

edited

Edited 1 time

est31 edited Nov 25, 2016 (most recent)

leonardo-m commented Nov 25, 2016 •

edited

Edited 1 time

leonardo-m edited Nov 25, 2016 (most recent)

leonardo-m commented Nov 25, 2016 •

edited

Edited 1 time

leonardo-m edited Nov 25, 2016 (most recent)

eddyb reviewed Nov 28, 2016

View changes

est31 Nov 28, 2016 •

edited

Edited 1 time

est31 edited Nov 28, 2016 (most recent)

est31 commented Dec 1, 2016 •

edited

Edited 1 time

est31 edited Dec 1, 2016 (most recent)

est31 commented Dec 1, 2016 •

edited

Edited 1 time

est31 edited Dec 1, 2016 (most recent)