Support 16-bit pointers as well as i/usize

[shepmaster]

I'm opening this pull request to get some feedback from the community.

Although Rust doesn't support any platforms with a native 16-bit pointer at the moment, the AVR-Rust fork is working towards that goal. Keeping this forked logic up-to-date with the changes in master has been onerous so I'd like to merge these changes so that they get carried along when refactoring happens. I do not believe this should increase the maintenance burden.

This is based on the original work of Dylan McKay (@dylanmckay).

[rust-highfive]

r? @nrc

(rust_highfive has picked a reviewer for you, use r? to override)

[alexcrichton]

This all seems pretty straightforward and easy to land, but in terms of keeping this up to date is there a way that we could add a simple regression test? There's a lot more places than I would have expected that need to be updated for 16-bit pointers, and ensuring that any new location stays up to date may be difficult.

[shepmaster]

is there a way that we could add a simple regression test

I'm 100% behind this, but I'm pretty stumped on how to do it. It seems like I need to be able to say "hey, this platform has 16-bit pointers", but there aren't any that do so (for now).

Do you think there's a way I could use a target description JSON file, specify "target-pointer-width": "16", and have it work on (say) x86? Then I could just do a bunch of math-like stuff, each corresponding to one of the changes being made?

a lot more places than I would have expected

You're telling me! 🌴 Unfortunately, most of them were found The Hard Way — by finding a compiler ICE building the compiler or my end code. It's a shame that there wasn't more "hey you forgot this enum value here", but most cases (reasonably) have a default value case.

[nagisa]

The way I remember, on 16-bit systems you usually would have a distinction between far (segment+ptr) and near pointers, which seems to not be addressed here. What gives?

[shepmaster]

@nagisa I have heard tales of Ye Olde Days of near and far pointers, but I'm not sure that AVR has need for it, as it isn't a segmented memory architecture. There's some references to it when one needs to access > 64K of memory, but those seem to just be "regular" 32-bit values.

[dylanmckay]

@shepmaster is right in saying AVR doesn't use segmented memory. Normal pointers are the usual 16 bits. There are a few AVR chips with a bit more memory that 16 bits can address - to access memory higher than this, there's a special IO register for which you must load the other address bits into first. It still works as an ordinary pointer, but spread over two registers.

When you try and address memory with a 32-bit pointer, the compiler does the special IO register loading for you.

[shepmaster]

a way I could use a target description JSON file

Mmm, that doesn't seem to work because it ultimately looks like cross compiling and there's no libcore or libstd for that made up platform. I'm open to any suggestions!

[shepmaster]

And If I try to follow the lead of the target-specification tests and eschew libstd and libcore, I end up triggering an LLVM assertion:

LLVM ERROR: Cannot select: 0x10e629000: i16 = FrameIndex<0>
In function: _ZN4main5_main17h49c054cd6e240b5cE

[shepmaster]

I can get a tiny bit of test with this:

let _a: usize = 0x1_0000; // literal out of range for usize

Which indicates that, yes, 16-bit pointers only take up 16-bits. I'm not sure of the value of the test, but I'll continue poking at it. I'd still appreciate suggestions that anyone has for useful tests.

[alexcrichton]

Hm yeah I suspect that our main recourse for testing a patch like this would be adding automation which generated at least libcore for a 16-bit target.

Also, forgot to do this earlier, but cc @rust-lang/compiler and @rust-lang/tools

[eddyb]

Neat.

[alexcrichton]

The tools team discussed this PR during triage yesterday and the conclusion was that from our perspective this seems fine. This would basically just be expanding our tier 3 platforms, which we regularly take PRs for, so no sweat on that front!

[shepmaster]

@alexcrichton that's great! I found one more tricky spot that needs to be updated, I can rebase and squash that to this commit and it should be good-to-merge then!

[shepmaster]

@alexcrichton was there any comment one way or the other about tests that would need to be added before the merge?

[alexcrichton]

At least from a tooling perspective we were ok not having a bunch of rigorous tests or anything, none of our tier 3 platforms do either :)

For merging, though, this cuts across a few other teams as well, so at least @rust-lang/compiler will probably want to sign off as well

[shepmaster]

I've fixed up the one extra spot I've found and rebased for good measure.

[nikomatsakis]

I gave it a quick read. Looks like a fairly straight-forward patch; I've not thought deeply about what else might be needed to support 16 bit platforms, but this at least doesn't look like it would harm 32 or 64 bit. 👍

[alexcrichton]

The libs team discussed this briefly at yesterday's triage and the conclusion was that from that perspective we're ok along the lines of "let's see how this plays out without bending over backwards too much"

[pnkfelix]

I'm cool with us landing this. Might give us more street cred with micro controller community

[shepmaster]

Thanks for the positive reviews all around! What other teams remain to sign off?

[alexcrichton]

Judging by the area of change, I'd expect that at least the compiler team should weigh in. If they'd like to escalate, however, we can discuss at the core meeting.

[pnkfelix]

@nikomatsakis we did discuss at compiler meeting (twice I think, right?)

We didn't really attempt to form any solid consensus (we almost never have a formal quorum anyway), but no one present objected...

[nikomatsakis]

@pnkfelix indeed, we did discuss, and everybody was more-or-less in favor of doing this on an experimental basis.

[Aatch]

I think the conclusion is that this is ok.

@bors r+

[bors]

📌 Commit bc7595c has been approved by Aatch

[bors]

⌛️ Testing commit bc7595c with merge 7738479...

[bors]

☀️ Test successful - auto-linux-32-nopt-t, auto-linux-32-opt, auto-linux-32cross-opt, auto-linux-64-cargotest, auto-linux-64-cross-armhf, auto-linux-64-cross-armsf, auto-linux-64-cross-freebsd, auto-linux-64-cross-netbsd, auto-linux-64-debug-opt, auto-linux-64-nopt-t, auto-linux-64-opt, auto-linux-64-opt-mir, auto-linux-64-opt-rustbuild, auto-linux-64-x-android-t, auto-linux-cross-opt, auto-linux-musl-64-opt, auto-mac-32-opt, auto-mac-64-nopt-t, auto-mac-64-opt, auto-mac-64-opt-rustbuild, auto-mac-cross-ios-opt, auto-win-gnu-32-nopt-t, auto-win-gnu-32-opt, auto-win-gnu-32-opt-rustbuild, auto-win-gnu-64-nopt-t, auto-win-gnu-64-opt, auto-win-msvc-32-cross-opt, auto-win-msvc-32-opt, auto-win-msvc-64-cargotest, auto-win-msvc-64-opt, auto-win-msvc-64-opt-mir, auto-win-msvc-64-opt-rustbuild