miri no longer builds after rust-lang/rust#66547

[rust-highfive]

Hello, this is your friendly neighborhood mergebot.
After merging PR #66547, I observed that the tool miri has failing tests.
A follow-up PR to the repository https://github.com/rust-lang/miri is needed to fix the fallout.

cc @leo60228, do you think you would have time to do the follow-up work?
If so, that would be great!

cc @dtolnay, the PR reviewer, and nominating for compiler team prioritization.

[dtolnay]

This test is failing: https://github.com/rust-lang/miri/blob/3bf51f591f7ffdd45f51cd1d42a4002482af2bd5/tests/run-pass/args.rs

normalized stdout:


expected stdout:
args


diff of stdout:

-args
-

[dtolnay]

rust-lang/miri#624 might give a hint what needs to be updated. I haven't gotten a chance to look closely yet.

[RalfJung]

Yeah, we'll need to emulate that new static thing added by #66547. Unfortunately I don't entirely understand the static magic that goes on there... there's just a fn ptr stored in a particular section and somehow that is enough?!?

[eddyb]

Yes, all platforms have sections that they treat as an array of fn pointers, filling the entire section, which makes it easy to append more entries, as linkers will happily concatenate sections with the same name.
This is how C++ global constructors are implemented everywhere.

So I guess there's two levels to this:

• miri should collect the platform-specific global ctors/dtors (on Linux I think there's also init/fini which work similarly) and execute them before C main (and after, in the case of dtors)
• when targer_env is "gnu", the ctors (or just the init flavor?) can also take 1, 2 or 3 arguments, not only 0 arguments (I believe this is also how C main technically works)

[RalfJung]

miri should collect the platform-specific global ctors/dtors (on Linux I think there's also init/fini which work similarly) and execute them before C main (and after, in the case of dtors)

Okay. This is basically rust-lang/miri#450. I hoped we'd never have to do this...

For dtors, we already run TLS dtors, but those are entirely separate (we hook the pthread calls where dtors are set, and then run those functions later). There's no way those can be unified to one mechanism, is there?

Also, this begs the question in which order stuff is called before/after main.

[eddyb]

Also, this begs the question in which order stuff is called before/after main.

C++ asks itself the same question and I personally haven't heard of there being an answer.

The safe thing to do is to make sure any order is safe (in Rust you have to initialize global state before any runtime initialization, so it's safer than C++ where before the constructor runs, the fields are uninitialized).

The implementations just go through the sections in order, so the unknown quantity is what order the linker concatenates all of these sections.
In miri you don't have the linker to think about, but you do need a mechanisms to access all statics from all crates.

[eddyb]

Wait, why can't libstd initialize the args from the start lang item as well? Surely there's an inexpensive way to do so?

[leo60228]

@eddyb That's what I did originally, but @joshtriplett said not to.

[leo60228]

Reverting d448ab0 should fix this.

[RalfJung]

We could either adjust the cfg's to no use the new method when cfg(miri) is set, or we could implement those initializers. Seems like we'll want them eventually anyway...

you do need a mechanisms to access all statics from all crates.

What would be the best way to do that?

[joshtriplett]

@RalfJung In the short term, excluding cfg(miri) from the new mechanism seems like a reasonable solution to unbreak miri.

Also, this begs the question in which order stuff is called before/after main.

That depends on two factors: what order functions get placed into the sections, and what order they get called.

• The default linker scripts will collect all .init_array.* sections in sorted order, followed by the .init_array section itself, into a single .init_array section. Likewise for .fini_array.* and .fini_array. The sort order here uses the linker's SORT_BY_INIT_PRIORITY, which takes a numeric section suffix as a numeric priority, and sorts sections with non-numeric suffixes asciibetically.
  • (There are also a couple of special cases to make sure that glibc can guarantee its own functions run first/last in a couple of those. And there are some very careful special cases to handle the old .ctors.*, .ctors, .dtors.*, and .dtors sections, which get translated into corresponding .init_array.*, .init_array, .fini_array.*, and .fini_array sections respectively in the final binary to preserve the behavior of those sections, keeping in mind that ctors sections get run in reverse order and the dtors sections get run in forwards order. SORT_BY_INIT_PRIORITY translates the priority in numbered ctors and dtors sections by subtracting it from 65535.)
• When loading a program or library, glibc calls the function pointers in .preinit_array in order, then the (single) _init function (which executes the contents of the .init section), then the function pointers in init_array in order. At shutdown, glibc calls the function pointers in fini_array in reverse order, then the (single) _fini function (which executes the contents of the .fini section).

Does that help?

[RalfJung]

Does that help?

It is a great reference, thanks! So far I have no idea how to wire up all these sections in Miri, but I guess we'll see about that. ;)

---

On a separate note, #66697 also broke Miri by removing the injected_panic_runtime getter from tcx. I am not sure yet if there is a replacement that we can use instead. Leaving that here just so that I hopefully can find it later.

[RalfJung]

The safe thing to do is to make sure any order is safe (in Rust you have to initialize global state before any runtime initialization, so it's safer than C++ where before the constructor runs, the fields are uninitialized).

I don't see any way to actually do that.

[joshtriplett]

@eddyb I would prefer to avoid duplicate initialization of arguments. I'd like to make Rust executables smaller, not larger.

[dtolnay]

In the short term, excluding cfg(miri) from the new mechanism seems like a reasonable solution to unbreak miri.

I agree, I would accept this change.

[RalfJung]

I opened a PR for that: #66873

[dtolnay]

Done. I set it to close this issue since we already have rust-lang/miri#450 to track the more elaborate approach.

[RalfJung]

Well, actually closing the issue will require also bumping the submodule to include rust-lang/miri#1085.

[eddyb]

@eddyb I would prefer to avoid duplicate initialization of arguments. I'd like to make Rust executables smaller, not larger.

I don't understand, this would only be a few bytes with #[inline(never)]?

We could either adjust the cfg's to no use the new method when cfg(miri) is set,

To unbreak miri it's probably fine but I'm increasingly worried about cfg(miri), as it allows divergence to exist in ways which are not controlled within miri itself.

[eddyb]

The safe thing to do is to make sure any order is safe (in Rust you have to initialize global state before any runtime initialization, so it's safer than C++ where before the constructor runs, the fields are uninitialized).

I don't see any way to actually do that.

@RalfJung I was talking about a user of these facilities, not miri implementation.
To expand a bit (although it's kind of offtopic):
Rust forces you to mutate a static that started out with a constant value (e.g. Mutex::new(None)), which means nothing else would observe an uninitialized value, just the initial constant value, if it happened to ran before the runtime initialization (which would e.g. replace the None with a Some).
This makes Rust + .init_array safer in general than C++ with global constructors.

[dtolnay]

Well, actually closing the issue will require also bumping the submodule to include rust-lang/miri#1085.

Ack.

[RalfJung]

Aand now we also need to wait for #66896 to land. That's the third independent Miri-breaking PR in 24h, impressive. ;)