Miri engine: support extra function (pointer) values #62245

RalfJung · 2019-06-30T13:32:56Z

We want to add basic support for dlsym in Miri (needed to run the latest version of getrandom). For that to work, dlsym needs to return something that can be stored in a function pointer and later called.

So we add a new ExtraFnVal type to the Machine trait, and enable Miri's memory to associate allocation IDs with such values, so that create_fn_alloc and get_fn can work on both Instance (this is used for "normal" function pointers) and ExtraFnVal.

Cc @oli-obk

rust-highfive · 2019-06-30T13:33:00Z

r? @zackmdavis

(rust_highfive has picked a reviewer for you, use r? to override)

src/librustc_mir/interpret/validity.rs

zackmdavis · 2019-07-02T18:31:27Z

I don't think I'm the most qualified reviewer for this.

r? @oli-obk

RalfJung · 2019-07-02T18:36:26Z

Oli is on vacation for two weeks. @zackmdavis do you think someone else could review?

zackmdavis · 2019-07-02T18:46:56Z

Well, I guess I could try to become a more qualified reviewer. I'll take a look tonight.

r? @zackmdavis

RalfJung · 2019-07-02T18:57:12Z

Thanks a lot! And feel free to ask as many questions as you want. :) I think I should be able to explain all this code fairly well.

zackmdavis · 2019-07-03T05:47:30Z

correction: tomorrow night

bors · 2019-07-04T03:18:39Z

☔ The latest upstream changes (presumably #62355) made this pull request unmergeable. Please resolve the merge conflicts.

…determined by the machine instance. So far, however, calling such a function will fail.

…prove the error that actually gets shown a bit

RalfJung · 2019-07-04T08:49:07Z

Rebased to fix conflict.

zackmdavis · 2019-07-05T03:00:35Z

src/librustc_mir/const_eval.rs

@@ -370,6 +371,16 @@ impl<'mir, 'tcx> interpret::Machine<'mir, 'tcx> for CompileTimeInterpreter<'mir,
        }))
    }

+    fn call_extra_fn(
+        _ecx: &mut InterpCx<'mir, 'tcx, Self>,
+        fn_val: !,


So, my prior understanding was that the never (!) type is never realized (and thus is the "return type" of functions that don't return, like panic!), so I'm confused about how we can use it as a function argument? If we can't get a value of !, then how can we call this function? And if we can't call this function, what is it for?

(really sorry, I told you I wasn't the most qualified reviewer 😰 )

If we can't get a value of !, then how can we call this function? And if we can't call this function, what is it for?

We can't call this function indeed. :)
It's a trait method, and the type of fn_val is an associated type. So there might be (and, in fact, there are) other implementations of the trait that pick a non-empty type, and then call_extra_fn takes that type. For example Miri instantiates this trait, and there call_extra_fn looks like

fn call_extra_fn( ecx: &mut InterpretCx<'mir, 'tcx, Self>, fn_val: Dlsym, args: &[OpTy<'tcx, Tag>], dest: Option<PlaceTy<'tcx, Tag>>, ret: Option<mir::BasicBlock>, ) -> InterpResult<'tcx>

Notice the Dlsym instead of the !.

By using ! in the const_eval instance, I basically opt-out of the ability (that the Machine trait offers) to have "other" function values, and because there are no "other" function values, call_extra_fn does not have to do anything. This surfaces in the type of the method as you said; so we can implement the method trivially without even panicking or so, just by matching on fn_val to "prove" to the compiler that we are in unreachable code.

(really sorry, I told you I wasn't the most qualified reviewer cold_sweat )

We could ask @eddyb to also take a look if you want -- but it's hard to get a time slice from his scheduler. ;)

zackmdavis · 2019-07-05T03:09:36Z

@bors delegate+

bors · 2019-07-05T03:09:37Z

✌️ @RalfJung can now approve this pull request

eddyb · 2019-07-05T09:00:26Z

src/librustc_mir/const_eval.rs

+        _dest: Option<PlaceTy<'tcx>>,
+        _ret: Option<mir::BasicBlock>,
+    ) -> InterpResult<'tcx> {
+        match fn_val {}


I think @nikomatsakis or @canndrew brought up the idea of omitting trait methods when they're uncallable due to uninhabited arguments (and presumably they could be null in the vtable and direct calls can be replaced with unreachable).

But for now this seems fine.

eddyb · 2019-07-05T09:55:00Z

r=me if you want, most of this seems straight-forward

RalfJung · 2019-07-05T09:59:36Z

@bors r=eddyb,zackmdavis

Thanks @zackmdavis and @eddyb!

bors · 2019-07-05T09:59:37Z

📌 Commit ceb496c has been approved by eddyb,zackmdavis

@oli-obk

…kmdavis Miri engine: support extra function (pointer) values We want to add basic support for `dlsym` in Miri (needed to run the latest version of `getrandom`). For that to work, `dlsym` needs to return *something* that can be stored in a function pointer and later called. So we add a new `ExtraFnVal` type to the `Machine` trait, and enable Miri's memory to associate allocation IDs with such values, so that `create_fn_alloc` and `get_fn` can work on *both* `Instance` (this is used for "normal" function pointers) and `ExtraFnVal`. Cc @oli-obk

@ghost

Rollup of 8 pull requests Successful merges: - #60260 (Add support for UWP targets) - #62151 (Update linked OpenSSL version) - #62245 (Miri engine: support extra function (pointer) values) - #62257 (forward read_c_str method from Memory to Alloc) - #62264 (Fix perf regression from Miri Machine trait changes) - #62296 (request at least ptr-size alignment from posix_memalign) - #62329 (Remove support for 1-token lookahead from the lexer) - #62377 (Add test for ICE #62375) Failed merges: r? @ghost

@oli-obk

…kmdavis Miri engine: support extra function (pointer) values We want to add basic support for `dlsym` in Miri (needed to run the latest version of `getrandom`). For that to work, `dlsym` needs to return *something* that can be stored in a function pointer and later called. So we add a new `ExtraFnVal` type to the `Machine` trait, and enable Miri's memory to associate allocation IDs with such values, so that `create_fn_alloc` and `get_fn` can work on *both* `Instance` (this is used for "normal" function pointers) and `ExtraFnVal`. Cc @oli-obk

@ghost

Rollup of 7 pull requests Successful merges: - #62151 (Update linked OpenSSL version) - #62245 (Miri engine: support extra function (pointer) values) - #62257 (forward read_c_str method from Memory to Alloc) - #62264 (Fix perf regression from Miri Machine trait changes) - #62296 (request at least ptr-size alignment from posix_memalign) - #62329 (Remove support for 1-token lookahead from the lexer) - #62377 (Add test for ICE #62375) Failed merges: r? @ghost

use Dlsym support to implement getentropy (and better thread spawn error) This is the Miri side of rust-lang/rust#62245. Fixes #789

rust-highfive assigned zackmdavis Jun 30, 2019

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jun 30, 2019

RalfJung mentioned this pull request Jun 30, 2019

use Dlsym support to implement getentropy (and better thread spawn error) rust-lang/miri#808

Merged

RalfJung commented Jul 1, 2019

View reviewed changes

src/librustc_mir/interpret/validity.rs Outdated Show resolved Hide resolved

rust-highfive assigned oli-obk and unassigned zackmdavis Jul 2, 2019

rust-highfive assigned zackmdavis and unassigned oli-obk Jul 2, 2019

RalfJung mentioned this pull request Jul 3, 2019

miri no longer builds after rust-lang/rust#62335 #62347

Closed

RalfJung added 8 commits July 4, 2019 10:38

Add basic support for "other" kinds of values for function pointers, …

1297a27

…determined by the machine instance. So far, however, calling such a function will fail.

add machine hook to handle calls to 'extra' function values

5612feb

fix determinig the size of foreign static allocations

486720f

fix for tidy

b4be08a

make Memory::get_fn take a Scalar like most of the Memory API surface

842bbd2

use get_size_and_align to test if an allocation is live

317c6ac

turns out that dangling pointer branch is dead code; remove it and im…

d9d6b3b

…prove the error that actually gets shown a bit

improve validity error range printing for singleton ranges

ceb496c

RalfJung force-pushed the miri-extra-fn branch from 40a7c34 to ceb496c Compare July 4, 2019 08:49

RalfJung mentioned this pull request Jul 4, 2019

Add tests for Intptrcast when doing explicit casts rust-lang/miri#803

Merged

zackmdavis reviewed Jul 5, 2019

View reviewed changes

eddyb reviewed Jul 5, 2019

View reviewed changes

eddyb approved these changes Jul 5, 2019

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 5, 2019

Centril mentioned this pull request Jul 5, 2019

Rollup of 8 pull requests #62424

Closed

Centril mentioned this pull request Jul 6, 2019

Rollup of 7 pull requests #62428

Merged

bors merged commit ceb496c into rust-lang:master Jul 6, 2019

RalfJung deleted the miri-extra-fn branch July 6, 2019 08:05

bors added a commit to rust-lang/miri that referenced this pull request Jul 6, 2019

Auto merge of #808 - RalfJung:extra-fn, r=RalfJung

226156f

use Dlsym support to implement getentropy (and better thread spawn error) This is the Miri side of rust-lang/rust#62245. Fixes #789

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Miri engine: support extra function (pointer) values #62245

Miri engine: support extra function (pointer) values #62245

RalfJung commented Jun 30, 2019

rust-highfive commented Jun 30, 2019

zackmdavis commented Jul 2, 2019

RalfJung commented Jul 2, 2019

zackmdavis commented Jul 2, 2019

RalfJung commented Jul 2, 2019

zackmdavis commented Jul 3, 2019

bors commented Jul 4, 2019

RalfJung commented Jul 4, 2019

zackmdavis Jul 5, 2019

RalfJung Jul 5, 2019

RalfJung Jul 5, 2019 •

edited

Loading

zackmdavis commented Jul 5, 2019

bors commented Jul 5, 2019

eddyb Jul 5, 2019

eddyb commented Jul 5, 2019

RalfJung commented Jul 5, 2019

bors commented Jul 5, 2019

Miri engine: support extra function (pointer) values #62245

Miri engine: support extra function (pointer) values #62245

Conversation

RalfJung commented Jun 30, 2019

rust-highfive commented Jun 30, 2019

zackmdavis commented Jul 2, 2019

RalfJung commented Jul 2, 2019

zackmdavis commented Jul 2, 2019

RalfJung commented Jul 2, 2019

zackmdavis commented Jul 3, 2019

bors commented Jul 4, 2019

RalfJung commented Jul 4, 2019

zackmdavis Jul 5, 2019

Choose a reason for hiding this comment

RalfJung Jul 5, 2019

Choose a reason for hiding this comment

RalfJung Jul 5, 2019 • edited Loading

Choose a reason for hiding this comment

zackmdavis commented Jul 5, 2019

bors commented Jul 5, 2019

eddyb Jul 5, 2019

Choose a reason for hiding this comment

eddyb commented Jul 5, 2019

RalfJung commented Jul 5, 2019

bors commented Jul 5, 2019

RalfJung Jul 5, 2019 •

edited

Loading