Tests output to global stdout

[mathstuf]

When running a test with cargo test which uses rayon, workers which use print! (e.g., from a stub logging implementation) skip the test-local output and don't get buffered/hidden/etc.

Is there some way using the scope API to set the output handler for the thread pool in use? I'm guessing something new would need to be added. Note that it would seem to require the set_stdio unstable feature, but I'm interested in having some way in rayon available once it is stable.

[cuviper]

Note that it would seem to require the set_stdio unstable feature, but I'm interested in having some way in rayon available once it is stable.

Also note the reason given on that unstable feature: "this function may disappear completely or be replaced with a more general mechanism", with no tracking issue.

Even with that, it's complicated. We would have to update this every time we steal a job, including when a job is pulled from the global injector queue (coming from outside the threadpool). Save the old value, clone the remote value (how?) to set locally, then restore the saved value on return. Writes would have to be synchronized too, because the handle as-is only requires Box<dyn Write + Send>.

[mathstuf]

Within a scope, would job-stealing be a thing since it has its own local thread pool? The global thread pool would still be outputting directly. Though if this isn't executed on the scope's thread pool and instead uses the global pool, that would have to be tackled for this issue to be resolved.

rayon::scope(|_| { some_vec.par_iter().for_each(|i| i.method();) })

[cuviper]

A scope does not imply a separate threadpool. Calling rayon::scope will use the "current" pool, which can be any pool if you're already in a rayon thread, or else it's the global pool. Calling ThreadPool::scope uses that specific pool, of course.

[cuviper]

For reference, there's also rust-lang/rust#42474. But as you noted there, they probably couldn't deal with a threadpool like Rayon automatically.

[mathstuf]

I guess an example would be best.

Basically, it'd be nice to be able to do:

#[test]
fn my_test() {
    // This API doesn't exist either.
    // This code could be wrapped up in a `test_pool()` utility function though.
    let output = std::stdio::get_stdio();
    let pool = rayon::ThreadPoolBuilder::new().with_output(&output).build();
    pool.install(|| my_rayon_using_fn());
}

Would that with_output method on the builder be possible?

[cuviper]

With some appropriate get/set methods from std, yes, it should be straightforward to assign those outputs for an entire threadpool.

[cuviper]

I think a start handler could also set the output, without any changes required in Rayon.