VEP output going to stderr

[danking]

See discuss post: https://discuss.hail.is/t/redirect-or-find-vep-or-other-error-output-from-a-hail-pipeline/1308/9?u=danking

It looks like Hail isn’t capturing all the VEP output. Can someone look into this? Probably the way we’re executing external commands needs to also capture stderr and print it.

Assigning Tim for delegation.

[danking]

Tim is on vacation. dice says chris.

[chrisvittal]

Ok. What's going on here is like a proper pipe, we're consuming vep's standard error on a background thread and outputting it to System.err , then in the failure path, we're attempting to consume it again. This goes poorly. Fix incoming.

[danking]

thank you much Chris!

[konradjk]

FWIW it does go into the container logs which is how I've always pulled out the true error, but I'm not sure how to get that on every system. For future reference, there's an even more pernicious issue, which is that when running VEP with -o STDOUT it actually suppresses certain error messages too - and there's not much you can do about that unless you actually go in and run VEP manually without that, in the environment that hail uses.

[chrisvittal]

This change will make it so that it no longer goes to the container logs, and instead will be placed in the exception message.