Need to check if the std{in,err,out} file descriptors status

[anastygnome]

Hello,

I was faced with a difficulty earlier today as I needed to port an application whose behaviour depends upon the status of those three file descriptors.

I have tracked the issue down to the following function :

rust/library/std/src/sys/unix/mod.rs

Lines 81 to 133 in 247e44e

	unsafe fn sanitize_standard_fds() {
	// fast path with a single syscall for systems with poll()
	#[cfg(not(any(
	miri,
	target_os = "emscripten",
	target_os = "fuchsia",
	target_os = "vxworks",
	// The poll on Darwin doesn't set POLLNVAL for closed fds.
	target_os = "macos",
	target_os = "ios",
	target_os = "watchos",
	target_os = "redox",
	target_os = "l4re",
	target_os = "horizon",
	)))]
	'poll: {
	use crate::sys::os::errno;
	#[cfg(not(all(target_os = "linux", target_env = "gnu")))]
	use libc::open as open64;
	#[cfg(all(target_os = "linux", target_env = "gnu"))]
	use libc::open64;
	let pfds: &mut [_] = &mut [
	libc::pollfd { fd: 0, events: 0, revents: 0 },
	libc::pollfd { fd: 1, events: 0, revents: 0 },
	libc::pollfd { fd: 2, events: 0, revents: 0 },
	];
	while libc::poll(pfds.as_mut_ptr(), 3, 0) == -1 {
	match errno() {
	libc::EINTR => continue,
	libc::EINVAL | libc::EAGAIN | libc::ENOMEM => {
	// RLIMIT_NOFILE or temporary allocation failures
	// may be preventing use of poll(), fall back to fcntl
	break 'poll;
	}
	_ => libc::abort(),
	}
	}
	for pfd in pfds {
	if pfd.revents & libc::POLLNVAL == 0 {
	continue;
	}
	if open64("/dev/null\0".as_ptr().cast(), libc::O_RDWR, 0) == -1 {
	// If the stream is closed but we failed to reopen it, abort the
	// process. Otherwise we wouldn't preserve the safety of
	// operations on the corresponding Rust object Stdin, Stdout, or
	// Stderr.
	libc::abort();
	}
	}
	return;
	}

I am of the opinion that, even if the std library needs to reopen those descriptors, we should be able to get an indication that those are not the original ones. Maybe through some kind of booleans or a dedicated structure?
Related: #47271
This has also affected the uutils project.
Any hope this will get fixed ?

[anastygnome]

uutils/coreutils#2873

For example, some coreutils behaviour need to take into account this fact too.

[the8472]

See also #88825. posix requires that descriptors 0-2 are open at program startup. So any program that wants to detect a closed descriptor is non-compliant.

[clubby789]

@rustbot label +T-libs +C-enhancement

[anastygnome]

See also #88825. posix requires that descriptors 0-2 are open at program startup. So any program that wants to detect a closed descriptor is non-compliant.

Sometimes, one needs to be non-compliant. Not all environments are, and I don't believe we should prevent people from doing it.

[bvinc]

@the8472 Correct me if I'm wrong, but I believe this is a misreading of the posix standard. The only line in posix that I can find is this:

If the utility would be executed with file descriptor 0, 1, or 2 closed, implementations may execute the utility with the file descriptor open to an unspecified file. If a standard utility or a conforming application is executed with file descriptor 0 not open for reading or with file descriptor 1 or 2 not open for writing, the environment in which the utility or application is executed shall be deemed non-conforming.

It says that an environment with FD 0, 1, or 2 closed is a non-conforming environment. It does not say that a program that checks that status of these file descriptors is non-compliant.

Furthermore, this is important, as currently, it would be impossible to re-create a drop-in replacement for cat in rust, as cat has the following behavior:

$ cat /dev/null; echo $?
0
$ cat - <&-; echo $?
cat: -: Bad file descriptor
cat: closing standard input: Bad file descriptor
1

I understand why rust needs to re-open FD 0, 1, and 2 if they're closed. I would support including boolean that allow for writing programs that may want to error out if they're started with closed file descriptors.

[the8472]

Furthermore, this is important, as currently, it would be impossible to re-create a drop-in replacement for cat in rust, as cat has the following behavior

Imo std should not support recreating exact behavior in non-compliant environments. This gets us into bug-for-bug compatibility territory which would mean accumulating more and more one-offs over time.
Has anyone investigated where that behavior comes from? Is it just an implementation artifact? Why is there a test for it (regression test vs. guaranteed behavior).

Sometimes, one needs to be non-compliant.

std requires this compilance, otherwise it would lead to violations of I/O safety. In other words non-compliance - such as bypassing std's initialization code in such a state or closing the FDs later - would be unsound.

[anastygnome]

@the8472 Thanks for you reply,

I 100% agree that std should reopen the fd, and give the program a safe, POSIX compliant environnement.
that's not an option given I/O safety as you quoted. However, it may be of interest to see that std had to step in to conform to POSIX, by giving access to a flag or any other indicator -- here for example, that stdin was initially invalid, yet reopened for safety reasons.

The cat snippet is indeed guaranteed behaviour precisely for notifying the user that something is off.

[the8472]

The cat snippet is indeed guaranteed behaviour precisely for notifying the user that something is off.

But why does cat need to guarantee it, why was it added? Because some shell allows closing the fds? But then the shell is creating non-compliant environments.
I'm assuming the path here looks something like this:

• shell does non-compliant things
• cat implements a workaround for it
• a test for a thing that ought not happen in the first place gets added
• someone then takes the test as gospel that it MUST have this behavior
• uutils tries to reimplement all of the above
• uutils asks std to add bespoke API to accomodate all of this

This is dragging on technical debt and inserting it into more things rather than fixing stuff.

I think this issue could use a pinch more cathedral than bazaar.

---

As mentioned in #88825 (comment) as of posix 2017 the parent (e.g. shell) spawning a child should already be fixing up the stdio file descriptors as part of their spawn procedure if they have previously closed them. Child processes (such as cat or rust's std) doing anything about that is merely defense-in-depth and shouldn't be necessary.

[the8472]

You could try adding a static initializer and recording the state of the file descriptors before main¹. Then use that information in main().

¹you should use libc calls before main because we don't guarantee much about std there.