New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
limit how much data we read from a command substitution #3822
Comments
Also (don't know if this should be a separate issue), ctrl+c doesn't work while a command substitution is running, leaving no method of escaping from a bad substitution. |
Yes, @ChrisJefferson, that is a distinct issue separate from this one. In fact, it's already been reported. See issue #3527. |
thanks, didn't want it lost. Sorry. |
ARG_MAX is probably of interest here. See e.g. http://www.in-ulm.de/~mascheck/various/argmax/ and http://pubs.opengroup.org/onlinepubs/7908799/xsh/limits.h.html. It is defined as "Maximum length of argument to the exec functions including environment data. Minimum Acceptable Value: _POSIX_ARG_MAX ". So that means once we exceed it, we can't rely on external commands being launchable anymore. We could allow more for our builtins, so instead of doing one invocation of an external command with a huge amount of data you'd store it in a variable and then run the external command with smaller chunks. But the goal here isn't to disallow "illegal" invocations, it's to avoid a hung shell, so we should probably just pick a high limit and use it for everything. If you need to e.g. remove a gazillion files, use (On my system, ARG_MAX is 2MiB, btw) |
We should not distinguish between the ARG_MAX limit for external commands and builtins (or functions) not subject to that limit. Doing so just leads to inconsistencies that confuses users. At least with respect to limiting the text interpolated by a command substitution. Not least because other text on the command line outside of the command substitution could easily result in ARG_MAX being exceeded. The two failure modes are independent of each other. Should we use the maximum of ARG_MAX or 10 MiB for command substitution? I vote no for consistency with the |
Which bit are you voting no to? :-) |
No to using ARG_MAX. Using that limit for any single command substitution would not guarantee that an external command could be launched because other arguments on the command line could push us over the OS limit. Too, on many systems ARG_MAX can be quite small (e.g., a few tens of KB). It would also be inconsistent with the recently imposed limit for the |
Should this perhaps be something queryable/set through our |
Yes, but only if we can be reasonably confident it won't need to be set prior to reading the users ~/.config/fish/config.fish. That's because all the other ulimit options affect child processes whereas this would not. So using |
Since no one has argued for not implementing a limit on how much data we will accept from a command substitution I'm going to remove the RFC label. The only question I think still needs an answer is what to do if the limit is exceeded for a non-interactive shell. We should obviously emit a warning to stderr whether or not the shell is interactive. But if non-interactive should the shell immediately exit or simply treat that command substitution has having failed with a non-zero status (and zero characters substituted)? |
I think the behaviour should mirror that of |
We snuck in the same change to the Also, it is a good sign that no one has reported a problem with the 10 MiB limit on |
This makes command substitutions impose the same limit on the amount of data they accept as the `read` builtin. It does not limit output of external commands or builtins in other contexts. Fixes #3822
At the moment we don't impose any limits on the amount of data that we accept from
read
or a command substitution (e.g.,(cmd)
). Issue #3712 proposes setting a limit of 100 MiB for any singleread
execution. This issue is to discuss whether the same, or a similar, limit should be imposed on command substitution output. See issue #3817 for examples of what can happen without such a limit.The text was updated successfully, but these errors were encountered: