Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Command substitutions randomly truncate large output #5611

Closed
faho opened this issue Feb 2, 2019 · 14 comments
Closed

Command substitutions randomly truncate large output #5611

faho opened this issue Feb 2, 2019 · 14 comments
Assignees
Milestone

Comments

@faho
Copy link
Member

@faho faho commented Feb 2, 2019

This is a regression since 3.0.0. It is not in Integration_3.0.1.

Running

for i in (seq 1 10000)
    echo $i
end

sometimes randomly stops before printing "10000". In particular it seems to like "5956".

@ridiculousfish: This bisects to 9a4153f.

@faho faho added the regression label Feb 2, 2019
@faho faho added this to the fish 3.1.0 milestone Feb 2, 2019
@ridiculousfish ridiculousfish self-assigned this Feb 3, 2019
@ridiculousfish
Copy link
Member

@ridiculousfish ridiculousfish commented Feb 3, 2019

Nice find. Easier repro is count (seq 1 10000)

@zanchey
Copy link
Member

@zanchey zanchey commented Feb 3, 2019

Easier repro is count (seq 1 10000)

That would make a good test, perhaps.

@ridiculousfish
Copy link
Member

@ridiculousfish ridiculousfish commented Feb 3, 2019

Fixed with added tests in 6f682c8

@faho
Copy link
Member Author

@faho faho commented Feb 18, 2019

Okay, this still happens on OpenIndiana/Illumos/Solaris/SunOS.

@ridiculousfish
Copy link
Member

@ridiculousfish ridiculousfish commented Feb 18, 2019

Hmm, which is easiest to test on? Looks like OpenIndiana has a nice Vagrant box.

@faho
Copy link
Member Author

@faho faho commented Feb 18, 2019

That's what I've been testing on (that word salad is a bit facetious - I'd love it if they could drop one or two of their names, because they still appear all over the place).

@ridiculousfish
Copy link
Member

@ridiculousfish ridiculousfish commented Feb 19, 2019

Looks like OpenSolilundiaunOS needs some _POSIX_C_SOURCE or other similar to have a thread-safe errno (!). Should be an easy fix.

@floam
Copy link
Member

@floam floam commented Feb 19, 2019

_REENTRANT probably

@ridiculousfish
Copy link
Member

@ridiculousfish ridiculousfish commented Feb 20, 2019

note you'll have to re-run cmake to pick this up

@faho
Copy link
Member Author

@faho faho commented Feb 20, 2019

I can confirm that the operating system formerly known as Solaris now seems to pick up all input.

ridiculousfish added a commit that referenced this issue Feb 20, 2019
This enables thread-safe errno on Solaris and its descendants.

Fixes #5611
@zanchey
Copy link
Member

@zanchey zanchey commented Feb 21, 2019

I suspect this was further fallout from a lack of anything analogous to AC_USE_SYSTEM_EXTENSIONS in CMake.

@floam
Copy link
Member

@floam floam commented Feb 21, 2019

So that did quite a bit:

# AC_USE_SYSTEM_EXTENSIONS
# ------------------------
# Enable extensions on systems that normally disable them,
# typically due to standards-conformance issues.
#
# Remember that #undef in AH_VERBATIM gets replaced with #define by
# AC_DEFINE.  The goal here is to define all known feature-enabling
# macros, then, if reports of conflicts are made, disable macros that
# cause problems on some platforms (such as __EXTENSIONS__).
AC_DEFUN_ONCE([AC_USE_SYSTEM_EXTENSIONS],
[AC_BEFORE([$0], [AC_COMPILE_IFELSE])dnl
AC_BEFORE([$0], [AC_RUN_IFELSE])dnl

  AC_CHECK_HEADER([minix/config.h], [MINIX=yes], [MINIX=])
  if test "$MINIX" = yes; then
    AC_DEFINE([_POSIX_SOURCE], [1],
      [Define to 1 if you need to in order for 'stat' and other
       things to work.])
    AC_DEFINE([_POSIX_1_SOURCE], [2],
      [Define to 2 if the system does not provide POSIX.1 features
       except with this defined.])
    AC_DEFINE([_MINIX], [1],
      [Define to 1 if on MINIX.])
    AC_DEFINE([_NETBSD_SOURCE], [1],
      [Define to 1 to make NetBSD features available.  MINIX 3 needs this.])
  fi

dnl Use a different key than __EXTENSIONS__, as that name broke existing
dnl configure.ac when using autoheader 2.62.
  AH_VERBATIM([USE_SYSTEM_EXTENSIONS],
[/* Enable extensions on AIX 3, Interix.  */
#ifndef _ALL_SOURCE
# undef _ALL_SOURCE
#endif
/* Enable general extensions on macOS.  */
#ifndef _DARWIN_C_SOURCE
# undef _DARWIN_C_SOURCE
#endif
/* Enable GNU extensions on systems that have them.  */
#ifndef _GNU_SOURCE
# undef _GNU_SOURCE
#endif
/* Enable threading extensions on Solaris.  */
#ifndef _POSIX_PTHREAD_SEMANTICS
# undef _POSIX_PTHREAD_SEMANTICS
#endif
/* Enable extensions specified by ISO/IEC TS 18661-5:2014.  */
#ifndef __STDC_WANT_IEC_60559_ATTRIBS_EXT__
# undef __STDC_WANT_IEC_60559_ATTRIBS_EXT__
#endif
/* Enable extensions specified by ISO/IEC TS 18661-1:2014.  */
#ifndef __STDC_WANT_IEC_60559_BFP_EXT__
# undef __STDC_WANT_IEC_60559_BFP_EXT__
#endif
/* Enable extensions specified by ISO/IEC TS 18661-2:2015.  */
#ifndef __STDC_WANT_IEC_60559_DFP_EXT__
# undef __STDC_WANT_IEC_60559_DFP_EXT__
#endif
/* Enable extensions specified by ISO/IEC TS 18661-4:2015.  */
#ifndef __STDC_WANT_IEC_60559_FUNCS_EXT__
# undef __STDC_WANT_IEC_60559_FUNCS_EXT__
#endif
/* Enable extensions specified by ISO/IEC TS 18661-3:2015.  */
#ifndef __STDC_WANT_IEC_60559_TYPES_EXT__
# undef __STDC_WANT_IEC_60559_TYPES_EXT__
#endif
/* Enable extensions specified by ISO/IEC TR 24731-2:2010.  */
#ifndef __STDC_WANT_LIB_EXT2__
# undef __STDC_WANT_LIB_EXT2__
#endif
/* Enable extensions specified by ISO/IEC TR 24747:2009.  */
#ifndef __STDC_WANT_MATH_SPEC_FUNCS__
# undef __STDC_WANT_MATH_SPEC_FUNCS__
#endif
/* Enable extensions on HP NonStop.  */
#ifndef _TANDEM_SOURCE
# undef _TANDEM_SOURCE
#endif
/* Enable X/Open extensions if necessary.  HP-UX 11.11 defines
   mbstate_t only if _XOPEN_SOURCE is defined to 500, regardless of
   whether compiling with -Ae or -D_HPUX_SOURCE=1.  */
#ifndef _XOPEN_SOURCE
# undef _XOPEN_SOURCE
#endif
/* Enable general extensions on Solaris.  */
#ifndef __EXTENSIONS__
# undef __EXTENSIONS__
#endif
])
  AC_CACHE_CHECK([whether it is safe to define __EXTENSIONS__],
    [ac_cv_safe_to_define___extensions__],
    [AC_COMPILE_IFELSE(
       [AC_LANG_PROGRAM([[
#         define __EXTENSIONS__ 1
          ]AC_INCLUDES_DEFAULT])],
       [ac_cv_safe_to_define___extensions__=yes],
       [ac_cv_safe_to_define___extensions__=no])])
  test $ac_cv_safe_to_define___extensions__ = yes &&
    AC_DEFINE([__EXTENSIONS__])
  AC_DEFINE([_ALL_SOURCE])
  AC_DEFINE([_DARWIN_C_SOURCE])
  AC_DEFINE([_GNU_SOURCE])
  AC_DEFINE([_POSIX_PTHREAD_SEMANTICS])
  AC_DEFINE([__STDC_WANT_IEC_60559_ATTRIBS_EXT__])
  AC_DEFINE([__STDC_WANT_IEC_60559_BFP_EXT__])
  AC_DEFINE([__STDC_WANT_IEC_60559_DFP_EXT__])
  AC_DEFINE([__STDC_WANT_IEC_60559_FUNCS_EXT__])
  AC_DEFINE([__STDC_WANT_IEC_60559_TYPES_EXT__])
  AC_DEFINE([__STDC_WANT_LIB_EXT2__])
  AC_DEFINE([__STDC_WANT_MATH_SPEC_FUNCS__])
  AC_DEFINE([_TANDEM_SOURCE])
  AC_CACHE_CHECK([whether _XOPEN_SOURCE should be defined],
    [ac_cv_should_define__xopen_source],
    [ac_cv_should_define__xopen_source=no
     AC_COMPILE_IFELSE(
       [AC_LANG_PROGRAM([[
          #include <wchar.h>
          mbstate_t x;]])],
       [],
       [AC_COMPILE_IFELSE(
          [AC_LANG_PROGRAM([[
             #define _XOPEN_SOURCE 500
             #include <wchar.h>
             mbstate_t x;]])],
          [ac_cv_should_define__xopen_source=yes])])])
  test $ac_cv_should_define__xopen_source = yes &&
    AC_DEFINE([_XOPEN_SOURCE], [500])
])# AC_USE_SYSTEM_EXTENSIONS

@floam
Copy link
Member

@floam floam commented Feb 21, 2019

It seems like we may want to be doing __EXTENSIONS__, _ALL_SOURCE, _DARWIN_C_SOURCE, _GNU_SOURCE, _POSIX_PTHREAD_SEMANTICS at least. I'm unfamiliar with features enabled with __STDC_* but I think using C++ they shouldn't do anything?

@floam
Copy link
Member

@floam floam commented Feb 21, 2019

but I think using C++ they shouldn't do anything?

It seems that __STDC_WANT_LIB_EXT2__ can make these available in string.h/stdio.h:

  • fmemopen
  • open_memstream
  • vasprintf
  • asprintf
  • getdelim
  • getline
  • strdup
  • strndup
  • open_wmemstream

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Apr 17, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

4 participants