Skip to content

v0.16.5: stable release that brings many improvements to 0.16.x

Latest
Latest
Compare
Choose a tag to compare
@McDutchie McDutchie released this 27 Jun 17:36
· 439 commits to master since this release
v0.16.5
bcbcab3

The installer can now bundle a reduced copy of modernish with your
scripts, so they can run portably with a known version of modernish
without requiring prior installation.

This version should not break existing modernish scripts. The new
features selected for backporting to this release seemed like
necessary improvements because they fix important shortcomings in
modernish. From now on, development of new features happens on the
master branch and 0.16.x only gets bugfixes. I mean it this time :)

This version is tested on AIX (7.1), Cygwin, DragonFlyBSD, FreeBSD,
HP-UX (B.11.11), Illumos (DilOS, OmniOS, OpenIndiana), Interix,
Linux (Debian, Gentoo, NixOS, Slackware, Ubuntu), macOS / Mac OS X,
NetBSD, OpenBSD, QNX (6.5), and Solaris (10.1-11.4). Testing on
other systems is wanted!

Some of the less common UNIX systems' default shells are not
compliant with the latest POSIX spec. If modernish reports fatal
shell bugs, make an alternative shell available in $PATH.

Differences between 0.16.3 and 0.16.5

Below is a summary of changes, optimisations and fixes.
See the git log for the complete overview with explanations:
git clone -b 0.16 https://github.com/modernish/modernish
cd modernish
git log -p --reverse v0.16.3..v0.16.5

Changed:

  • NEW: install.sh -B: bundle the modernish library with your
    scripts.
    This version adds a -B (Bundle) option to install.sh which
    allows bundling any number of scripts with an reduced copy of
    modernish into a separate directory tree (indicated using the -D
    option) that is ready to run, archive and distribute. This allows
    scripts to run immediately and portably with a known version of
    modernish, even on systems that don't have modernish installed.
    You can indicate a preferred shell with the -s option or you
    can have your script be shell-agnostic. Even if a preferred shell
    is indicated, modernish will fall back to other shells if the
    preferred one isn't available. It is left to the script to check
    for required features (typically with thishellhas).
    Details: README.md, Appendix F

  • var/loop/find: Enhancements for compatibility and the use of
    nonstandard primaries:

    • Now allows specifying, as an argument to use var/loop/find, a
      preferred find utility to use for the find loop. For example:
      use var/loop/find bsdfind # prefer utility by this name
      use var/loop/find /opt/local/bin # look for a utility here first
      use var/loop/find /opt/bin/gfind # try this one first
    • New --try option: try an expression primary. It makes "unknown
      primary" errors non-fatal: upon encountering a primary that is
      not supported by the utility, the loop will quietly abort with
      exit status 128, leaving the -name of the invalid primary in
      the REPLY variable. This allows a script to test for the
      presence of certain primaries and decide what to do.
    • New optional compatibility mode for obsolete/broken find
      utilities. Passing the -b or -B option to use var/loop/find
      allows broken find as a last resort; modernish will fall back
      to older and very inefficient methods using -exec ... {} \;.

  • var/local, var/loop/for, var/loop/select, var/loop/find:
    New --base option: prefix a string/directory before doing --split
    or --glob. This provides a way to have an untrusted prefix part
    of a path or string not subject to split or glob while doing
    these expansions in safe mode. For example, this is more secure
    when expanding wildcards on files within a base directory path
    input by the user, or passed from untrusted data.
    Rationale: 739c959

  • The var/mapr module is renamed to sys/cmd/mapr. A compatibility
    redirect is left so that use var/mapr will quietly continue to
    work in modernish 0.16.x. After 0.16.x, this will display a
    warning; still later, it will be removed.
    Rationale: 92fea83

    • In addition, sys/cmd/mapr now depends on sys/cmd/procsubst,
      and no longer depends on sys/cmd/extern.

  • sys/cmd/harden: The harden function now will not harden read.
    Rationale: 62862b2

  • thisshellhas:

    • There are no more "internal" bug/feature/quirk detection tests;
      all are now implemented as cap/*.t detection scripts.
    • The output of thisshellhas --show is now sorted.

  • Initialisation for non-interactive shells (i.e. scripts):

    • Ensure IFS (field splitting) has a standard default value; this
      is to counteract /bin/sh (dash) on Debian-based systems
      inheriting IFS from the environment, which is a glaring attack
      vector on those systems.
    • Avoid inheriting CDPATH, so that the cd builtin is not
      influenced.

  • New shell features detected: PROCREDIR, PROCSUBST, TESTERE

  • New shell bugs detected: BUG_ALIASCSHD, BUG_ALIASPOSX,
    BUG_CDPCANON, BUG_CSUBRMLF, BUG_DEVTTY, BUG_LNNONEG,
    BUG_PSUBIFSNW, BUG_PSUBIFSWH, BUG_SHIFTERR0, BUG_ZSHNAMES,
    BUG_ZSHNAMES2

Optimised:

  • var/loop: use PROCSUBST (bash/ksh/zsh process substitution) or
    PROCREDIR (yash process redirection) where available. This makes
    loop entry faster on shells with one of these features, and more
    reliable on buggy kernels with a FIFO-related race condition.
    Details: 4279d67
    92844ef

  • var/loop/find: LOOP find handles loop entry considerably faster
    and generates loop iterations more efficiently.
    Details: 95417d4

  • shellquote:

    • Improved size optimisation for multiple levels of quoting.
    • Allow full size optimisation for shellquote -P.
      Details: 98e0848

Fixed:

  • Modernish is now tested on two new systems:

    • AIX 7.1.0. Requires GNU utilities (from "AIX Toolbox for Linux
      Applications") in /opt/freeware/bin; its default utilities are
      not up to the recent POSIX standard. On AIX, modernish now
      prefixes /opt/freeware/bin to $DEFPATH.
    • HP-UX B.11.11. Requires adding a recent POSIX-compliant shell.
      Thanks to polarhome.com for providing shell accounts on these
      operating systems.

  • Modernish 0.16.x will now once again initialise on ancient mksh
    versions with BUG_NOOCTAL + BUG_CMDSPEXIT (such as mksh R44).

  • use: When querying (-q or -e options), use now doesn`t throw
    a fatal error on encountering an invalid module name, but returns
    exit status 2 instead.

  • thisshellhas:

    • Fixed: On bash versions compiled with "minimal configuration",
      thisshellhas falsely detected a builtin command if a readonly
      function by that name was set.
    • Fixed: On mksh <= R50f (with BUG_CMDVRESV), thisshellhas had
      become incapable of detecting shell reserved words/keywords.

  • shellquote: Security fix: in "double" quoting, control
    characters following a $, `, , or " were not translated to $CC*
    expansions, so a quoted string could still span lines.

  • str ematch: awk fallback version: add bug-compatibility with
    Busybox awk.
    Details: d188787

  • On bash 5.0 and later, modernish now sets the localvar_unset
    shopt on initialisation, so the shell no longer has the
    treacherous QRK_LOCALUNS2 shell quirk.

  • Bash on NetBSD crashes when detecting DBLBRACKETERE on NetBSD in
    a UTF-8 locale. Fix: make this crash non-fatal for modernish.
    Details: 4ff76e8

  • The PSREPLACE feature is broken on mksh versions <= R51. At least
    $CCa (\7) fails to be replaced. Modernish now will not detect
    this feature on these mksh versions. Among other things, this
    fixes shellquote on them.

  • sys/base/readlink:

    • Fixed: on macOS, readlink died with Internal Error 2 on a
      symlink with no read permission (which macOS enforces).
    • Fixed: readlink -m was canonicalising incorrectly when
      traversing through a recursive symlink as a pathname component.
    • Fixed: readlink -f/-e/-m did not resolve symlinks in the
      $PWD when given a current-dir filename like foo or ./foo.

  • sys/base/tac:

    • Fix -r: accept a proper POSIX Extended Regular Expression as
      documented (instead of an awk regular expression).
      Details: 57e661d
    • Bugfix: a non-regex field separator of \ was broken.
    • Bugfix: a single-character field separator was not interpreted
      as a regular expression even if -r was given.
    • Bugfix: tac failed to throw a fatal error if awk failed.

  • sys/base/which: which -a now avoids duplicate output for
    duplicate $PATH entries.

  • sys/cmd/extern, sys/cmd/harden:
    Fixed: Empty $PATH elements were not correctly handled as
    equivalent to . (the present working directory).

  • sys/cmd/mapr (formerly var/mapr):
    Fixed: Severe bug that caused data loss in an input steam > 4 MB.
    Details: dae120b

  • var/loop/find: Fixed use of -ok and -okdir in LOOP find:

    • Fixed: If -iterate was used twice after -ok or -okdir within the
      same group of parentheses, LOOP find would die with a syntax
      error.
    • Fixed: When -ok or -okdir are used, the loop disables an
      internal optimisation so it iterates immediately after positive
      user input. This effect is no longer limited to the current set
      of parentheses.
    • LOOP find now correctly handles the -ok somecommand {} +
      extension that some find utilities support.

  • var/loop/find: Init now rejects find utilities that cannot
    correctly handle more than one -exec ... {} + clause (which
    caused multiple -iterate primaries to malfunction). Utilities
    with this bug include the standard find on HP-UX B.11.11.

  • var/stack/trap, var/local: Fixes for use on an interactive shell.
    Details: a111a3c

  • Fixes for install.sh:

    • Now require the user to confirm the response with the Return
      key for interactive yes/no questions. Before, install.sh used
      sys/term/readkey to only require a single keystroke. But that
      was doing more harm than good: users might still press Return
      and accidentally give the default answer to the next question.
    • Fixed: Relaunching the installer to run with another shell
      failed if the present working directory was outside of the
      source directory.
    • Install all top-level documentation files into
      share/doc/modernish, including EXAMPLES.md.
    • For bash, add -p to hashbang path in bin/modernish so
      portable-form scripts aren't at risk from inheriting exported
      functions from the environment.
    • Various fixes for safe directory handling for the -D (destdir)
      option.
    • Fix handling of relative path arguments on the command line.
    • Allow installing in directories pointed to by symlinks.
    • Cygwin compatibility fix: Avoid double initial slash if the
      installation root directory or $HOME is /.
    • Fixed: due to a typo, the $MSH_SHELL path was not ensured to be
      absolute.
    • Can now be invoked by /bin/sh on Solaris 10 and other ancient
      Bourne shells, as long as a POSIX shell is in getconf PATH.
Assets 2
Loading