main: work around sourcing "ble.sh" inside subshells

CONTRIBUTING.md

@@ -1,6 +1,8 @@
 # Contribution Guide
 
-## Issues
+## How to contribute
+
+### Issues
 
 You can freely create an issue using the following links:
 
@@ -9,21 +11,109 @@ You can freely create an issue using the following links:
 - Feature request [[Here]](https://github.com/akinomyoga/ble.sh/issues/new?template=help.md)
 - Others (suggestions, projects, discussion, complaints, news, information or anything) [[Here]](https://github.com/akinomyoga/ble.sh/issues/new?template=free_style.md)
 
-## Pull requests
+### Pull requests
 
 We always welcome following types of pull requests. Any changes will be considered to be provided under the BSD 3-Clause License.
 If you do not know whether your changes would be appropriate for merge, please feel free to create a pull request and let us talk with each other!
 
 - Better translation to English, typo fixes
 - Fixes, optimization, test cases
-- Feature suggestions
-- New color theme
+- New features
+- New color themes ... We accept new themes in [`contrib`](https://github.com/akinomyoga/blesh-contrib/pulls) repository.
 - Others
 
-## Wiki
+### Wiki
 
 You can freely edit [wiki pages](https://github.com/akinomyoga/ble.sh/wiki).
 
 - Translations
 - Typo fixes
 - Create new pages
+
+## For package maintainers
+
+If you are a package maintainer for a repository of Linux distribution, etc.
+you may provide a package-specific setting by preparing
+a file `/path/to/blesh/lib/_package.sh` (e.g. `/usr/share/blesh/lib/_package.sh`)
+which will be sourced after the load of `ble.sh` just before sourcing user's configuration (`~/.blerc`).
+
+- In the file, the shell variable `_ble_base_package_type=TYPE` should be
+   set up to have a repository-specific name (such as `AUR`). 
+
+- The function named `ble/base/package:TYPE/update` (where `TYPE`
+  matches with a value assigned to `_ble_base_package_type`) may be
+  provided to define a custom updating procedure.  The exit status of the function can be
+  - `0` ... when the update successed
+  - `6` ... when the update was skipped because the package was up to date.
+  - `125` ... when it wants to fallback to the built-in updating procedure of `ble.sh`.
+  - Other ... when the update failed
+
+An example `lib/_package.sh` might be
+
+```bash
+_ble_base_package_type=apt
+
+function ble/base/package:apt/update {
+  sudo apt upgrade blesh
+}
+```
+
+You can also find a real example for AUR (Arch User Repository) [here](https://aur.archlinux.org/cgit/aur.git/tree/blesh-update.sh?h=blesh-git).
+
+## Summary of codebase
+
+The core script file `ble.sh` is generated by combining the following files:
+
+- `ble.pp` ... Basic initialiation
+- `src/def.sh` ... Prototype definitions
+- `src/util.sh` ... Basic utility functions
+- `src/decode.sh` ... User-input decoder and keybindings
+- `src/color.sh` ... Terminal graphic attributes
+- `src/canvas.sh` ... Terminal layout engine
+  - `src/canvas.emoji.sh` ... Emoji database
+- `src/history.sh` ... Command history management
+- `src/edit.sh` ... Line editor
+- `src/benchmark.sh` ... Measure processing time
+- `lib/core-completion.sh` ... Prototype definition for completions
+- `lib/core-syntax.sh` ... Prototype definitions for syntax analyzer
+
+Useful features are implemented in separate modules:
+
+- `keymap/vi.sh` ... Vim mode
+  - `lib/vim-arpeggio.sh` ... `vim-arpeggio`-like plugin
+  - `lib/vim-surround.sh` ... `vim-surround`-like plugin
+- `keymap/emacs.sh` ... Emacs mode
+- `lib/core-syntax.sh` ... Shell parser and syntax highlighting
+  - `lib/core-syntax-ctx.def` ... Definition of parser states
+- `lib/core-complete.sh` ... Completions including `menu-complete`, `auto-complete`, `menu-filter`, `dabbrev`, `sabbrev`, etc.
+
+The following files are initialization scripts:
+
+- `lib/init-term.sh` ... Initialize terminal escape sequences (host-to-terminal, i.e. control sequences)
+- `lib/init-cmap.sh` ... Initialize terminal escape sequences (terminal-to-host, i.e. key sequences)
+- `lib/init-bind.sh` ... Initialize readline attacher
+- `lib/init-msys1.sh` ... Workaround for MSYS1 with broken pipes
+  - `lib/init-msys1-helper.c` ... Helper C program for the workaround of broken pipes
+
+## Tests
+
+Tests can be run by one of the following commands:
+
+```bash
+$ make check
+$ make check-all
+$ bash out/ble.sh --test
+
+```
+
+Currently test coverage is very small
+partly because the testing for interactive behavior and terminal rendering results is hard.
+Nevertheless, the tests are defined in the following files:
+
+- `lib/core-test.sh` ... Library for tests
+- `lib/test-main.sh`
+- `lib/test-util.sh`
+- `lib/test-canvas.sh`
+- `lib/test-edit.sh`
+- `lib/test-complete.sh`
+- `lib/test-syntax.sh`

README-ja_JP.md

@@ -15,6 +15,11 @@
 現時点では、文字コードとして `UTF-8` のみの対応です。
 このスクリプトは [**BSD License**](LICENSE.md) (3条項 BSD ライセンス) の下で提供されます。
 
+免責: ラインエディタ本体は **ピュア Bash** で書かれていますが、
+ユーザーコマンド実行時には TTY 設定の為に `stty` (POSIX) を呼び出します。
+他にも処理の高速化の為に、初期化・終了処理、
+巨大なデータの処理 (補完、貼り付けなど) の局面でPOSIX 標準コマンドを利用しています。
+
 ## 簡単設定
 
 `ble.sh` インストールには `git`, `make` (GNU make), and `gawk` が必要です。
@@ -34,10 +39,14 @@ git clone --recursive https://github.com/akinomyoga/ble.sh.git
 make -C ble.sh
 source ble.sh/out/ble.sh
 
-# 更新 (ble.sh をロードした状態で実行して下さい)
+# 更新 (ble.sh をロードした状態で)
 
 ble-update
 
+# 更新 (ble.sh 外部から)
+
+bash /path/to/ble.sh --update
+
 # パッケージ作成用コマンド
 
 git clone --recursive https://github.com/akinomyoga/ble.sh.git
@@ -52,18 +61,34 @@ make -C ble.sh install DESTDIR=/tmp/blesh-package PREFIX=/usr/local
 
 - **構文着色**: `fish` や `zsh-syntax-highlighting` のような文法構造に従った着色を行います。
   `zsh-syntax-highlighting` のような単純な着色ではなく、構文の入れ子構造や複数のヒアドキュメントなども正しく解析して着色します。
-- **補完増強**: 補完を大幅に増強します。
-  文法構造に応じた補完、クォートやパラメータ展開を展開した上でのプログラム補完、**曖昧補完**に対応しています。
-  また、候補をカーソルキーや <kbd>TAB</kbd>, <kbd>S-TAB</kbd> で選択できる**メニュー補完**、
-  `fish` や `zsh-autosuggestions` のような**自動補完** (Bash 4.0 以上) の機能もあります。
-  更に、従来 `peco` や `fzf` を呼び出さなければならなかった補完候補の絞り込みも**メニュー絞り込み** (Bash 4.0 以上) として自然な形で組み込んでいます。
-  他に、**動的略語展開**や、`zsh-abbreviations` のような**静的略語展開**にも対応しています。
+  着色は[全て設定可能](https://github.com/akinomyoga/ble.sh/wiki/%E8%AA%AC%E6%98%8E%E6%9B%B8-%C2%A72-%E6%8F%8F%E7%94%BB)です。
+- **補完増強**: [補完](https://github.com/akinomyoga/ble.sh/wiki/%E8%AA%AC%E6%98%8E%E6%9B%B8-%C2%A77-%E8%A3%9C%E5%AE%8C)を大幅に増強します。
+  **文法構造に応じた補完**、クォートやパラメータ展開を展開した上でのプログラム補完、**曖昧補完**に対応しています。
+  また、候補をカーソルキーや <kbd>TAB</kbd>, <kbd>S-TAB</kbd> で選択できる
+  [**メニュー補完**](https://github.com/akinomyoga/ble.sh/wiki/%E8%AA%AC%E6%98%8E%E6%9B%B8-%C2%A77-%E8%A3%9C%E5%AE%8C#user-content-sec-menu-complete)、
+  `fish` や `zsh-autosuggestions` のような
+  [**自動補完**](https://github.com/akinomyoga/ble.sh/wiki/%E8%AA%AC%E6%98%8E%E6%9B%B8-%C2%A77-%E8%A3%9C%E5%AE%8C#user-content-sec-auto-complete)
+  (Bash 4.0 以上) の機能もあります。
+  更に、従来 `peco` や `fzf` を呼び出さなければならなかった補完候補の絞り込みも
+  [**メニュー絞り込み**](https://github.com/akinomyoga/ble.sh/wiki/%E8%AA%AC%E6%98%8E%E6%9B%B8-%C2%A77-%E8%A3%9C%E5%AE%8C#user-content-sec-menu-filter)
+  (Bash 4.0 以上) として自然な形で組み込んでいます。
+  他に、[**動的略語展開**](https://github.com/akinomyoga/ble.sh/wiki/%E8%AA%AC%E6%98%8E%E6%9B%B8-%C2%A77-%E8%A3%9C%E5%AE%8C#user-content-sec-dabbrev)
+  や、`zsh-abbreviations` のような
+  [**静的略語展開**](https://github.com/akinomyoga/ble.sh/wiki/%E8%AA%AC%E6%98%8E%E6%9B%B8-%C2%A77-%E8%A3%9C%E5%AE%8C#user-content-sec-sabbrev)
+  にも対応しています。
 - **Vim編集モード**: `set -o vi` による編集モードを増強します。
   挿入・ノーマルモードの他に(行・矩形)ビジュアルモード、置換モードなどの各種モードに対応しています。
   テキストオブジェクト・各種レジスタ・オペレータ・キーボードマクロなどにも対応しています。
   拡張として `vim-surround` も提供しています。
-
-★`ble.sh` はよくある Bash 設定集のようにプロンプト (`PS1`)、エイリアス、関数を提供するものではありません。
+- 他にも
+  [**ステータス行**](https://github.com/akinomyoga/ble.sh/wiki/%E8%AA%AC%E6%98%8E%E6%9B%B8-%C2%A74-%E7%B7%A8%E9%9B%86#user-content-bleopt-prompt_status_line),
+  [**コマンド履歴共有**](https://github.com/akinomyoga/ble.sh/wiki/%E8%AA%AC%E6%98%8E%E6%9B%B8-%C2%A74-%E7%B7%A8%E9%9B%86#user-content-bleopt-history_share),
+  [**右プロンプト**](https://github.com/akinomyoga/ble.sh/wiki/%E8%AA%AC%E6%98%8E%E6%9B%B8-%C2%A74-%E7%B7%A8%E9%9B%86#user-content-bleopt-prompt_rps1),
+  [**過渡的プロンプト**](https://github.com/akinomyoga/ble.sh/wiki/%E8%AA%AC%E6%98%8E%E6%9B%B8-%C2%A74-%E7%B7%A8%E9%9B%86#user-content-bleopt-prompt_ps1_transient),
+  [**xterm タイトル**](https://github.com/akinomyoga/ble.sh/wiki/%E8%AA%AC%E6%98%8E%E6%9B%B8-%C2%A74-%E7%B7%A8%E9%9B%86#user-content-bleopt-prompt_xterm_title),
+  など様々な機能に対応しています。
+
+注意: `ble.sh` は、(プロンプト (`PS1`)、エイリアス、関数などを提供する) 典型的な Bash 設定集と異なります。
 `ble.sh` はより低層の基盤を提供するもので、ユーザは自分でプロンプトやエイリアスを設定する必要があります。
 勿論 `bash-it` や `oh-my-bash` の様な他の Bash 設定と一緒に使っていただくことも可能です。
 
@@ -123,9 +148,9 @@ Make 変数 `DESTDIR` または `PREFIX` が指定されている時、`ble.sh`
 現在、安定版は開発版に比べてかなり古いので様々な機能が欠けている事にご注意下さい。
 
 - 開発版 [v0.4.0-devel2](https://github.com/akinomyoga/ble.sh/releases/tag/v0.4.0-devel2) (2020-12)
-- 安定版 [v0.3.3](https://github.com/akinomyoga/ble.sh/releases/tag/v0.3.3) (2019-02 fork)
-- 安定版 [v0.2.6](https://github.com/akinomyoga/ble.sh/releases/tag/v0.2.6) (2018-03 fork)
-- 安定版 [v0.1.14](https://github.com/akinomyoga/ble.sh/releases/tag/v0.1.14) (2015-12 fork)
+- 安定版 [v0.3.3](https://github.com/akinomyoga/ble.sh/releases/tag/v0.3.3) (2019-02 fork) +拡張補完
+- 安定版 [v0.2.6](https://github.com/akinomyoga/ble.sh/releases/tag/v0.2.6) (2018-03 fork) +Vim モード
+- 安定版 [v0.1.14](https://github.com/akinomyoga/ble.sh/releases/tag/v0.1.14) (2015-12 fork) 構文着色
 
 ## 1.2 `ble.sh` をダウンロードして試す (旧バージョン ble-0.3 201902版)<sup><a id="get-from-tarball" href="#get-from-tarball">†</a></sup>
 
@@ -190,6 +215,12 @@ Git (`git'), GNU awk (`gawk`), 及び GNU make (`make`) が必要になります
 $ ble-update
 ```
 
+`ble-0.4` 以上をお使いの場合は `ble.sh` をロードしなくても以下のコマンドで更新可能です。
+
+```bash
+$ bash /path/to/ble.sh --update
+```
+
 それ以外の場合には、以下のように `git pull` で最新版を入手・インストールできます。
 
 ```bash

README.md

@@ -15,9 +15,15 @@ This script supports Bash 3.0 or higher although we recommend to use `ble.sh` wi
 Currently, only `UTF-8` encoding is supported for non-ASCII characters.
 This script is provided under the [**BSD License**](LICENSE.md) (3-clause BSD license).
 
+Disclaimer: The core part of the line editor is written in **pure Bash**, but
+`ble.sh` relies on POSIX `stty` to set up TTY states before and after the execution of user commands.
+It also use other POSIX utilities for acceleration
+in some part of initialization and cleanup code,
+processing of large data in completions, paste of large data, etc.
+
 ## Quick instructions
 
-Installation requires the commands `git`, `make` (GNU make), and `gawk`.
+Installation requires the commands `git`, `make` (GNU make), and `gawk` (in addition to `bash` and POSIX standard utilities).
 For detailed descriptions, see [Sec 1.1](#get-from-source) and [Sec 1.2](#get-from-tarball) for trial/installation,
 [Sec 1.3](#set-up-bashrc) for the setup of your `~/.bashrc`.
 
@@ -38,6 +44,10 @@ source ble.sh/out/ble.sh
 
 ble-update
 
+# UPDATE (outside ble.sh sessions)
+
+bash /path/to/ble.sh --update
+
 # PACKAGE (for package maintainers)
 
 git clone --recursive https://github.com/akinomyoga/ble.sh.git
@@ -51,17 +61,32 @@ You may also install `ble.sh` through package-management systems (currently only
 ## Features
 
 - **Syntax highlighting**: Highlight command lines input by users as in `fish` and `zsh-syntax-highlighting`.
-  Unlike the simple highlighting in `zsh-syntax-highlighting`, `ble.sh` performs syntactic analysis to enable the correct highlighting of complex structures such as nested command substitutions, multiple here documents, etc.
+  Unlike the simple highlighting in `zsh-syntax-highlighting`, `ble.sh` performs syntactic analysis
+  to enable the correct highlighting of complex structures such as nested command substitutions, multiple here documents, etc.
+  Highlighting colors and styles are [fully configurable](https://github.com/akinomyoga/ble.sh/wiki/Manual-%C2%A72-Graphics).
 - **Enhanced completion**:
-  Support syntax-aware completion, completion with quotes and parameter expansions in prefix texts, ambiguous candidate generation, etc.
-  Also **menu-complete** supports selection of candidates in menu (candidate list) by cursor keys, <kbd>TAB</kbd> and <kbd>S-TAB</kbd>.
-  The feature **auto-complete** supports the automatic suggestion of completed texts as in `fish` and `zsh-autosuggestions` (with Bash 4.0+).
-  The feature **menu-filter** integrates automatic filtering of candidates into menu completion (with Bash 4.0+).
-  There are other functionalities such as **dabbrev** and **sabbrev** like `zsh-abbreviations`.
+  Extend [completion](https://github.com/akinomyoga/ble.sh/wiki/Manual-%C2%A77-Completion)
+  by **syntax-aware completion**, completion with quotes and parameter expansions in prefix texts, **ambiguous candidate generation**, etc.
+  Also, [**menu-complete**](https://github.com/akinomyoga/ble.sh/wiki/Manual-%C2%A77-Completion#user-content-sec-menu-complete)
+  supports selection of candidates in menu (candidate list) by cursor keys, <kbd>TAB</kbd> and <kbd>S-TAB</kbd>.
+  The feature [**auto-complete**](https://github.com/akinomyoga/ble.sh/wiki/Manual-%C2%A77-Completion#user-content-sec-auto-complete)
+  supports the automatic suggestion of completed texts as in `fish` and `zsh-autosuggestions` (with Bash 4.0+).
+  The feature [**menu-filter**](https://github.com/akinomyoga/ble.sh/wiki/Manual-%C2%A77-Completion#user-content-sec-menu-filter)
+  integrates automatic filtering of candidates into menu completion (with Bash 4.0+).
+  There are other functionalities such as
+  [**dabbrev**](https://github.com/akinomyoga/ble.sh/wiki/Manual-%C2%A77-Completion#user-content-sec-dabbrev) and
+  [**sabbrev**](https://github.com/akinomyoga/ble.sh/wiki/Manual-%C2%A77-Completion#user-content-sec-sabbrev) like `zsh-abbreviations`.
 - **Vim editing mode**: Enhance `readline`'s vi editing mode available with `set -o vi`.
-  Vim editing mode supports various vim modes such as char/line/block visual/select mode, replace mode, command mode, operator pending mode as well as insert mode and normal mode.
+  Vim editing mode supports various vim modes such as char/line/block visual/select mode, replace mode,
+  command mode, operator pending mode as well as insert mode and normal mode.
   Vim editing mode supports various operators, text objects, registers, keyboard macros, marks, etc.
   It also provides `vim-surround` as an option.
+- Other interesting features include
+  [**status line**](https://github.com/akinomyoga/ble.sh/wiki/Manual-%C2%A74-Editing#user-content-bleopt-prompt_status_line),
+  [**history share**](https://github.com/akinomyoga/ble.sh/wiki/Manual-%C2%A74-Editing#user-content-bleopt-history_share),
+  [**right prompt**](https://github.com/akinomyoga/ble.sh/wiki/Manual-%C2%A74-Editing#user-content-bleopt-prompt_rps1),
+  [**transient prompt**](https://github.com/akinomyoga/ble.sh/wiki/Manual-%C2%A74-Editing#user-content-bleopt-prompt_ps1_transient),
+  [**xterm title**](https://github.com/akinomyoga/ble.sh/wiki/Manual-%C2%A74-Editing#user-content-bleopt-prompt_xterm_title), etc.
 
 Note: ble.sh does not provide a specific settings for the prompt, aliases, functions, etc.
 ble.sh provides a more fundamental infrastructure so that users can set up their own settings for prompts, aliases, etc.
@@ -77,7 +102,7 @@ Of course ble.sh can be used in combination with other Bash configurations such
 
 ### Generate
 
-To generate `ble.sh`, `gawk` (GNU awk) and `gmake` (GNU make) is required.
+To generate `ble.sh`, `gawk` (GNU awk) and `gmake` (GNU make) (in addition to Bash and POSIX standard utilities) is required.
 The file `ble.sh` can be generated using the following commands.
 If you have GNU make installed on `gmake`, please use `gmake` instead of `make`.
 ```bash
@@ -123,9 +148,9 @@ For download, trial and install, see the description at each release page.
 The stable versions are significantly old compared to the devel version, so many features are unavailable.
 
 - Devel [v0.4.0-devel2](https://github.com/akinomyoga/ble.sh/releases/tag/v0.4.0-devel2) (2020-12)
-- Stable [v0.3.3](https://github.com/akinomyoga/ble.sh/releases/tag/v0.3.3) (2019-02 fork)
-- Stable [v0.2.6](https://github.com/akinomyoga/ble.sh/releases/tag/v0.2.6) (2018-03 fork)
-- Stable [v0.1.14](https://github.com/akinomyoga/ble.sh/releases/tag/v0.1.14) (2015-12 fork)
+- Stable [v0.3.3](https://github.com/akinomyoga/ble.sh/releases/tag/v0.3.3) (2019-02 fork) +Enhanced completions
+- Stable [v0.2.6](https://github.com/akinomyoga/ble.sh/releases/tag/v0.2.6) (2018-03 fork) +Vim mode
+- Stable [v0.1.14](https://github.com/akinomyoga/ble.sh/releases/tag/v0.1.14) (2015-12 fork) Syntax highlighting
 
 
 ## 1.3 Set up `.bashrc`<sup><a id="set-up-bashrc" href="#set-up-bashrc">†</a></sup>
@@ -164,12 +189,18 @@ If you want to change the default path of the init script, you can add the optio
 ## 1.5 Update
 
 You need Git (`git`), GNU awk (`gawk`) and GNU make (`make`).
-For `ble-0.3+`, run `ble-update` in the session with `ble.sh` loaded:
+For `ble-0.3+`, you can run `ble-update` in the session with `ble.sh` loaded:
 
 ```bash
 $ ble-update
 ```
 
+For `ble.0.4+`, you can also update it outside the `ble.sh` session using
+
+```bash
+$ bash /path/to/ble.sh --update
+```
+
 You can instead download the latest version by `git pull` and install it:
 
 ```bash