Support running btests on Windows

.github/workflows/btest.yml

@@ -9,23 +9,67 @@ jobs:
   Run-BTest:
     strategy:
       matrix:
-        # Even though python-3.5 is EOL it is still the minimum Python version
-        # required by Zeek so we should validate that our code works with it.
-        #
-        # Since that Python version is not available on ubuntu-22.04 (currently
-        # ubuntu-latest), we run the Linux job on ubuntu-20.04 which still has
-        # it.
-        python-version: [3, 3.5]
-        os: [macos-latest, ubuntu-20.04]
+        python-version: ["3.7", "3.8", "3.9", "3.10", "3.11"]
+        os: [macos-latest, ubuntu-latest, windows-latest]
 
     runs-on: ${{ matrix.os }}
     steps:
-      - uses: actions/checkout@v2
+      - name: Set up git config
+        # Set autocrlf mode to false so that actions/checkout doesn't
+        # modify the line endings in all of the files (mostly in the
+        # test Baselines) to be \r\n on Windows.
+        run: |
+            git config --global core.autocrlf false
+            git config --global core.eol lf
+      - uses: actions/checkout@v3
       - name: Set up Python ${{ matrix.python-version }}
-        uses: actions/setup-python@v2
+        uses: actions/setup-python@v4
         with:
             python-version: ${{ matrix.python-version }}
-      - name: Install dependencies
+      - name: Install dependencies (Windows)
+        if: matrix.os == 'windows-latest'
+        run: |
+            python -m pip install sphinx multiprocess
+      - name: Install dependencies (Linux/macOS)
+        if: matrix.os != 'windows-latest'
         run: |
             python -m pip install sphinx
+      - name: Rename wsl bash
+        # Something about github's runners sometimes puts WSL bash in the
+        # path before git bash. WSL bash has problems with permissions
+        # when it comes to writing files to the runner's disk, which
+        # causes tests to fail. This step renames WSL bash to something
+        # else so that git bash will execute instead.
+        if: matrix.os == 'windows-latest'
+        run: |
+            takeown /F C:\Windows\System32\bash.exe
+            icacls C:\Windows\System32\bash.exe /grant administrators:F
+            ren C:\Windows\System32\bash.exe wsl-bash.exe
       - run: make test
+
+  Test-SetupPY:
+    strategy:
+      matrix:
+        python-version: ["3.7", "3.8", "3.9", "3.10", "3.11"]
+        os: [ubuntu-latest]
+
+    runs-on: ${{ matrix.os }}
+    steps:
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v4
+        with:
+            python-version: ${{ matrix.python-version }}
+      - uses: actions/checkout@v3
+      - name: "Install pip and the btest package"
+        run: |
+            python3 -m pip install --upgrade pip
+            python3 -m pip install setuptools wheel
+            python3 -m pip install .
+      - name: "Run btests with installed version"
+        run: |
+            cd testing
+            which btest
+            make
+      - name: "Test building package"
+        run: |
+            python3 setup.py sdist bdist_wheel

.gitignore

@@ -4,3 +4,5 @@
 build
 dist
 MANIFEST
+.idea
+sphinx

README

@@ -27,20 +27,35 @@ Prerequisites
 
 BTest has the following prerequisites:
 
-- Python version >= 3.5 (older versions may work, but are not
+- Python version >= 3.7 (older versions may work, but are not
   well-tested).
 
-- Bash (note that on FreeBSD and Alpine Linux, bash is not installed
-  by default).
+- Bash. Note that on FreeBSD and Alpine Linux, bash is not installed by
+  default. This is also required on Windows, in the form of Git's msys2, Cygwin,
+  etc.
 
 BTest has the following optional prerequisites to enable additional
 functionality:
 
-- Sphinx.
+- Sphinx. Sphinx functionality is currently disabled on Windows.
 
 - perf (Linux only).  Note that on Debian/Ubuntu, you also need to install
   the "linux-tools" package.
 
+Windows Caveats
+---------------
+
+When running BTest on Windows, you must have a bash shell installed of some
+sort. This can be from WSL, Cygwin, msys2, Git, or any number of other methods,
+but ``bash.exe`` must be available. BTest will check for its existence at
+startup and exit if it is not available.
+
+A minor change must be made to any configuration value that is a path list. For
+example, if you are setting the ``PATH`` environment variable from your
+btest.cfg. In these cases, you should use ``$(pathsep)s`` in the configuration
+instead of bare ``:`` or ``;`` values to separate the paths. This ensures that
+both POSIX and Windows systems handle the path lists correctly.
+
 Download and Installation
 =========================
 
@@ -330,6 +345,13 @@ and 1 otherwise. Exit code 1 can also result in case of other errors.
     Markdown. In the output each test includes the documentation
     string that's defined for it through ``@TEST-DOC``.
 
+-s <kv>, --set=<kv>
+    Takes a ``key=value`` argument and uses it to override a value
+    used during parsing of the configuration file read by btest at
+    startup. This can be used to override various default values
+    prior to parsing. Can be passed multiple times to override
+    different keys. See `defaults`_ for an example.
+
 -t, --tmp-keep
     Does not delete any temporary files created for running the
     tests (including their outputs). By default, the temporary
@@ -375,8 +397,8 @@ and 1 otherwise. Exit code 1 can also result in case of other errors.
     If the file exists already, it is overwritten.
 
 -z RETRIES, --retries=RETRIES
-     Retry any failed tests up to this many times to determine if
-     they are unstable.
+    Retry any failed tests up to this many times to determine if
+    they are unstable.
 
 .. _configuration file: configuration_
 .. _configuration:
@@ -410,6 +432,30 @@ include the output of external commands (e.g., xyz=`\echo test\`).
 Note that the backtick expansion is performed after any ``%(..)``
 have already been replaced (including within the backticks).
 
+.. _default: `defaults`_
+.. _defaults:
+
+Defaults
+~~~~~~~~
+
+There is a special section that can be added to the configuration file that will
+set default values to be used during the parsing of other configuration
+directives. For example::
+
+    [DEFAULT]
+    val=abcd
+
+    [environment]
+    ENV_VALUE=%(val)s
+
+The configuration parser reads the keys and values from the DEFAULT section
+prior to reading the other sections. It uses those keys to replace the ``%()s``
+macros as described earlier. The values stored in these keys can be overridden
+at runtime by using the ``-s``/``--set`` command-line argument. For example to
+override the ``val`` default above, the ``-s val=other`` argument can be
+passed. In that case, ``ENV_VALUE`` would be set to ``other`` instead of
+``abcd``.
+
 .. _option: `options`_
 .. _options: