bitarray 1.0: packed integer arrays for Tcl 9 (C extension)

bitarray 1.0: packed integer arrays that behave like Tcl lists (Tcl 9.1, or Tcl 9.0.4 and later with one limitation)

What 1.0 has (the SSE2 loops, the faster logical and relational operators, minmax and the Tcl 9.0 fallback are the newest parts):

• Element widths 1, 8, 16, 32 and 64 bits, signed or unsigned (-width 8, -8, 16, -16, ...), not only 1-bit flags.
• Whole-array commands that run in C: bitarray op (and, or, xor, andnot, not, land, lor, lnot, eq, ne, lt, le, gt, ge, between arrays or an array and an integer, in place or into a new array), count, next, minmax, fill, in/ni, and conversions to and from bytes and characters.
• Speed: on 1,000,000 elements the whole-array commands are hundreds to thousands of times faster than the same work on a plain list (bit operations on 1-bit arrays about 10,000x; 8-bit land/lor/lnot and the relational operators 300x to 800x; minmax 400x to 1,400x for 8 to 32 bits and about 50x for 64 bits; measured on Windows with MSVC /O2), using word-at-a-time loops and, on x86-64, SSE2 (on by default; build.sh CFLAGS=-DBA_NO_SSE2 or build.bat --no-sse2 turns it off). The numbers are from extras/listbench.tcl, which anyone can run. Single-element commands (lindex, lset, lappend, lrange, ...) cost the same as on a list. This is new code for new programs, not a drop-in speed-up of an existing list.
• Tcl 9.0.4 and later are accepted (the extension loads on 9.0). One limitation, a Tcl core problem: on 9.0, a plain lappend (and linsert of exactly one element at the end, where the command is not compiled) can turn a bitarray into an ordinary list; the values stay right but the packed type and its speed are lost. 9.1 narrows this to lappend inside a proc; the core ticket is 0c82728f85760f36361fe42a15f55453c94d285d and a core maintainer is working on it. README section "Tcl 9.0" has the details and workarounds. The tests print the Tcl version, with a warning box on anything older than 9.1.
• 434 tests (428 passed, 6 skipped on 9.1.1; on 9.0.4 417 passed, 17 skipped, 0 failed), valgrind of the whole suite with no invalid reads or writes, 98.7% line coverage, and a full mutation run of 407 deliberate bugs (the two gaps it found in the tests were closed; the rest are explained in RESULTS.txt).

Prebuilt libraries (optional): bitarray-1.0-linux-x86_64.zip (x86-64 Linux) and bitarray-1.0-windows.zip (Windows x64, Visual Studio 2022). Each zip holds the library, pkgIndex.tcl and a short readme: unzip it, add its parent folder to auto_path, and run package require bitarray. They work with Tcl 9.1 and with Tcl 9.0.4 or later. They are built from bitarray.c in this repository; building it yourself (build.sh or build.bat, see README.md) is the safer way. The Windows library needs the Microsoft Visual C++ 2015-2022 x64 runtime, which most machines already have.

SHA-256 of the Linux library, libbitarray.so inside the zip: cc76cbfe85d98446ff3369c7b72c1a29ca9512b516fdaf2f8d4d45555c7de7ef
SHA-256 of the Windows library, bitarray.dll inside the zip: dda0b794960c82bb564f2a763147ed7f19936783992ad0651ac03e073856d972

Tested on Linux (gcc) and Windows (Visual Studio 2022, Tcl 9.1.1 and 9.0.2). A real 32-bit build has not been tested.

Build and test: see README.md (build.sh / build.bat, then run_tests.sh / run_tests.bat). Expected last line of the tests on 9.1: Total 434 Passed 428 Skipped 6 Failed 0 (Windows: Total 434 Passed 427 Skipped 7 Failed 0).

The code, tests and documentation were written by Claude (an AI model made by Anthropic) working with Eric Taylor. See README.md, "Reviewing it, with or without an AI assistant".

bitarray 0.8

bitarray 0.8: packed integer arrays that behave like Tcl lists (Tcl 9.1 or later)

New since the 0.7 release (the compiled code is the same as in 0.7; this release is the stepper, the comments and the tests):

• The stepper page (extras/bitarray_stepper.html, also at https://rocketship88.github.io/tcl-bit-array-extension/extras/bitarray_stepper.html) is a walk-through of an abstract list: pick a command, step through what the Tcl core and bitarray.c do, see the call stack, the Tcl_ObjType hook reached, and the bits. "Show code" shows the real bitarray.c with syntax colour; double-click a name to jump to its code, or to its definition in tcl.h / the Tcl declarations (embedded, one level deep). A "Fold" box collapses each switch case and if / else branch for an overview.
• bitarray.c has 220 short side comments on the lines that are not obvious. The code did not change.
• Six tests (run_tests.sh --testbug) that show a bug in the Tcl core: lappend inside a compiled proc turns an abstract list into a plain list (Tcl ticket 0c82728f85760f36361fe42a15f55453c94d285d). They are skipped by default, fail on Tcl 9.1.1 and pass on a core with the patch in extras/core_lappend_typeloss.patch.
• 288 tests (282 run, 6 skipped); a full mutation run of 206 deliberate bugs: 184 caught, 22 explained in RESULTS.txt.
• The repository has a CLAUDE.md: unzip it, drag the folder or the zip into Claude, and Claude can build it, run the tests and explain the code.

Tested on Linux and on Windows (Visual Studio 2022, Tcl 9.1.1). A real 32-bit build has not been tested.

Build and test: see README.md (build.sh / build.bat, then run_tests.sh / run_tests.bat). Expected last line of the tests: Total 288 Passed 282 Skipped 6 (Windows: Passed 281 Skipped 7).

bitarray 0.7

bitarray 0.7: packed integer arrays that behave like Tcl lists (Tcl 9.1 or later)

New since the 0.6 release:

• Signed 8, 16 and 32 bit elements: -width -8, -16, -32 (a minus sign means signed). -64 is accepted as another name for 64, which was already signed. bitarray width returns the signed form, the bitwise commands need the same width and sign, and tobytes, tochars and fromchars refuse signed arrays.
• in on a 1-bit array now tests whole 64-bit words and stops at the first word that decides the answer, instead of counting every bit.
• Four new in rows in the benchmark (extras/listbench.tcl), and a fix for a warm-lsearch row that read 0.0 on Windows.
• 282 tests (259 in 0.6, plus 21 for the signed widths and 2 for in).
• Mutation testing: a full run of 206 deliberate bugs, 184 caught; the 22 that were not caught are explained in RESULTS.txt.
• performance_analysis.md has the new in figures; the simulator (extras/bitarray_stepper.html) has the new in text, shows the isSigned header field, and its Show code button now scrolls to the code.
• CLAUDE.md, README.md and extras/CLAUDE_NOTES.md now tell an AI assistant to show the simulator and what to check (Code execution and file creation, Allow network egress) if a download or command is refused.

Tested on Linux and on Windows (Visual Studio 2022, Tcl 9.1.1): 282 tests (281 passed and 1 skipped on Windows), the sieve passes, and the benchmark's answer check agrees on 42 operations. A real 32-bit build has not been tested.

Build and test: see README.md (build.sh / build.bat, then run_tests.sh / run_tests.bat). Expected last line of the tests: Total 282 Passed 282 (Windows: Passed 281 Skipped 1).

Simulator (no download needed): https://rocketship88.github.io/tcl-bit-array-extension/extras/bitarray_stepper.html

Updates for bitwise logicals

Full Changelog: v0.5...v0.6

bitarray 0.5

bitarray 0.5, packed arrays of small integers for Tcl 9.1 that behave like ordinary Tcl lists (widths 1, 8, 16, 32 and 64 bits). It is an extension only, with no change to Tcl itself.

11 commands: new, newfrom, width, is, count, next, fill, and frombytes, tobytes, fromchars, tochars for files and text.
fill, count and next accept optional lseq-style helper words (to, .., with, by).
package require bitarray works once the folder with pkgIndex.tcl and the built library is on auto_path.
222 tests (standard tcltest), clean under valgrind. Mutation testing: 109 of 123 deliberate bugs caught.
Tried on Linux and on two Windows 10 machines with Tcl 9.1.1 (x64). Not tested: 32-bit builds, other Visual Studio versions, macOS, big-endian, threads, Tcl 9.0.x.
Prime-sieve benchmark included, with Frank van Bakel’s sieve used with his permission.

Build with sh ./build.sh (Linux) or build.bat (Windows); see the README.

This was written by Eric Taylor together with Claude (Anthropic’s AI model), which wrote most of the code and tests under his direction.