v2.1.0 - architecture-aware PIC hashing and corrected matcher scoring

Most of this release comes from @r0ny123 in #3, with follow-up cleanup in #4.

Basic blocks are hashed for the architecture they are

calculateBlockhash always escaped instructions with IntelInstructionEscaper, whatever the code actually was. SMDA 4 picks an escaper per function — Intel, AArch64, CIL or Dalvik — and picblocks now uses that one, falling back to Intel only when nothing else is available.

This showed up through mcrit, whose MemoryStorage and MongoDbStorage call getBlockhashesForFunction(..., hash_size=8) on every add, so non-Intel functions were being stored under Intel-style escaping.

Measured over SMDA's own fixtures:

Fixture	Architecture	Blocks whose hash changed
cutwail	intel, 32bit	0 of 128
bashlite	intel, 64bit	0 of 768
aarch64_static	aarch64	1736 of 1758

Intel hashes are unchanged, so existing Intel databases and block reports stay valid. AArch64 was wrong on 98.7% of blocks and is now correct: every block in all three fixtures hashes identically to SMDA's own SmdaBasicBlock.getPicBlockHash().

The SHA-256 prefix is unpacked little-endian (<I / <Q), which matches SMDA and makes the integer the same on a big-endian host.

Matcher percentages were too low

match() credited each family once per block hash, while block_bytes — the denominator behind every percentage — counts every block occurrence. A block shared by ten functions contributed its size ten times to the denominator and once to the numerator, so all four percentages read low. Families are now credited once per matching function.

Over a database built from the Malpedia block reports, matching a sample that is itself in that database moved from 87.6–99.0% to 98.2–99.4%.

Match reports from earlier versions are not comparable with new ones. If you keep stored match results, re-run them.

One known gap remains: a block report stores the function ids a hash was seen in as a set, so the same hash occurring twice inside one function counts twice in block_bytes but once when scoring. That is ~2.4% of block_bytes over the Malpedia reports, and it is why a self-match lands just under 100% rather than exactly on it. Closing it would change the block report format.

unmatched_hashes was never written, and a hash hit with a size miss bumped unmatched_score without unmatched_blocks. Both are fixed.

Requirements

• smda >= 4.2.13 (was pinned to 1.12.7), matching mcrit's floor
• Python >= 3.11

smda 4.4.1 and later need 3.11; 4.2.13 through 4.3.10 still allowed 3.10, so an unstated floor let pip quietly resolve an older smda on 3.10 rather than report the Python as too old.

Other fixes

• Memory dumps are recognised by Malpedia-style names (dump_0x..., dump7_0x...) on the basename. processFile matched the substring "dump" anywhere in the path, so /data/dumps/malware.exe was disassembled as a mapped buffer.
• baseaddress=0 is a valid mapped base and is no longer discarded as falsy.
• Bitness is left to SMDA where the file name does not settle it. Reading it off the width of a base address called every module mapped below 4GB 32bit, and 64bit modules are mapped there routinely.
• SMDA error reports (status="error", xcfg is None) no longer crash hashing.
• num_functions_hashed counted blocks rather than functions.
• hash_malpedia.py: a leftover file_index % 10 sampling skipped 9 of every 10 files, e_lfanew was read as a WORD instead of a DWORD, Pool(cpu_count() - 2) died on 1–2 CPUs, and Python 3.10+ runtime DLLs were not recognised.
• Web UI: the result page never passed block_b through, the default DB timestamp was %Y-%d-%d, a hidden bitness radio outvoted the 32bit control, family links are HTML-escaped, /stats falls back to db/stats.json when Mongo is offline, and /blocks and /api/blocks validate their payloads.
• Packaging is single-sourced in pyproject.toml; setup.py is gone. The wheel contains picblocks/ and the licence, where find_packages(exclude=('tests')) had been passing a string rather than a tuple and excluding nothing.

Tests and CI

61 tests covering the hasher, matcher and demo app, plus parity tests that disassemble a real buffer and check picblocks against SMDA's own PIC hash — the fakes the rest of the suite uses cannot catch an API drift on their own. CI runs lint and types (ruff, ty), the unit tests, and a job that stands up Mongo and a live mcrit server/worker and pushes SMDA's cutwail, aarch64_static and bashlite fixtures through the same call path mcrit uses.