Repository navigation
Releases: Cynosureprime/Cpcfg
Releases · Cynosureprime/Cpcfg
Release list
v1.20
Train on the bytes the hash covered, and learn more than one keyboard utf8_decode never checked continuation bytes, so a lead-byte pattern alone chose a multi-byte branch and swallowed the characters after it. CP1252 k-oumlaut-ln1964 has 0xF6 for the o-umlaut, (0xF6 & 0xF8) == 0xF0 took the four-byte branch, and 'l', 'n' and '1' were consumed as continuations. 56 of the 128 high bytes take a multi-byte branch that way, which is the whole accented Latin-1 range, so every German, Spanish, French, Portuguese, Italian and Nordic word in a single-byte corpus was destroyed rather than misread. That matters because the corpora are split: pass.len256 is 24 G and 98.8 percent single-byte, and a trainer that mangles it cannot reach any hash built from those bytes. Re-encoding then had to stop widening those bytes. A single-byte terminal written back as two UTF-8 bytes is a different password as far as the digest is concerned. utf8_encode_width preserves the width a codepoint arrived in, through training, the multiword trie and generation. CP1252 in now gives CP1252 out, byte for byte. Latin Extended-A was treated as one even-is-uppercase run. It is three: the run holds to U+0137, inverts for U+0139-U+0148, and resumes at U+014A. 18 of 120 codepoints were mapped wrongly, and in the inverted run the error turned a lowercase letter into a different letter, Polish l-stroke to N-acute. utf8_is_upper carried the same wrong parity, so the two faults cancelled and neither was visible alone; fixing either by itself corrupts Polish, Slovak, Czech and Latvian. The year pass tested one byte and wrote four, overwriting whatever an earlier pass owned and leaving keyboard terminals shorter than the four keys detection requires. Adjacent keyboard walks merged into one rare terminal instead of the several common ones actually typed. Keyboard tables were byte-indexed at 128 entries, so every key that is not ASCII was dropped: n-tilde, c-cedilla, a-ring, o-diaeresis, sterling, eszett and the whole Cyrillic alphabet. JCUKEN held one row, the digits, and could not match a single Cyrillic walk. Keys are now codepoints across Latin-1, Latin Extended-A and Cyrillic, with UK, Spanish, Portuguese, Nordic, Turkish-Q and Brazilian ABNT2 added. The two-type filter classified by byte, so a Russian home-row run scored two types and its English equivalent one, making the same string a walk or not according to its alphabet; it also refused symbol-only runs, which is what a shifted number row is. Classification is now by codepoint, symbol-only runs are accepted at the four-key minimum, and a single-type run needs six keys and a non-US key among them. That last condition is measured: without it, 60,000 dictionary words yielded 20 false terminals including desert, which really is six adjacent QWERTY keys. With it, zero. Three of the six were reported by matrix (github.com/matrix) in the hpcfg README: the Latin Extended-A runs, the year overwrite and the walk merge. The other three were found while confirming them. ASCII training is unchanged: the same 101 structures and a byte-identical 50,000-candidate stream on a 50k corpus.
v1.05
v1.05: 1.5x faster generation, fix status line and segfault Generation: - Eliminate threading overhead (single-threaded is faster for PCFG) - Cache GrammarEntryList lookups (avoid repeated Judy string lookups) - Stack-allocated scratch buffer in find_children (eliminate malloc/free per PT) - Remove atomic increment on guess_count - 100M guesses: 17.3s vs 43s before (2.5x improvement) Bug fixes: - Fix status line showing 0 guesses when no -n limit set - Fix segfault in single-threaded mode (write_lock NULL)