v1.1.0

What's Changed

• Forensic coverage by @akhil-dara in #1

New Contributors

• @akhil-dara made their first contribution in #1

Full Changelog: v1.0.2...v1.1.0

v1.0.2 — history that keeps going, exports that change nothing but the encryption

History recovery that keeps going, and exports that change nothing but the encryption.

History (--history)

• Fixed the stall. WhatsApp only delivers a linked device's offline queue in batches the client asks for. The tool asked for one batch of 100, so the phone's answers queued behind the rest and seemed to never arrive. It now drains the whole queue first, as WhatsApp Web does; the phone typically answers within seconds.
• Each history chunk is released on the server after it is saved, as WhatsApp Web does.
• Progress per chat is saved, so a run continues where the last one stopped.
• Late answers are counted, and the run says plainly when the phone has reached a chat's start.
• Recovered messages merge into the same chats as the browser cache: one timeline per chat, with a source field saying where each row came from. Chats no longer split between a phone-number form and a LID form.
• Text recovered from the phone is kept exactly as sent. Message types inside wrapper messages, and button/list replies, are read correctly.

Exports

• No message is truncated. Text was cut at 2,000 characters in JSON and the transcript, and at 8,000 in SQLite.
• Text the old string-scraper dropped is back: numbers someone sent, pasted hashes, *bold* text, and newer emoji.
• Names are exported exactly as saved, with no trimming.
• SQLite: deleted messages are type 15 and system messages type 7, as on Android. Quoted text goes in the column Android parsers read. Every message type gets its text.

Raw output and chain of custody

• raw manifest: a SHA-256 for every file, and empty stores are listed instead of omitted. JavaScript Dates come out as epoch ms + UTC. RegExp flags are kept.
• The audit database hashes every evidence file it reads, and records the exact build (1.0.2, or release + git commit when run from source).
• --log-file is UTF-8. On Windows, lines holding a non-Latin name or an arrow were being dropped from the log file.

Notes

• --history needs Node.js and this repository (npm install in tools/reconnect). It is not bundled in the .exe; see the README.
• Authorized use only: your own account, or a lawful investigation with permission.

v1.0.1 — history recovery that works, raw output, and a lot of correctness work

51 commits since v1.0.0. The headline is that on-demand history recovery actually works — and
most of the rest of this release is the correctness work that got it there.

History recovery now works

Verified end to end against a live account: one request, one chat, 50 messages in 3.0s, all older
than the anchor.

The blocker was that chatJid was being sent in the wrong server form. WhatsApp Web sends
createWid(chatRecord.historyChatId).toJid(), and toJid() maps server c.us to s.whatsapp.net;
we were sending the stored @c.us value, so 127 of 276 chats were asked for by a jid the client
never sends and could never be answered. Confirmed independently against the client's own
history_initial_sync_boundry, which is keyed by the same expression: the wire form matched 132/132
and the raw form 0/132.

Alongside that, the request now matches the reference client field for field — supportInlineResponse
added, accountLid removed (the real client never sets it here), onDemandMsgCount confirmed at 50 —
and peer messages are routed to the LID jid's device 0, which is what
getMyPrimaryForOutgoingPeerMessage() resolves to.

Rate limiting is respected rather than fought. WhatsApp's limit counts failures, not requests —
its counter resets on every success — so the delay now doubles on each unanswered request (800 ms
base, 30 s cap) and decays back on each answered one. When the breaker trips, the tool waits out the
7200-second cooldown and resumes from its checkpoint instead of abandoning the sweep. Expect tens of
minutes for a first pass over a few hundred chats, and hours with two-hour pauses to drain a long
history. Progress is checkpointed after every page.

30 chats are now skipped up front: each chat's endOfHistoryTransferType says whether the primary
still holds anything older, and asking the ones that don't cannot succeed.

New output format: --out raw:DIR

Every other format reshapes the data. The text writer runs the decrypted protobuf through a heuristic
string scraper that discards field identity, every non-string field, duplicates and anything past
2000 characters.

raw decrypts and changes nothing else. One JSONL file per object store — 48 stores, 70,476
records on a typical profile — each record exactly as the browser stored it: same field names, same
ordering, <Undefined> markers preserved. The decryption is added beside the ciphertext rather than
replacing it:

• _wawf_plaintext_b64 — the decrypted bytes verbatim, so the decryption can be re-verified
• _wawf_plaintext_fields — the same bytes as a schema-free protobuf field tree, so you can see
  which field a string actually came from

signal-storage and wawc_db_enc are excluded by default: they hold the account's Signal sessions,
prekeys and the at-rest root key, and a dump meant to travel with a case should not be key material.
--raw-include-keys overrides it.

~750 more messages now have text

The text extractor required at least one alphabetic character, so any message without a letter was
silently dropped — emoji-only replies, numbers, dates, prices, @mention-only messages. The row was
still written with its key, timestamp, chat and sender all correct; only the text was missing, which
is the worst shape for a forensic tool because nothing looks wrong.

Measured on a 22,121-message store: 767 messages lost their text to that one condition, 745 of
them emoji. After the fix, msgstore.db goes from 18,997 to 19,747 rows with text and chats.json
from 19,221 to 19,976. A bare run of 12 or more digits is still rejected, since that is a phone number
or jid user part rather than anything typed — which removes all 3 such strings while losing 0 real
messages.

BREAKING: --enc-static B64 is now --key FILE

The at-rest key is no longer accepted as a command-line value, and is no longer printed. An inline
secret lands in shell history, in process listings and in anything auditing commands, and the key was
also being written into the log and passed to helper subprocesses in their argv, where any process can
read it. It now travels by file and by environment variable, and wawf history finds a previously
captured companion_enc_static.txt beside the output automatically, so in normal use no flag is
needed at all.

wa_reconnect.py no longer prints the key to stdout either; --print-key restores that for anyone
who was piping it.

Stale credentials are detected instead of silently failing

When WhatsApp Web re-pairs, the account gets a new device slot, a new registrationId and a new Signal
identity — and the exported credentials then describe a device that no longer exists. A revoked
companion still completes the Noise handshake, so the old behaviour was to connect happily, ask every
chat, receive nothing, and report a mystery. Measured on one profile: stored regId 10115 / device :43
against a live profile on regId 11966 / device :44.

scripts/check_identity.py now compares the exports against each other and against the live profile,
wawf history verifies its own exports between steps, and an auto-detected at-rest key that turns out
to be stale is discarded and recaptured rather than trusted. --out raw: validates the key against 40
reference bodies before writing, so a wrong key produces a clear refusal instead of a complete-looking
dump with nothing decrypted in it.

New diagnostics

• diag_onehistory.js — ONE request for ONE chat with every inbound stanza logged, and it separates
  causes that used to look identical: no ack means the stanza was not accepted, an explicit error
  means refused with a readable reason, an ack then silence means throttled or declined by the phone,
  and an ack then a notification means served.
• diag_abprops.js — asks the server which features are enabled for this client, via the
  experiment-config iq WhatsApp Web sends during its loading screen and we never did. That turns
  "is on-demand history even enabled for us?" from a guess into a value you can read.
• diag_health.js — a 60-second probe reporting CONNECT, REGISTERED and SERVED separately. Only a
  REGISTERED failure means the device is gone; the registration check is a USync device query against
  the account, which is definitive rather than inferred from silence.

Also

• The client version announced in the ClientPayload now comes from the profile's own WAVersionBase
  rather than a third-party lookup, since WhatsApp assigns feature flags per client version.
• History chunks are fetched from the signed CDN url when the notification carries one, with the
  header set the real client sends. The previous unsigned reconstruction worked only briefly.
• A chunk that arrives after its request timed out is downloaded and banked rather than discarded.
• Media recovery works across Chromium and Firefox cache layouts (4/4 verified byte-identical).
• Chain-of-custody auditing with --audit: every file read, key derivation, artifact hash and error.

Verification

The published binaries are built in CI from wawf.spec on a clean runner, and the workflow
smoke-tests each one (./<asset> --help) before publishing.

The functional verification below was run against a local build of this same commit, from a fresh
folder as a user would, with all four formats and --audit:

at-rest key VALIDATED against 40/40 reference bodies
raw:     48 stores, 70,476 records, 22,121 of 22,121 encrypted bodies decrypted
sqlite:  jids=3090 chats=292 messages=22121 mentions=676 reactions=454 polls=17/votes=86
json/txt: 22,121 bodies decrypted, 0 failed

Every decrypted plaintext was then compared against an independent reference implementation written
from first principles — its own HKDF, its own protobuf reader, none of this tool's code: 22,084
plaintexts, 0 hash mismatches, 0 on either side only. The at-rest key appears in none of the output
files.

Note for Windows users

The binary is unsigned. On a machine with Smart App Control enabled, Windows may refuse to run it
with "An Application Control policy has blocked this file". The block is reputation-based rather
than signature-based, so it affects a binary Windows has not seen before and can clear by itself after
a while. Smart App Control cannot be re-enabled once turned off, so that is a one-way decision
about the whole machine and not a workaround to reach for. Running from source
(pip install -r requirements.txt && python wawf.py ...) avoids the question entirely, and is
arguably better provenance for forensic work. See BUILD.md.

Own account or authorised investigation only. --history is networked and contacts the account.

---

Assets — built in CI from this tag, each smoke-tested by the workflow before publishing.

asset	SHA-256
wawf-windows-x64.exe	d97c04fec698fd8a46d2094b181e502739225d26fc99343be7f7f495807432fb
wawf-linux-x64	ed385a8a6f2100438bf23c3a959697ab802ff2bfc3ef5843e5628b815797e7a5

Verify before use:

# Windows (PowerShell)
Get-FileHash .\wawf-windows-x64.exe -Algorithm SHA256

# Linux / macOS
sha256sum wawf-linux-x64

v1.0.0

Full Changelog: https://github.com/Mihir-Choudhary/Whatsapp-Web-Extractor/commits/v1.0.0