Repository navigation
v2026.09
Highlights
This is a big and long-overdue Unfurl release. It has new parsers for Gmail, Safe Links, Facebook, Instagram, GitHub, and more; a YAML-based "site definitions" system; Tree and Text views (and dark mode) in the web UI; and a reworked configuration system.
⚠️ Upgrade notes
- API key environment variables have new names. Use
UNFURL_BITLY_API_KEY,UNFURL_VIRUSTOTAL_API_KEY, andUNFURL_GOOGLE_KG_API_KEY. The old bare lowercase names (bitly,virustotal,google_kg) still work but log a deprecation warning. - Put your own settings in
unfurl.local.ini. It overridesunfurl.ini, is gitignored, and is kept out of the Docker image and the sdist. Config files are now found next to the installed package as well as in the current directory (previously only./unfurl.iniwas read). UNFURL_REMOTE_LOOKUPSturns remote lookups on or off from the environment and overrides both config files. Remote lookups remain off by default, and the CLI's-lstill wins over everything.- New dependency:
pyyaml(pip installs it automatically).
🚀 New parsers
- Gmail: legacy hex message/thread IDs and new-style (2018+) URL tokens, decoded to Gmail IDs and their embedded timestamps. Also handles
th,compose,permmsgid/permthid, andprojector. (#268, closes #45) - Microsoft Defender Safe Links: the wrapped destination URL, plus the pipe-delimited
datafield (format versions 01 through 05): recipient, message and tenant GUIDs, timestamp, scan verdict, and scan payload. (#260, closes #153) - Facebook: URL paths (posts, photos, videos, groups, events, stories, pages, Marketplace, profiles, comments) and decoding of modern
fbclidvalues (browser ID, app ID, ad ID, AEM hash). (#252) - Instagram: posts, reels, stories, highlights, and profiles. Shortcodes and story IDs are decoded to a creation timestamp, shard ID, and sequence number. (#250)
- GitHub: repos, commits, issues, PRs, branches, releases, Actions runs, compare views, OAuth flows, line-number fragments, and more. (#249)
- Fernet tokens: version, the cleartext creation timestamp, IV, ciphertext, and HMAC. Includes the re-padded form used by ChatGPT Ads click IDs (
oppref,olref). (#279) - itsdangerous signed tokens (used by Flask and many other Python web apps): payload (decompressed if needed), signing timestamp, and signature. (#263)
- MongoDB ObjectIDs and their embedded timestamps. (#255, closes #94)
- Base32 and Base58. (#262, closes #208)
- Defanged indicators: inputs like
hxxps://evil[.]com/payload[.]exe,1.2.3[.]4, anduser[at]evil.comare refanged into a new node whose hover lists every replacement made. The original input node is left untouched. (#283) - File extensions: path segments that look like file names get a file type, with notes on extensions commonly abused to deliver malware. (#270)
✨ Improvements
Site definitions
- New YAML-based site definitions teach Unfurl a site's URL structure without writing Python. See
unfurl/parsers/site_defs/README.mdto write your own. (#249, #252) - Google's
/urland/imgreshandling moved togoogle.yaml, and/aclkad click URLs (adurl,ai,gclid,sig, and more) are now parsed. (#258, #259)
URL parsing
- Text Fragments (
#:~:text=) and SPA-style fragment routes (#/route?key=value) are parsed. (#257, closes #140) - Query parameters with blank values (
?a=&b=1) are kept instead of dropped. (#256, closes #21) - Single-segment paths are now split into a path segment like longer paths are, so site parsers fire on URLs like
infosec.exchange/@user. Empty segments (//) are flagged. (#264, closes #220) - Runs of
;-delimitedkey=valuepairs, as used by ad servers and in path parameters, are split into individual pairs. (#274) +is only decoded as a space when the value came from a query string, so values that use a literal+(like Gmail compose tokens) survive intact. (#273)- Input cleanup steps (whitespace removal, quote removal, refanging) now chain into one branch instead of producing sibling partial results. (#283)
IDs and timestamps
- UUID versions 6, 7, and 8, with timestamps extracted from v6 and v7. (#254, closes #182, #183, #184)
- New timestamp formats: WebKit milliseconds (#253); Epoch nanoseconds, PostgreSQL, and Mac Absolute Time nanoseconds (#265).
- Epoch seconds auto-detection now covers 2010 onward (previously 2015). (#255)
- Discord snowflakes created after July 2022 are now decoded in general Discord URL paths, and snowflakes in query values and token payloads on Discord domains are decoded too. (#263)
Search engines
- Bing:
formcodes (including indexed variants) and image search parameters (thid,mediaurl,sbifnm,iss,view, and more). (#276)
Short links and remote lookups
- 40 more shortener domains (such as
aka.ms,flic.kr,redd.it,pin.it,t.cn, andwa.link) plus Constant Contact click-tracking links. (#271) - The MISP shortener list is now checked before the "short domain" guess (it was unreachable for domains under eight characters), and that guess no longer fires for popular sites like
x.comorcnn.com, so Unfurl doesn't send them requests. (#271) - LinkedIn short link expansion handles sign-in walls, rate limits, and page changes without tracebacks. (#269)
- MISP warninglists are parsed once per process instead of on every web request. (#271)
Web UI
- New Tree and Text views alongside the Graph, with copy-to-clipboard for the text tree. (#261)
- Dark mode, with your preference remembered. (#261)
Packaging, config, and CI
- Layered configuration:
unfurl.ini,unfurl.local.ini,UNFURL_*environment variables, and CLI flags. (#266, #267) - Tests now ship with the package, so you can validate an installed copy with
python -m unittest discover -s unfurl.tests.unit. (#267) - Releases go to Test PyPI as a pre-release first, then to PyPI. (#267)
- CI has a separate lint job, pip caching, and cancels superseded runs. (#272)
🐛 Fixes
- The CLI no longer crashes with
UnicodeEncodeErroron a console that isn't UTF-8, such as a default Windows console. (#278) - JWT detection now requires a decodable JOSE header, ending false positives on dotted path segments like
Discovery-Cove-Orlando.d6068683.Vacation-Attraction. (#277) - An
sxsrfvalue without a colon no longer crashes the Google parser, which had skipped every Google parameter after it. (#281) - Plain-text output no longer runs words together where hover text had been wrapped. (#280)
Unfurl(remote_lookups=False)now disables lookups even when the config file enables them. (#269)- Fixed a parsing loop where a
1drv.mslink expanded to a URL containing itself, filling the graph with duplicate branches and repeating HTTP requests. (#270) - 15-digit Epoch ten-microsecond timestamps were decoded as microseconds, producing dates in the 1970s. (#265)
- Discord snowflake process IDs were always shown as 0. (#263)
- Remote lookups now time out, and short link and magnet lookups handle errors cleanly. (#248)
- Defanged input no longer logs an
Invalid IPv6 URLtraceback. (#283)
Full Changelog: v2026.04...v2026.09