Skip to content

Releases

v0.0.8

Choose a tag to compare

@nightshift-labs-ai nightshift-labs-ai released this 19 Nov 00:00

In short

  • Static keys get a class (hardening, feature, tracepoint or debug, read from the binary): List static-key sites has a Class column with hardening and feature keys first, Annotate's summary names the hardening keys and their build default (cr_pinning off (on in this file) on the MemLens demo kernel, a memory image), and --list jump --class <class> keeps one class.
  • More dynamic-debug sites read as pr_debug file:line "format" instead of descriptor.13+0x28: on the MemLens demo kernel 2,198 pr_debug sites (was 1,941).
  • The decompile check's wait can be cancelled: the comments and references stay and the summary says how to run the check later. Tested only with fake ida_* modules; not yet run in IDA.

Details

  • Decompile check (menu): you can cancel its wait for IDA to analyse the new references. Prompted by a judge-style review: the wait has no time limit, so on a whole vmlinux it can be long, and it had no Cancel button. The wait has a box of its own with a Cancel button; writing the comments and references, decompiling again and counting for the summary keep their boxes without one. On Cancel (ida_auto.auto_wait() returns false, as its IDA 9.1-9.5 stubs document) the second decompile pass is skipped, the comments and references Annotate wrote stay, and the summary window still opens, with decompile check cancelled: IDA goes on analysing; when it has finished, Remove annotations and Annotate again to run the check. The WARN fall-through waits keep no Cancel button: they read the reanalysed sites right after each wait. Tested only with fake ida_* modules (an auto_wait() that returns false); not yet run in IDA. tests/check_ida_api.py also checks the wait-box and auto_wait() documentation it relies on (replace_wait_box is no longer used).

  • Static keys get a class: hardening (a fixed list of 11 key names such as init_on_alloc, randomize_kstack_offset, cr_pinning, switch_mm_cond_ibpb, arm64's dynamic_scs_enabled), feature, tracepoint or debug (dynamic debug), all read from the binary. kflow: List static-key sites has a Class column and lists hardening and feature keys first; Annotate's summary window has one line with the hardening keys and their build default, only when there are any (at most 3 names, keys whose state in the file differs from the default picked first, then +N more); the command line prints the sites per class and the defaults, and --list jump --class <class> keeps one class (or says that it has no sites, or that the __jump_table was not found or not decoded); --json static-key records, also the headless runner's, have a class field. The build default is bit 0 (JUMP_TYPE_TRUE) of the key's struct static_key type word, not likely/unlikely (switch_mm_cond_ibpb is tested with static_branch_likely but is off by default); where the key's state in the file differs (a memory image), the line says so: cr_pinning off (on in this file) on the MemLens demo kernel (the window in the fake-IDA test: init_on_alloc on, randomize_kstack_offset on, cr_pinning off (on in this file), +7 more); bypass_usercopy_checks adds (checks on) / (checks off). python3 -m kflow <file>, re-run on this branch: MemLens demo kernel hardening 82, feature 5,229, tracepoint 1,728, debug 2,198 sites, init_on_alloc and randomize_kstack_offset on, the other 8 off; GKI 6.1 hardening 40, feature 7,906, tracepoint 2,683, debug 0, init_on_alloc on, init_on_free and randomize_kstack_offset off, as its embedded config says; try module dm-mod.ko feature 12, tracepoint 3, debug 7. In IDA a key in a SEG_BSS segment is off; the new IDA calls are covered by tests/check_ida_api.py and tests/check_ida_names.py (IDA 9.1-9.4 stubs). README: one sentence on the chooser line (1,906 -> 1,943 words, wc -w).

  • Dynamic-debug keys of pr_debug_ratelimited, dev_dbg_ratelimited and net_dbg_ratelimited (a struct _ddebug called descriptor.N) and _ddebug records without a symbol of their own now read pr_debug file:line "format" like the others, instead of descriptor.13+0x28 or __UNIQUE_ID_ddebug514+0x60 (the struct must read as one: a source-file name and a line). python3 -m kflow <file>, re-run: try module dm-mod.ko 5 of its 22 keys (pr_debug sites 2 -> 7, symbol+offset 5 -> 0); MemLens demo kernel pr_debug sites 1,941 -> 2,198 (68 descriptor* and 189 unnamed records), symbol+offset 439 -> 182, so the debug class is 257 sites larger than the 1,941 the summary used to print; Ubuntu 6.8 amd64 3,313 -> 3,382; GKI 6.1 has none.

  • Current totals in docs/VERIFICATION.md: 235 checks (tests/check_ida_api.py, IDA stubs 0.0.11; v0.0.7 said 232) and 96 ida_* names.

Built by an AI agent (Claude, operated by Nightshift Labs). Not yet run in real IDA.

v0.0.7

Choose a tag to compare

@nightshift-labs-ai nightshift-labs-ai released this 18 Nov 13:39

In short

  • Annotate ends with a summary window (menu only): N of M static-key NOP sites can jump to code nothing else in this database reaches, one line per table, and buttons Static-key sites, BUG/WARN sites and Close that open the lists. Tested only with fake ida_* modules; not yet run in IDA.
  • Pseudocode says when code is probably hidden: a static-key site's comment adds that nothing else in the database jumps or falls through to the code the site does not run here, so the pseudocode probably does not show it. The flow check is tested; what Hex-Rays then shows is not checked in real IDA, hence "probably".
  • Go to oops/WARN line can be tried on any kernel: python3 -m kflow <kernel> --oops-example prints a WARNING: line built from the file's own __bug_table, the IDA dialog offers the same kind of line the first time, and --oops also takes the line itself.
  • List static-key sites has a Block column (bytes / insns of the out-of-line block at each target), and the README has a "Without kflow / with kflow" table right after Status and a quick path on dm-mod.ko from the kflow-try-module.zip asset.
  • Requires IDA 9.1 or later (was 9.0): the IDA API names are checked only against the IDAPython stubs of 9.1 to 9.5.

Details

  • Supported IDA versions: 9.1 or later (ida-plugin.json idaVersions >=9.1; README, docs and the headless runner's messages say the same). The IDA API names are checked against the IDA 9.1 to 9.5 stubs only, so 9.0 was never checked; same rule as the sister plugin btf2ida. hcli no longer offers this version to IDA 9.0.
  • README front door: the opening leads with static keys (96% of the 8,349 NOP sites on the MemLens demo kernel, now explained at first mention as a Debian 13 6.12 x86_64 kernel exported from a memory image; re-measured with tools/reach.py: 8,037 of 8,349, 96.3%), then file:line without DWARF, then the oops report; the before/after and pseudocode blocks put each kflow comment under its instruction, wrapped (labelled so), so no code line is over 110 characters (before: 121, 176 and 230); the table's static-key row says what is lost without the reference, the WARN/BUG and oops cells get a verb and units (586 source lines map to 2+ traps, 506 of them in 2+ named functions, re-counted with python3 -m kflow's decoder), and a new Pseudocode row; the Features list is shorter (README 1,775 words before this release's additions, 1,906 now, wc -w). tests/test_core.py fails on a README code line over 110 characters; on a kernel with the README's addresses (the demo kernel), tests/test_entry_stub.py checks that each wrapped comment, unwrapped, is the code's own text. docs/VERIFICATION.md lists the try module dm-mod.ko (74 / 22 / 2 sites; six modules now).
  • docs/USAGE.md: "Try a real module" now starts with kflow-try-module.zip (Debian 13 x86_64 dm-mod.ko, as in the README): the two commands (python3 -m kflow dm-mod.ko: 74 / 22 / 2 sites; --oops "$(... --oops-example)": exact, rc 0, re-run) and the same three what-you-should-see steps; the arm64 ext4.ko download follows as the second option, its command lines wrapped to at most 110 characters (before: 122, 126 and 135).
  • Go to oops/WARN line, first try: when the dialog starts with the example WARNING: line built from the database, its prompt says so (kflow: below is an example built from this database's __bug_table; replace it with your kernel log ...); with an empty dialog or after a paste it shows the usual prompt (fake-IDA test). Command line: --oops also takes the line itself, so python3 -m kflow kernel.elf --oops "$(python3 -m kflow kernel.elf --oops-example)" gives exact, rc 0, on Debian 13 dm-mod.ko and the MemLens demo kernel (before: [Errno 2] No such file or directory, rc 2). An argument that is no existing file and has a newline, WARNING:, BUG, RIP:, pc :, +0x or :<digit> is read as the log, with a note on stderr, unless it reads like a path (no whitespace, and it starts with /, ~ or . or ends in .log/.txt: a missing /tmp/BUG-report.txt gave no match, rc 1, now no such file, rc 2; a bare func+0x674/0x9d0 is still read as the log); any other missing file still gives rc 2, now with a message naming the log file, - (stdin) and the quoted line. - reads stdin as before. README step 3 shows the round trip.
  • README quick path (kflow-try-module.zip): the command-line counts and Annotate are separate sentences, and three steps say what to look at after Annotate: the window's headline (22 of 22 in the fake-IDA test, which has no other references; real IDA may show fewer), F5 in dm_submit_bio (4 zoned_enabled sites and the block_split tracepoint; in the fake, all 5 comments end with the hidden-path sentence), and Go to oops/WARN line's first text WARNING: CPU: 0 PID: 1 at drivers/md/dm-zone.c:35 dm_blk_do_report_zones.isra.0+0x91/0xc0 [dm_mod], which lands on that ud2. Labelled as fake-IDA text; kflow has not yet run in real IDA. On dm-mod.ko, tests/test_entry_stub.py checks every number and name of that paragraph against its run and the command line, and tests/test_oops.py checks the quoted line against --oops-example.
  • Annotate's summary window (menu only) starts with the line kflow: Annotate finished (a plain line: the TITLE prefix that IDAPython passes to ask_yn is not documented for ask_buttons), and reads as a result: the headline N of M static-key NOP sites can jump to code nothing else in this database reaches: kflow's reference is the only one that leads there (with the decompiler: ; the site's pseudocode comment says so; when N is 0: ...reaches: every target has another way in.), one line per table in the README's nouns (74 BUG/WARN sites (37 WARN): a comment with the source file:line, 22 static-key sites: ..., 2 exception-table fixups: ... on Debian 13 dm-mod.ko), the sites left without a reference when IDA has no instruction there, the decompile check's line, and a next step: with the decompiler, press F5 in <function> (<n> hidden-path static-key sites) for the function with the most such NOP sites, and Go to oops/WARN line starts with a WARNING line built from this database when it can build one and nothing was pasted yet (if building it fails, the window shows without that line). The raw counts line (comments: ..., jump references added: ...) is now only in the Output window and in tools/kflow_headless.py's output, unchanged. Fake-IDA test on dm-mod.ko: 22 of 22, press F5 in dm_submit_bio (5 hidden-path static-key sites); tested only with fake ida_* modules; not yet run in IDA.
  • Annotate's summary window is an ida_kernwin.ask_buttons window with buttons Static-key sites and BUG/WARN sites (they open the two lists) and Close (the default; Esc). In its headline, M counts the static-key sites that are NOPs here and N those whose target has no other way in (the rule of the pseudocode sentence). No window when nothing was annotated; if the window fails, one line in the Output window says so. tools/kflow_headless.py prints text as before. Tested only with fake ida_* modules (on the MemLens demo kernel the fake gives 8349 of 8349 in one test and 8348 of 8349 in another, each the count that test works out itself); not yet run in IDA. tests/check_ida_api.py also checks ask_buttons (argument order, return values): 232 checks.
  • README: a "Without kflow / with kflow" table right after Status, static keys first. It says plainly what IDA and a debug build already give (IDA's Jump dialog takes func+off; a DWARF vmlinux has file:line; ordinary flow already reaches the fixups of 51% of the exception-table sites) and where kflow adds something (static keys, kernels without DWARF, WARN vs BUG, whole-report parsing and file:line to every inlined copy), credits linux_und.py and the SDK example ana_emu_out.py, and dates the "no public __jump_table decoder found" search. Every number re-measured with tools/reach.py and python3 -m kflow on the MemLens demo kernel (static keys also on Ubuntu 6.8).
  • Go to oops/WARN line: the dialog's pre-filled WARNING: line no longer names a function at an offset past its printed size. On GKI 6.1 arm64 two static functions are both called gic_handle_irq; IDA's name lookup gives the first one's start for both, so the dialog offered gic_handle_irq+0x190/0xf4, which pasted back was not an exact match. oops.example_line now skips a site whose offset is not below the printed size (the CLI's --oops-example already skipped such names; its output is unchanged).
  • python3 -m kflow <kernel> --oops-example prints a WARNING: CPU: 0 PID: 1 at file:line func+0xoff/0xsize line (with [module] for a module) built from the file's own __bug_table: the first WARN site with a file:line in a uniquely named function, so Go to oops/WARN line and --oops can be tried on any kernel, not only on the demo. In IDA, Go to oops/WARN line offers the same kind of line, built from the open database, as its default text the first time (after a paste it offers the last text, as before); tested with fake ida_* modules. Fed back to --oops it gives exact on the MemLens demo kernel, Debian 13 dm-mod.ko, Ubuntu 6.8 amd64 and Android GKI 6.1 (arm64); tests/test_oops.py checks the round trip.
  • kflow: List static-key sites has a Block column: the size of the out-of-line block at the site's target, as bytes / insns. It is decoded with ida_ua.decode_insn (no change to the database) from the target up to and including the first instruction that does not go on to the next one (IDA's CF_STOP; on arm64, by their bytes: b, brk, ret, br and eret end it (also retaa, retab, braa, brab, braaz, brabz, eretaa, eretab), b.cond and blr (also blraa, blrab, blraaz, blrabz) never do), and ends before a join (an instruction a jump or call reference r...
Read more

v0.0.6

Choose a tag to compare

@nightshift-labs-ai nightshift-labs-ai released this 17 Nov 11:24
  • Annotate (menu and tools/kflow_headless.py), with the Hex-Rays decompiler: a decompile check. kflow picks up to 20 functions with a static-key site that gets a new jump reference (spread evenly over those functions, the same every run; exception-table references are not checked), decompiles them before adding any reference and again afterwards, once IDA's auto-analysis has processed the new references (ida_auto.auto_wait(), with no time limit; from the menu, a wait box says kflow is waiting for IDA to analyse the new references) and after ida_hexrays.mark_cfunc_dirty(), so the second result is not the cached first one, and adds one line to the summary: all fine, or which functions decompiled before but fail after (each with Hex-Rays' own reason when it gives one, cut to 80 characters and ...), with a pointer to kflow: Remove annotations. Functions that failed before are counted apart and not blamed on kflow. While IDA's auto-analysis is still running the check is skipped (the line says so): run Remove annotations and Annotate again after analysis to get it. The same when kflow's references from an earlier Annotate are still in the database. In tools/kflow_headless.py these lines, and the pointer to Remove, give its own steps instead (run with --remove, then again, without --no-analysis when that skipped the analysis). It decompiles for about 5 s before Annotate and starts no new decompile once about 5 s more have passed after it (a slow function can stretch either part; functions over 16 KB are not picked); any error in the check gives decompile check skipped: <reason> and Annotate goes on. Prompted by a judge-style review: nobody has checked what Hex-Rays does with kflow's jump references from NOP sites. Tested only with fake ida_* modules (a fake decompiler that can fail chosen functions always or only after Annotate, with a cache); not yet run in IDA.

Built by an AI agent (Claude, operated by Nightshift Labs). Not yet run in real IDA.

v0.0.5

Choose a tag to compare

@nightshift-labs-ai nightshift-labs-ai released this 16 Nov 00:07

In short

  • tools/reach.py measures what kflow's references add: on the MemLens demo kernel, 96% of the static-key NOP sites jump to code that nothing else leads to.
  • python3 -m kflow <kernel> --json writes labelled records (the same keys as the headless --json), and both report a table that was found but not decoded.
  • --limit refuses a negative number instead of printing a wrong count.
  • Not yet run in real IDA.

Details

  • tools/reach.py measures what kflow's references add (results in tools/reach_results.md, rules checked by tests/test_reach.py): on the MemLens demo kernel, 96% of the 8,349 static-key NOP sites jump to code that no direct branch targets and no instruction falls into (8,011 out-of-line blocks, 264 KB), so kflow's reference is the only one that leads there. After 90% of the 7,504 WARN traps the code is hidden too (1.7 instructions on average up to and including the next unconditional jmp, ret or ud2, or up to the next branch target); Annotate adds no reference at WARN sites, and only the experimental WARN fall-through reaches that code. x86_64, needs objdump; about 15 s per kernel.
  • --limit refuses a negative number (exit 2, must be 0 or more); --limit -1 used to print all rows but the last and then a wrong ... N more count.
  • python3 -m kflow <kernel> --json out.json writes one labelled record per site with the keys tools/kflow_headless.py --json uses (BUG/WARN: addr, kind, function, file, line, plus flags; static key: addr, key, kind, target, function, without the headless now; exception table: addr, fixup, type, handler, function), with the ELF's symbol names and hex-string addresses. It wrote unlabelled arrays before, in a different order from the bug_layout it printed, plus ex_layout, the raw exception-table data word and the static-key init bit, which it no longer writes. Both writers now add errors: a table found but not decoded, whose list is then empty. 19,758 records on the MemLens demo kernel.
  • README Features: Go to oops/WARN line says that a BUG: KASAN:, lockdep or hung-task report gives no match unless its first reliable stack entry is a BUG/WARN site.

Built by an AI agent (Claude, operated by Nightshift Labs). Not yet run in real IDA.

v0.0.4

Choose a tag to compare

@nightshift-labs-ai nightshift-labs-ai released this 15 Nov 00:02

In short

  • New: kflow: Go to oops/WARN line (also python3 -m kflow <kernel> --oops <log>). Paste a WARNING: or kernel BUG at line, a whole oops or syzbot report, func+0xoff or file:line, and kflow goes to that BUG/WARN site; when the log contradicts the site at that address (for example a log from another build), it says mismatch and goes nowhere.
  • New, experimental (x86_64, off by default): kflow: Let WARN sites fall through (experimental). IDA ends code flow at a WARN ud2; this adds the flow after it, so the code after the trap becomes part of its function.
  • Safer Annotate: no references from bytes that are not instructions yet (Annotate during auto-analysis), and a static-key target or fixup inside another function gets a data reference instead of a jump.
  • Pseudocode comments also appear when the decompiler loads after kflow; tools/kflow_headless.py --json exports every decoded site; the README starts with a Features list.
  • Tested on real kernel files with fake ida_* modules; not yet run in real IDA.

Details

  • Annotate run while IDA's auto-analysis is still going no longer adds references from sites that are not instructions yet. The IDA SDK says the source of a cross-reference must be an item head and that get_item_end() counts unexplored bytes as 1-byte items, so on a freshly loaded kernel kflow's "the fixup is the next instruction" test could not match: such exception-table entries (common on arm64 from 5.16 and at x86's plain rdmsr/wrmsr, EX_TYPE_RDMSR/WRMSR) got a user jump reference that a later Annotate kept. Now a site that is not code (ida_bytes.is_code(ida_bytes.get_flags(ea))) gets its comment but no jump, data or static-key reference, and Annotate reports how many such sites there are and says to run it again after auto-analysis (also tools/kflow_headless.py --no-analysis). A later Annotate deletes a reference to the next instruction that an earlier one recorded (also one added by an older kflow), and reports it. This follows from reading the IDA SDK (ida_xref, ida_bytes stubs), not from a run in IDA; tested with fake ida_* modules only, which now model instruction heads (get_flags/is_code), real instruction ends where known and no functions while analysis runs. Found by a pre-release review of the IDA calls. tests/check_ida_api.py also checks ida_bytes.get_flags and is_code. When that later Annotate (or Remove) deletes such a reference and IDA's ordinary flow to the next instruction is gone afterwards (the SDK does not say whether del_cref() clears it), kflow puts the ordinary flow back (fl_F). On a database whose auto-analysis has finished, sites that are still not code are reported as "not code in IDA" with a hint to press C there, instead of "run Annotate again when auto-analysis has finished".
  • Pseudocode comments when the Hex-Rays decompiler loads after kflow: kflow tried its pseudocode hook only when it loaded (and on Annotate), so a database annotated earlier showed no comments until Annotate ran again. IDAPython initialises its decompiler bindings when the decompiler plugin loads (ui_plugin_loaded in IDAPython's py_hexrays.hpp), so when the decompiler is not there at kflow's start, kflow now installs a ida_kernwin.UI_Hooks whose plugin_loaded tries again until the hook is in, and removes it when kflow unloads. Whether IDA loads the decompiler before or after kflow was not checked. This follows from reading the IDA SDK; tested with fake modules only (a fake decompiler that is missing at kflow's init and present later). tests/check_ida_api.py: 205 checks, 0 missing; tests/check_ida_names.py: 86 names, 0 missing in the stubs of IDA 9.1 to 9.5.
  • New action kflow: Go to oops/WARN line and python3 -m kflow <kernel> --oops <log> (- for stdin; --json writes the match): paste a WARNING:, kernel BUG at, RIP: or pc : line, a whole oops or syzbot report, func+0xoff or file:line. kflow takes the function+offset (from the report's own RIP: / pc : line first, then the WARNING: or kernel BUG at line) and goes to the BUG/WARN site at exactly that address (a name the log prints without a Clang LTO suffix, foo for foo.123, foo.llvm.N or foo.cfi, tries all those functions, the plain one too; GCC's foo.part.N, .isra.N, .constprop.N and .cold are printed in full, so foo+0x10 never means them; the output names the function used); if there is none it says so, goes to the log's address and lists that function's sites (those at the log's file:line first, then nearest first; the function as this file has it, whatever size the log prints (in IDA up to the end of its last chunk before the next name: code after a gap, such as after a WARN ud2, can be a tail chunk), and an offset past its end, from another build, lists them without going anywhere, also when the next function has a site at that address); with only a file:line shared by several sites it lists just those (in IDA: the BUG/WARN chooser with only those rows). When the log contradicts the site at its address (its WARNING: / kernel BUG at line names another file:line or kind; or that line has no file:line (a kernel without CONFIG_DEBUG_BUGVERBOSE) and its function+offset names another address than the RIP: / pc : line (Kernel BUG at %pB names a BUG at a function's first byte as the previous symbol plus its size: the same address); or, without a WARNING: / kernel BUG at line that names the site, it prints another function size than this file's, kallsyms' size to the next symbol counted: a log from another build, or two reports pasted together) kflow says mismatch, exits 1 and goes nowhere; it lists the sites at the log's file:line, else the site at that address (in IDA: a warning and the chooser). The same when the log names a function+offset this file does not have and its file:line has one site here (that line's site may be another one). A paste with several reports is cut into them (at cut here / end trace, the next WARNING: / kernel BUG at line, or another report's first line after a register dump; not at syzbot's repeated WARNING: line: syzbot prints a WARNING: or RIP: line once for each inlined frame, func file:line [inline], before the one with the function+offset, which is the one used; not at Kernel panic - not syncing:: that line and the stack dump after it belong to the report in progress, so a panic_on_warn report, whose panic dump on x86 shows the trap's RIP: line again, counts as one; not at x86's executive summary after end trace, the trap's RIP: line and registers printed again, which belongs to the report before it); the first one with a WARNING: / kernel BUG at line or a trap RIP: / pc : line is used, and a note says how many there are. A RIP: / pc : line after the report's call trace began (x86 prints an interrupted context's registers where the trace crosses an IRQ or exception entry, as in a KASAN report from an interrupt) or after NMI backtrace for cpu N (another CPU's registers, in hung-task and RCU-stall reports) is not the report's trap, nor is a call-trace entry that repeats it (arm64 repeats the pc there). The trap's own lines after the trace are used, so the bottom of a report pasted alone still finds its trap: the executive summary of Linux 4.9 (RIP [<%p>] %pS) and of 4.10 to 4.16 (RIP: %pS RSP: %016lx), and the RIP: right after the #UD entry frame asm_exc_invalid_op outside an NMI backtrace (panic()'s dump under panic_on_warn). A report with no WARNING: / kernel BUG at line and no trap RIP: / pc : line (KASAN, lockdep, hung task) or a pasted call trace of two or more entries counts only if its first reliable entry is itself a BUG/WARN site (that entry is a caller, not a trap); otherwise kflow says there is no BUG/WARN trap in it (naming such a RIP: / pc : line, if there is one) and exits 1; ? entries are never used. --oops with --list or --limit is an error (exit 2); a module file named foo.ko.debug (Fedora / RHEL debuginfo) is module foo; in IDA a jump that fails (an address outside the database) gives a warning. On the demo kernel file:line alone matched four sites at mm/userfaultfd.c:755; mfill_atomic_copy+0x674 picks 0xffffffff8e629bc4. The line formats are taken from the kernel's printk calls (__warn() up to 6.18 and from 6.19, report_bug(), x86 show_ip() and call traces, arm64 __show_regs(), %pS with [module], the 4.9 forms, 2.6 / 3.x WARNING: at file:line func+0xoff() and Kernel BUG at <address>) and from syzbot's reports (the source line syzbot appends after the symbol of a WARNING: / Kernel BUG at line is not taken as that line's file:line: for an inlined trap it is the outer frame's); the prefixes of dmesg (also -T, -H, -d, -x, --time-format iso), /dev/kmsg, journalctl (short, -precise, -iso, -monotonic) and logcat -b kernel (its time and level; logd makes the first word of a pc :, lr : or RIP: line the tag, so from logcat such lines are not recognised as pc : / RIP: lines) are removed, also before call-trace entries. tests/test_oops.py: log samples built from those formats; on each test kernel and module, a RIP: / pc : line made for every BUG/WARN site inside a function finds that site (for example all 24,386 on Android GKI 5.10; same-named static functions with a site at the same offset are listed as candidates; the 231 sites there in functions with a Clang LTO suffix also from the name the kernel prints, cpu_replace_ttbr1 for cpu_replace_ttbr1.989: found, or listed among the copies of that name), also with the site's own WARNING: / kernel BUG at line before it, also with and from the line a kernel without CONFIG_DEBUG_BUGVERBOSE prints for it (WARNING: ... at %pS, Kernel BUG at %pB), and with another line or kind is a mismatch (among same-named functions: listed). The IDA action is tested ...
Read more

v0.0.3

Choose a tag to compare

@nightshift-labs-ai nightshift-labs-ai released this 14 Nov 00:00
  • Hex-Rays pseudocode comments: with the decompiler, each annotated BUG/WARN, static-key and exception-table site whose address has a pseudocode statement gets a display-only // kflow: ... comment at the end of that line; a static-key site with no statement of its own (usually a NOP) gets it on the line of the next instruction that has one if that is at most 16 bytes on in the same function with no other kflow site in between, otherwise none. Nothing is saved; Annotate and Remove apply to functions decompiled afterwards (press F5 in an open pseudocode window). Tested only against a fake ida_hexrays module; not yet run in the real decompiler.
  • kflow: List static-key sites now says when the __jump_table was found but could not be decoded, and why (it said "the table is empty").
  • Kernel modules in IDA: a static key inside an imported symbol at an offset (a tracepoint's key is __tracepoint_X + 8; arrays such as nf_hooks_needed) could be named after a different import if IDA packs the imports closely in its extern segment (likely; not verified); kflow now names it from its own decode of the .ko on disk when both decodes agree site by site, and points the key reference at that import (a key reference an earlier Annotate put elsewhere at that site is replaced, so Remove still finds every reference kflow added). Tested only on fake ida_* modules (Debian 13 bridge.ko: 14 sites, 9 keys); not yet run in real IDA.
  • kflow: Remove annotations (and kflow_headless.py --remove) also reports how many static-key site -> key references it removed; it counted only the jump references.
  • Annotate (menu and tools/kflow_headless.py) reports its counts in words, including the static-key site -> key references: comments: 18999, jump references added: 9006, static-key site -> key references: 9012, user comments kept: 0 on the MemLens demo kernel (fake ida_* modules); the headless runner printed a Python dict before.
  • tests/run_all.sh: every test in one command, on the pinned Debian arm64 ext4.ko package (377 KB download, sha256 checked) or on files you give it.
  • README: how to go from a WARNING: ... at file:line func+0xoff log line to the site (filter the BUG/WARN list, or G and func+0xoff).

Built by an AI agent (Claude, operated by Nightshift Labs). Not yet run in real IDA.

v0.0.2

Choose a tag to compare

@nightshift-labs-ai nightshift-labs-ai released this 13 Nov 10:06
  • ELF files with extended section numbering (0xff00 or more sections: e_shnum 0, e_shstrndx SHN_XINDEX, symbol section indexes in .symtab_shndx) are read; before, kflow saw no sections in them. Synthetic test.
  • A kernel module's relocations are applied in place instead of in a second copy of the file (output unchanged on nf_tables.ko and ext4.ko).
  • A raw arm64 kernel Image (also gzip-compressed, as Ubuntu's arm64 vmlinuz) or an x86 bzImage / vmlinuz now gets a message naming the tool that turns it into an ELF (vmlinux-to-elf, or extract-vmlinux on x86), not just "is not an ELF file"; in IDA, the warning for a database with none of the three tables says the same for a raw Image / vmlinuz loaded as a binary file.
  • Checked on two more kernels, Ubuntu 24.04's 6.8.0-149-generic for x86_64 and for arm64 (each vmlinuz through vmlinux-to-elf): x86_64 11,027 BUG/WARN sites (8,259 WARN), 8,304 static-key sites (all keys named), 632 exception-table entries (the 45 WRMSR and 41 RDMSR entries all sit on wrmsr/rdmsr); arm64 10,803 BUG/WARN sites (8,085 WARN, all brk #0x800), 31,250 static-key sites (all keys named, 22,918 of them tracepoints), 935 exception-table entries. README table rows added.
  • README "Try it": the quickest sample is the vmlinuz your own Linux machine boots, through vmlinux-to-elf (with the copy step for a root-only /boot).
  • Static-key comments say when the site jumps: "jumps to X when the key is off" (static_branch_likely sites) or "... is on" (static_branch_unlikely), from the kernel's rule that a site is a jump when the key's state XOR the entry's branch bit is set (jump_label_type()). Found by a judge-style review.
  • Modules: a static key inside an imported array was named after the NEXT import (Debian 13 bridge.ko: sprintf+0x40 for nf_hooks_needed + 0x270): each import now gets a 64 KB slot and a key in it is named from that import, exactly (checked against readelf -r). Found by a judge-style review.
  • README opens with who it is for and a before/after of three real sites (objdump instruction vs kflow's comment). Found by a judge-style review.
  • New action kflow: List static-key sites: every static-key site with its key, likely/unlikely, whether its bytes are a jump to the target now, the target and the function (the chooser's filter finds all sites of one key). Suggested by a judge-style review. Fake-IDA test on the demo, GKI 5.10, nf_tables.ko and a stripped vmlinux (no jump table: one warning).
  • Keys inside an array of keys are named after the array: when nothing is named within 64 bytes before a key, the nearest name up to 4 KB before it is used (cpu_hwcap_keys+0x50: 24,470 sites on Android GKI 5.10 that showed only an address; also nf_hooks_needed, sched_feat_keys, cgroup_bpf_enabled_key). Every key on the seven test kernels now has a name (demo kernel: 9,237 of 9,237, was 8,898). In IDA the name comes from IDA's name list (get_nlist_idx and its neighbour); the fake-IDA test checks that IDA and the command line give every key the same name. Found by a judge-style review.
  • Annotate also adds a user data reference from each static-key site to its key, so the key's cross references (X) list every site it patches; Remove deletes only the ones kflow added (recorded in the database, also after a rebase), and a reference you made yourself stays. Suggested by a judge-style review; fake-IDA tests only.
  • Names in comments and in the static-key list never come from IDA's placeholder names (unk_..., byte_..., which IDA gives referenced addresses, keys included once Annotate adds its key references): kflow asks for the name with get_ea_name(ea, GN_NOT_DUMMY). Found by a pre-release review of the IDA calls; the fake-IDA test's get_name now returns placeholder names like IDA, and the test fails if one shows.

Built by an AI agent (Claude, operated by Nightshift Labs). Not yet run in real IDA.

v0.0.1

Choose a tag to compare

@nightshift-labs-ai nightshift-labs-ai released this 12 Nov 00:36

First release.

kflow reads three tables that every Linux kernel carries and IDA does not decode:

  • __bug_table: every BUG() / WARN() trap (ud2 on x86_64, brk #0x800 on arm64; on x86_64 from Linux 6.19 a WARN with a message is a call, and kflow says so) with its source file and line, and whether the kernel continues after it (WARN) or not (BUG).
  • __jump_table: every static-key site (a NOP or branch the kernel patches at runtime), its key and its branch target.
  • __ex_table: every instruction allowed to fault and its fixup.

In IDA (Edit > Plugins > kflow): Annotate kernel sites comments every site and adds the jump references IDA cannot see (static-key targets, fixups) where IDA has none; List BUG/WARN sites lists every trap with its function and file:line; Remove annotations takes out kflow's comments and the references Annotate recorded adding (also after a rebase), and leaves earlier references (IDA's or yours) and your comments alone. tools/kflow_headless.py does the same through idalib, and python3 -m kflow <kernel ELF> decodes the tables without IDA.

Kernel modules (.ko) work too: the command line applies the module's relocations, and in IDA the tables are found by segment name. Static keys are named where possible: tracepoints as tracepoint <event>, dynamic-debug keys as pr_debug file:line "format" (8,898 of 9,237 keys on the demo kernel). Checked on five kernels and five modules outside IDA (README "What was verified": trap bytes, branch targets, source-file names): Debian 13 6.12 x86_64 (MemLens demo: 9,850 BUG/WARN sites, 9,237 static-key sites, 671 exception-table entries), a stripped Debian x86_64 vmlinux (tables found by section name), Android GKI 5.10.198-android12-9 arm64 (24,386 / 75,316 / 1,488; the 54,896 static-key sites outside init code that are compiled as a branch go to the decoded target); Android GKI 6.1.57-android14-11 arm64 (12,223 / 10,629 / 1,043); Debian 11 5.10.223 x86_64 (6,573 / 3,820 / 548; the older exception-table layout whose third field is a handler function: all 548 handlers are the kernel's ex_handler_* functions); module: Debian 13 nf_tables.ko (141 BUG/WARN sites, every file:line matches the v6.12.115 sources; 8 static-key sites with named keys) and Debian 13 arm64 fat.ko, vfat.ko, ext4.ko, jbd2.ko (676 BUG/WARN sites, every file:line on a BUG/WARN or assertion macro or a fortified copy in the v6.12.94 sources; 178 static-key sites, 174 of them tracepoints; 36 exception-table entries).

x86_64 and arm64, 64-bit kernels only. IDA API use: 64 names and object members checked against the official IDAPython stubs, 0 missing.

Built by an AI agent (Claude, operated by Nightshift Labs). Not yet run in real IDA.

Try a real module (Debian 13 dm-mod.ko, x86_64)

Choose a tag to compare

@nightshift-labs-ai nightshift-labs-ai released this 13 Nov 17:28

A small real kernel module with all three tables kflow reads, for trying kflow in IDA in a few minutes (IDA analyses a module in seconds).

  • dm-mod.ko: Debian 13's device-mapper module (kernel 6.12.115+deb13.1-cloud-amd64, x86_64), uncompressed.
  • README.txt: where it comes from, the GPL-2.0 source pointer, the steps and the expected command-line output.

python3 -m kflow dm-mod.ko finds 74 BUG/WARN sites (37 WARN), 22 static-key sites and 2 exception-table fixups. In IDA: open dm-mod.ko, then Edit > Plugins > kflow: Annotate kernel sites.

Not a plugin release (no plugin inside). Built by an AI agent (Claude, operated by Nightshift Labs).