Mem4J 2.0: cross-platform, embedding-friendly, ref-counted, tested#5
Merged
Conversation
The README previously stated 'all rights reserved'; with no license file downstream consumers could not legally integrate Mem4J in their projects. The CHANGELOG documents the breaking and additive changes shipped on top of 1.0.0.
…tect
Embedding-friendly error handling
* New exception hierarchy under it.adrian.code.exceptions:
Mem4JException (root), PrivilegeException, ProcessNotFoundException,
ModuleNotFoundException, MemoryAccessException.
* NativeAccess.ensurePrivileged() throws PrivilegeException; missing
process or module throws ProcessNotFoundException /
ModuleNotFoundException. The library no longer calls System.exit(-1)
or pops MessageBoxes, so it is safe to embed inside larger apps.
Pointer / Memory
* Pointer implements AutoCloseable; close() releases the OS handle / fd.
* Long offsets honoured end-to-end (Memory.readMemory / writeMemory and
Pointer.add(long)); previously truncated to int silently.
* Bulk I/O: readBytes, writeBytes, readString / writeString (any
Charset, NUL-terminated decode), readByte / writeByte,
readShort / writeShort.
* Configurable endianness via Pointer.withByteOrder(ByteOrder).
* Pointer.indirect32() for 32-bit pointer chains; 64-bit zero-extended.
* Failed reads throw MemoryAccessException instead of returning zero
bytes silently.
* Pointer.force() returns a sibling pointer whose writes flip the
affected pages to PAGE_EXECUTE_READWRITE, write, and restore the
original protection (no-op on Linux: /proc/<pid>/mem ignores
page protection for CAP_SYS_PTRACE callers).
Cross-platform module enumeration
* NativeAccess.listModules(int pid) + ProcessUtil.listModules return
List<ModuleInfo> { name, path, baseAddress, size } on both Windows
and Linux.
Cross-platform AOB scanning
* SignatureUtil.findSignature(ProcessSession, ...) reads in 64 KiB
chunks so ranges larger than the heap can be searched.
* SignatureManager has new constructors taking Pointer or
(ProcessSession, moduleName); no longer closes the handle itself.
* Windows-only overloads kept as @deprecated for backward compatibility.
Memory protection / allocation
* NativeAccess.protect / allocate / free + queryProtection added.
* Windows wraps VirtualProtectEx / VirtualAllocEx / VirtualFreeEx /
VirtualQueryEx via the custom Kernel32 interface.
* Linux supports queryProtection by parsing /proc/<pid>/maps; the
write/alloc/free trio throws UnsupportedOperationException (would
require syscall injection).
* pom: attach a sources jar (maven-source-plugin) and a Javadoc jar (maven-javadoc-plugin) so downstream consumers see Mem4J docs and sources in their IDE. * CI matrix: run mvn package on ubuntu-latest and windows-latest for every push and PR to master. The dependency-graph submission still runs only on Linux (pushes only) to avoid duplicate snapshots. * README rewritten to reflect the embedding-friendly exception model, the new bulk I/O / endianness / force-write APIs, the cross-platform AOB scanner, and the Windows memory-protection wrappers. New Linux-only section with three end-to-end examples (ELF magic, patching a heap global, following a pointer chain in a 64-bit Linux target), plus notes on CAP_SYS_PTRACE and ptrace_scope.
The original Quick start only showed a Windows process name with a parenthetical comment about Linux; split into two side-by-side snippets so the Linux path is just as discoverable as the Windows one.
Removed leftover claims that contradicted the new code:
* The Architecture section still listed SignatureManager / SignatureUtil
as Windows-only; they are now cross-platform via NativeAccess.
* Several examples (Attaching, Pointer chains) used bare assignment
instead of try-with-resources, contradicting the AutoCloseable
guidance stated earlier in the document.
* Snippets used Windows-only process names without the corresponding
Linux equivalent.
Restructured the document:
* Quick start keeps the side-by-side Windows / Linux examples.
* Usage section now opens with a single try-with-resources scaffold
that applies to every snippet that follows, so each snippet shows
only the body (no duplicated boilerplate).
* 'Linux examples' was tacked on at the end of Usage; merged it with
new Windows-specific notes into a single 'Platform notes' section,
preserving the ELF-magic recipe and the Hollow_Knight pointer-chain
example.
* Utilities table reordered: NativeAccess rows grouped, helpers below,
Pointer.force placed next to queryProtection.
* Cheat sheet rewritten to match the actual class shape, including
force() and queryProtection.
* New overload Pointer.getBaseAddress(String name, int pid) for when several processes share the same executable name. The single-arg overload (name only, OS process-list lookup) is preserved as the common-case convenience. * LinuxAccess.protect / allocate / free now have a real implementation on x86_64 via ptrace syscall injection: PTRACE_ATTACH, save RIP and user_regs_struct, patch in 'syscall; int3' at the current RIP, configure RAX + SysV-ABI argument registers for mprotect(2) / mmap(2) / munmap(2), PTRACE_CONT, capture RAX on the int3 trap, restore the original bytes and registers, PTRACE_DETACH. Non-x86_64 Linux still throws UnsupportedOperationException. The injection helper is currently *experimental*: the integration test that round-trips mmap → write → read → munmap deadlocks when the target is attached mid-nanosleep, so it ships @disabled until the helper is hardened against interrupted-syscall entry. The implementation itself compiles and links on every Linux JVM.
* pom: declare junit-jupiter-{api,engine} test dependencies, pin
maven-surefire-plugin to 3.2.5 (JUnit 5 native discovery).
* src/test/java/it/adrian/code/Mem4JTests.java: 11 end-to-end tests
exercising the active backend against the running JVM, no mocks.
Privilege- and OS-aware: tests that need /proc/<pid>/mem or ptrace
use Assumptions.assumeTrue to skip cleanly on unprivileged or
Windows runners instead of failing.
Covered: backend selection (Windows vs Linux), findPidByName for a
missing process, ProcessNotFoundException, listModules of the self
process, reading the ELF magic of /proc/self, the new PID overload,
MemoryAccessException on unmapped reads, byte-order flip, long-range
add(), queryProtection of a mapped page.
The mmap/mprotect/munmap round-trip through the ptrace injection
helper is included but marked @disabled until the helper is hardened
(see previous commit).
* Features bullet on memory protection now mentions the Linux x86_64 ptrace-injection path (experimental). * 'Attaching to a process' section documents Pointer.getBaseAddress(name, pid) alongside the existing name-only overload, with a short snippet for the multi-instance disambiguation case. * 'Memory protection and allocation' is no longer marked 'Windows-only'; the section now describes both backends and is explicit that the Linux implementation is experimental (integration test @disabled). * New 'Testing' section explains 'mvn -B test', the privilege-aware @EnabledOnOs / Assumptions design, and the CAP_SYS_PTRACE recipe for local runs. * Utilities table and cheat sheet updated: protect/allocate/free row flipped from 'Windows' to 'both', new Pointer.getBaseAddress(name, pid) row, cheat sheet exposes the second overload. * Limitations rewritten to reflect the new reality (PID overload exists, non-x86_64 Linux still lacks memprotect, test suite shipped).
The bullet list was last updated when the API surface was still small; several real footguns shipped with the recent overhaul were missing. * Restate the macOS / bitness / anti-cheat bullets unchanged. * Replace 'no PID overload, first match wins' with the actual current behaviour: there IS a name-based default plus a (name, pid) overload. * Split memory-protection into two bullets: experimental ptrace path on Linux x86_64 (still @disabled in tests, deadlocks mid-nanosleep) vs. outright unimplemented on non-x86_64 Linux. * Document that Pointer.copy() shares the underlying session — closing any sibling closes them all, which is the most likely cause of 'handle invalid' errors mid-flow. * Note that Pointer is not thread-safe (add/indirect/withByteOrder/force mutate the receiver) and recommend per-thread copy(). * Note that AOB scans skip unreadable 64 KiB chunks silently — a match inside such a region cannot be found. * Add anti-debug caveat: Mem4J does not try to hide from targets that inspect ptrace state or handle counts.
* Architecture diagram now annotates each backend with the system
calls it actually wraps (Virtual{Protect,Alloc,Free,Query}Ex on
Windows; ptrace syscall injection for mprotect/mmap/munmap on
Linux x86_64 alongside the /proc/<pid>/* set).
* Features list: process attachment bullet mentions the new
Pointer.getBaseAddress(name, pid) overload; force() bullet spells
out the Linux no-op (was previously vague).
* Utilities table: force() row now distinguishes Windows
flip-and-restore from Linux no-op explicitly.
* Cheat sheet:
- Pointer block: add getMemory(int), getSession(),
getBaseAddressValue(), getOffset(), and the second
getBaseAddress overload. Marginal comments call out that
copy() shares the handle and add(long) is mutating/fluent.
- NativeAccess block: throwProcessNotFound listed; the
protect/allocate/free comments reflect the *Linux x86_64*
ptrace path, not just 'Linux'.
- Signature block: explicit @deprecated WinNT.HANDLE shim
constructor; SignatureUtil's deprecated overloads noted.
- ProcessUtil block: deprecated getModule shown so readers
don't think it's gone.
- Shell32Util block restored after it was dropped during the
previous structural rewrite.
* Testing section: replaces the (now incorrect) claim that
'sudo mvn -B test runs everything, including the ptrace
round-trip' — the ptrace test is currently @disabled and is
skipped regardless of privileges. Adds the observed count
(11 tests, 1 skipped) for transparency.
The 'Limitations & caveats' list was carrying three bullets that
described features which are now implemented:
* Process attachment limited to executable name → fixed by
Pointer.getBaseAddress(String, int pid).
* Memory protection / allocation Windows-only → fixed by the Linux
x86_64 ptrace syscall-injection helper.
* No test suite → fixed by the JUnit 5 integration suite.
Keeping them in the limitations list misleads readers into thinking
the gaps are still open. The experimental status of the ptrace path
is still documented in its dedicated 'Memory protection and
allocation' section, and the PID overload is documented in 'Attaching
to a process'. The non-x86_64 Linux gap is also covered in the
dedicated section, so it doesn't need to be re-stated here either.
Limitations now lists only genuine constraints (no macOS, bitness
match, no anti-cheat bypass, shared session across copies, no thread
safety, AOB chunking, no stealth).
ProcessSession is now reference-counted (AtomicInteger). Pointer.copy()
calls retain(); Pointer.close() calls release() and only tears down
the underlying OS handle when the refcount hits zero. A
java.lang.ref.Cleaner registered on each Pointer performs the same
release on GC, so a Pointer that is never explicitly closed no longer
leaks the OS handle. Concretely:
* Closing a copy no longer invalidates the original (or any other
sibling) — the 'copy()' + 'close()' footgun is gone.
* Pointer.close() is now idempotent — calling it twice or letting
try-with-resources fire after a manual close() is harmless.
SignatureUtil.findSignature now consults NativeAccess.queryProtection
before each 64 KiB read. Unreadable regions (MemoryProtection.NONE)
are skipped *explicitly* instead of surfacing as silent readMemory
failures.
Two new tests:
* closing_a_copy_does_not_invalidate_the_root verifies the refcount
behaviour and that reads through the original keep working after
the copy is closed.
* close_is_idempotent verifies a second close() is a no-op.
…xed in user space
The previous list mixed real constraints with bullets describing
behaviours that were addressed by the recent commits. After the
session-refcount fix, the explicit queryProtection skip in AOB
scanning, and the Cleaner-driven release on GC, the genuine
limitations of the library reduce to two:
* macOS is not yet supported. A Mach-backed MacOSAccess is on the
roadmap; implementing it speculatively without a real Mac to test
against would ship code that almost certainly misbehaves.
* No anti-cheat / kernel bypass and no anti-debug. These are
deliberate: Mem4J goes through documented OS APIs and does not
try to hide its activity (ptrace traces, handle counts).
Bullets removed:
* 'Pointer shares its session across copies' — refcount + Cleaner
means copy() and close() now compose correctly.
* 'Pointer is not thread-safe' — moved into a proper 'Concurrency
and lifecycle' subsection under Usage with the recommended
per-thread copy() pattern and a concrete executor example. The
bullet's content survives where it actually helps the reader.
* 'AOB scanning reads in 64 KiB chunks' — the silent-skip
behaviour is now an explicit queryProtection-driven skip; not a
library limitation.
* 'Bitness must match' — already documented in the Requirements
table as a hard prerequisite, not a Mem4J limitation.
* The 'No anti-debug / stealth' bullet has been folded into the
anti-cheat bullet, which now mentions both.
The Pointer cheat-sheet row for close() has been reworded to mention
that it's idempotent and refcount-aware, and the copy() row notes
that it now bumps the reference count.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Porta in
mastertutto il lavoro accumulato suexperimental. Sostituisce la chiusa #4 (era stata chiusa con il tip vecchio).What ships
Cross-platform backend (
it.adrian.code.platform)NativeAccessabstraction with reflective backend loading —WindowsAccessandLinuxAccessare loaded lazily, so the unused backend's native libraries are never initialised./proc/<pid>/{maps,mem,comm,exe}+libc geteuidfor the privilege check.ProcessUtil.listModules(pid)returnsList<ModuleInfo>on both platforms.SignatureUtil.findSignature(ProcessSession, ...), newSignatureManager(Pointer)/SignatureManager(ProcessSession, name)constructors. LegacyWinNT.HANDLEoverloads kept as@Deprecated.NativeAccess.queryProtection / protect / allocate / free. Windows wrapsVirtual{Protect,Alloc,Free,Query}Ex; Linux x86_64 implementsprotect/allocate/freevia experimental ptrace syscall injection (integration test currently@Disabled— deadlock when target is mid-nanosleep).Embedding-friendly error model
Mem4JException→PrivilegeException,ProcessNotFoundException,ModuleNotFoundException,MemoryAccessException.System.exit(-1)/MessageBoxpop-ups; every failure throws.MemoryAccessExceptioninstead of returning zero bytes silently.PointeroverhaulAutoCloseable.copy()retains,close()releases. The OS handle is only torn down by the last livePointer.close()is idempotent. Ajava.lang.ref.Cleanerreleases the reference on GC, so forgettingclose()no longer leaks the handle.readBytes/writeBytes,readString/writeString(anyCharset),readByte/Short+ write.withByteOrder(ByteOrder).indirect32()for 32-bit pointer chains.force(): sibling pointer that bypasses page protection on writes (Windows flipsPAGE_EXECUTE_READWRITEthen restores; Linux no-op because/proc/<pid>/memalready ignores protection withCAP_SYS_PTRACE).Memory.readMemory/writeMemory(was silently truncated to 32 bits).Pointer.getBaseAddress(String name, int pid)for when several processes share the same executable name.Build, CI, testing
LICENSE(MIT) +CHANGELOG.md.pom.xml: sources & Javadoc jars produced bymvn package, JUnit 5 + Surefire 3.2.5.jitpack.ymlpins the JitPack build to OpenJDK 11.ubuntu-latest+windows-latest;setup-java@v4(no moreset-outputwarning);permissions: contents: writeso the dependency-graph submission no longer 403s.src/test/java/it/adrian/code/Mem4JTests.java, 13 tests, end-to-end against the running JVM): backend selection,ProcessNotFoundException, module enumeration, ELF magic read, PID overload,MemoryAccessException, byte-order flip, long-offset add,queryProtection, ref-count copy semantics, idempotent close. The ptrace round-trip is included but@Disableduntil the helper is hardened.README
Completely rewritten to match the shipped code: Quick start with side-by-side Windows / Linux examples, Architecture diagram, full Usage walkthrough (attach / read-write / bulk / endianness / chains / AOB /
force()/ memprotect / concurrency-and-lifecycle), Platform notes for Linux and Windows, API cheat sheet, Limitations & caveats now down to just the two genuine constraints (macOS not yet supported; no anti-cheat / anti-debug).Breaking changes (consumers upgrading from 1.0.0)
System.exit(-1)paths → exceptions; wrapPointer.getBaseAddress/Memory.read*calls in try/catch onMem4JExceptionif you used to rely on the process exit.Pointer.add(int)is still there as an overload ofadd(long); source-compatible. Behaviour ofoffsetis now fulllongrange.Out of scope (deliberate)
Test plan
mvn -B teston Linux (root): 13 tests, 0 failures, 1 skipped (ptrace round-trip).mvn -B packageproduces main / sources / javadoc jars.VirtualProtectEx/AllocEx/FreeEx/QueryExnot yet exercised on real Windows.1.0.1/2.0.0) so JitPack rebuilds with the JDK 11 config in place.