-
Notifications
You must be signed in to change notification settings - Fork 562
Description
Hey there,
so currently i'm facing problem in using Volatility 3 to analyse the ram dump file from MacOS Monterey 12.6 build 21G115, I had successfully created the symbol table for that OS version:
./dwarf2json mac --macho /Library/Developer/KDKs/KDK_12.6_21G115.kdk/System/Library/Kernels/kernel.dSYM/Contents/Resources/DWARF/kernel --macho-symbols /Library/Developer/KDKs/KDK_12.6_21G115.kdk/System/Library/Kernels/kernel > 12.6.json
After that I copied 12.6.json to /path_to_volatility3/symbols/mac/ directory.
ISFinfo shows:
python3 ./volatility3/vol.py isfinfo
file:///Users/test/volatility3/volatility3/symbols/mac/allmacho.json Unknown 19 0 64681 392 b'Darwin Kernel Version 21.6.0: Mon Aug 22 20:17:10 PDT 2022; root:xnu-8020.140.49~2/RELEASE_X86_64\x00'
Banners of image show:
python3 ./volatility3/vol.py -f raw_dump_only_osxpmem.dump banners
0x18d60273 Darwin Kernel Version 21.6.0: Mon Aug 22 20:17:10 PDT 2022; root:xnu-8020.140.49~2/RELEASE_X86_64
0x18d602d6 Darwin Kernel Version 21.6.0: Mon Aug 22 20:17:10 PDT 2022; root:xnu-8020.140.49~2/RELEASE_X86_64
0x19301fc2 Darwin Kernel Version 21.6.0: Mon Aug 22 20:17:10 PDT 2022; root:xnu-8020.140.49~2/RELEASE_X86_64
0x1f960273 Darwin Kernel Version 21.6.0: Mon Aug 22 20:17:10 PDT 2022; root:xnu-8020.140.49~2/RELEASE_X86_64
0x1f9602d6 Darwin Kernel Version 21.6.0: Mon Aug 22 20:17:10 PDT 2022; root:xnu-8020.140.49~2/RELEASE_X86_64
0x1ff01fc2 Darwin Kernel Version 21.6.0: Mon Aug 22 20:17:10 PDT 2022; root:xnu-8020.140.49~2/RELEASE_X86_64
0x47fae08e Darwin Kernel Version 21.6.0: Mon Aug 22 20:17:10 PDT 2022; root:xnu-8020.140.49~2/RELEASE_X86_64
0x124601008 Darwin Kernel Version 21.6.0: Mon Aug 22 20:17:10 PDT 2022; root:xnu-8020.140.49~2/RELEASE_X86_64
0x12548e2b9 Darwin Kernel Version 21.6.0: Mon Aug 22 20:17:10 PDT 2022; root:xnu-8020.140.49~2/RELEASE_X86_64
0x12548e31c Darwin Kernel Version 21.6.0: Mon Aug 22 20:17:10 PDT 2022; root:xnu-8020.140.49~2/RELEASE_X86_64
0x1f5db09c6 Darwin Kernel Version 21.6.0: Mon Aug 22 20:17:10 PDT 2022; root:xnu-8020.140.49~2/RELEASE_X86_64
0x30cd8be4e Darwin Kernel Version 21.6.0: Mon Aug 22 20:17:10 PDT 2022; root:xnu-8020.140.49~2/RELEASE_X86_64
0x3f0865046 Darwin Kernel Version 21.6.0: Mon Aug 22 20:17:10 PDT 2022; root:xnu-8020.140.49~2/RELEASE_X86_64
0x3f2ee408e Darwin Kernel Version 21.6.0: Mon Aug 22 20:17:10 PDT 2022; root:xnu-8020.140.49~2/RELEASE_X86_64
0x3f3ae38c6 Darwin Kernel Version 21.6.0: Mon Aug 22 20:17:10 PDT 2022; root:xnu-8020.140.49~2/RELEASE_X86_64
0x3f4af0d57 Darwin Kernel Version 21.6.0: Mon Aug 22 20:17:10 PDT 2022; root:xnu-8020.140.49~2/RELEASE_X86_64
0x400e3f88e Darwin Kernel Version 21.6.0: Mon Aug 22 20:17:10 PDT 2022; root:xnu-8020.140.49~2/RELEASE_X86_64
It looks like ok, but any operations with image (mac.bash/mac.lsof/mac.pstree and etc.) don't work:
python3 ./volatility3/vol.py -vvvvvv -f raw_dump_only_osxpmem.dump mac.bash
Volatility 3 Framework 2.3.0
INFO volatility3.cli: Volatility plugins path: ['/Users/test/volatility3/volatility3/plugins', '/Users/test/volatility3/volatility3/framework/plugins']
INFO volatility3.cli: Volatility symbols path: ['/Users/test/volatility3/volatility3/symbols', '/Users/test/volatility3/volatility3/framework/symbols']
Level 6 volatility3.framework: Importing from the following paths: /Users/test/volatility3/volatility3/plugins, /Users/test/volatility3/volatility3/framework/plugins
INFO volatility3.plugins.yarascan: Python Yara (>3.8.0) module not found, plugin (and dependent plugins) not available
DEBUG volatility3.framework: No module named 'yara'
DEBUG volatility3.framework: Failed to import module volatility3.plugins.yarascan based on file: /Users/test/volatility3/volatility3/framework/plugins/yarascan.py
INFO volatility3.plugins.yarascan: Python Yara (>3.8.0) module not found, plugin (and dependent plugins) not available
DEBUG volatility3.framework: No module named 'yara'
DEBUG volatility3.framework: Failed to import module volatility3.plugins.windows.svcscan based on file: /Users/test/volatility3/volatility3/framework/plugins/windows/svcscan.py
DEBUG volatility3.framework: No module named 'Crypto'
DEBUG volatility3.framework: Failed to import module volatility3.plugins.windows.hashdump based on file: /Users/test/volatility3/volatility3/framework/plugins/windows/hashdump.py
DEBUG volatility3.framework: No module named 'pefile'
DEBUG volatility3.framework: Failed to import module volatility3.plugins.windows.skeleton_key_check based on file: /Users/test/volatility3/volatility3/framework/plugins/windows/skeleton_key_check.py
DEBUG volatility3.framework: No module named 'Crypto'
DEBUG volatility3.framework: Failed to import module volatility3.plugins.windows.cachedump based on file: /Users/test/volatility3/volatility3/framework/plugins/windows/cachedump.py
INFO volatility3.plugins.yarascan: Python Yara (>3.8.0) module not found, plugin (and dependent plugins) not available
DEBUG volatility3.framework: No module named 'yara'
DEBUG volatility3.framework: Failed to import module volatility3.plugins.windows.vadyarascan based on file: /Users/test/volatility3/volatility3/framework/plugins/windows/vadyarascan.py
INFO volatility3.plugins.yarascan: Python Yara (>3.8.0) module not found, plugin (and dependent plugins) not available
DEBUG volatility3.framework: No module named 'yara'
DEBUG volatility3.framework: Failed to import module volatility3.plugins.windows.mftscan based on file: /Users/test/volatility3/volatility3/framework/plugins/windows/mftscan.py
INFO volatility3.plugins.windows.verinfo: Python pefile module not found, plugin (and dependent plugins) not available
DEBUG volatility3.framework: No module named 'pefile'
DEBUG volatility3.framework: Failed to import module volatility3.plugins.windows.netscan based on file: /Users/test/volatility3/volatility3/framework/plugins/windows/netscan.py
INFO volatility3.plugins.windows.verinfo: Python pefile module not found, plugin (and dependent plugins) not available
DEBUG volatility3.framework: No module named 'pefile'
DEBUG volatility3.framework: Failed to import module volatility3.plugins.windows.netstat based on file: /Users/test/volatility3/volatility3/framework/plugins/windows/netstat.py
DEBUG volatility3.framework: No module named 'Crypto'
DEBUG volatility3.framework: Failed to import module volatility3.plugins.windows.lsadump based on file: /Users/test/volatility3/volatility3/framework/plugins/windows/lsadump.py
INFO volatility3.plugins.windows.verinfo: Python pefile module not found, plugin (and dependent plugins) not available
DEBUG volatility3.framework: No module named 'pefile'
DEBUG volatility3.framework: Failed to import module volatility3.plugins.windows.verinfo based on file: /Users/test/volatility3/volatility3/framework/plugins/windows/verinfo.py
INFO volatility3.cli: The following plugins could not be loaded (use -vv to see why): volatility3.plugins.windows.cachedump, volatility3.plugins.windows.hashdump, volatility3.plugins.windows.lsadump, volatility3.plugins.windows.mftscan, volatility3.plugins.windows.netscan, volatility3.plugins.windows.netstat, volatility3.plugins.windows.skeleton_key_check, volatility3.plugins.windows.svcscan, volatility3.plugins.windows.vadyarascan, volatility3.plugins.windows.verinfo, volatility3.plugins.yarascan
Level 6 volatility3.framework: Importing from the following paths: /Users/test/volatility3/volatility3/framework/automagic
Level 7 volatility3.cli: Cache directory used: /Users/test/.cache/volatility3
INFO volatility3.framework.automagic: Detected a mac category plugin
Level 6 volatility3.framework: Importing from the following paths: /Users/test/volatility3/volatility3/framework/layers
INFO volatility3.framework.automagic: Running automagic: ConstructionMagic
Level 6 volatility3.framework: Importing from the following paths: /Users/test/volatility3/volatility3/framework/layers
Level 9 volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Bash.kernel.layer_name
Level 9 volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Bash.kernel.symbol_table_name
Level 6 volatility3.framework: Importing from the following paths: /Users/test/volatility3/volatility3/framework/layers
Level 9 volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Bash.kernel.layer_name
Level 9 volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Bash.kernel.symbol_table_name
Level 6 volatility3.framework: Importing from the following paths: /Users/test/volatility3/volatility3/framework/layers
Level 9 volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Bash.kernel
Level 6 volatility3.framework: Importing from the following paths: /Users/test/volatility3/volatility3/framework/layers
Level 9 volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Bash.kernel.layer_name
Level 6 volatility3.framework: Importing from the following paths: /Users/test/volatility3/volatility3/framework/layers
Level 9 volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Bash.kernel.layer_name
Level 9 volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Bash.kernel.layer_name
Level 9 volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Bash.kernel
Level 6 volatility3.framework: Importing from the following paths: /Users/test/volatility3/volatility3/framework/layers
Level 9 volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Bash.kernel.symbol_table_name
Level 6 volatility3.framework: Importing from the following paths: /Users/test/volatility3/volatility3/framework/layers
Level 9 volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Bash.kernel.symbol_table_name
Level 9 volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Bash.kernel.symbol_table_name
Level 9 volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Bash.kernel
Level 9 volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Bash.kernel.layer_name
Level 9 volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Bash.kernel.symbol_table_name
Level 9 volatility3.framework.automagic.construct_layers: Failed on requirement: plugins.Bash
Level 6 volatility3.framework: Importing from the following paths: /Users/test/volatility3/volatility3/framework/layers
Level 6 volatility3.framework: Importing from the following paths: /Users/test/volatility3/volatility3/framework/layers
Level 6 volatility3.framework.automagic.construct_layers: Construction Exception occurred: Unexpected config value found: None
INFO volatility3.framework.automagic: Running automagic: SymbolCacheMagic
Level 6 volatility3.framework.symbols.intermed: Searching for symbols in /Users/test/volatility3/volatility3/symbols, /Users/test/volatility3/volatility3/framework/symbols
INFO volatility3.framework.automagic: Running automagic: LayerStacker
Level 6 volatility3.framework: Importing from the following paths: /Users/test/volatility3/volatility3/framework/layers
Level 9 volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Bash.kernel.layer_name
Level 9 volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Bash.kernel.symbol_table_name
Level 7 volatility3.framework.layers.resources: Available URL handlers: HTTPErrorProcessor, HTTPDefaultErrorHandler, HTTPRedirectHandler, ProxyHandler, HTTPBasicAuthHandler, ProxyBasicAuthHandler, HTTPDigestAuthHandler, ProxyDigestAuthHandler, AbstractHTTPHandler, HTTPHandler, HTTPSHandler, HTTPCookieProcessor, UnknownHandler, FileHandler, FTPHandler, CacheFTPHandler, DataHandler, VolatilityHandler, JarHandler, OfflineHandler
Level 8 volatility3.framework.automagic.stacker: Attempting to stack using AVMLStacker
Level 8 volatility3.framework.automagic.stacker: Attempting to stack using LimeStacker
Level 8 volatility3.framework.automagic.stacker: Attempting to stack using Elf64Stacker
Level 6 volatility3.framework.layers.elf: Exception: Bad magic 0x4034b50 at file offset 0x0
Level 8 volatility3.framework.automagic.stacker: Attempting to stack using QemuStacker
Level 8 volatility3.framework.automagic.stacker: Attempting to stack using WindowsCrashDumpStacker
Level 8 volatility3.framework.automagic.stacker: Attempting to stack using VmwareStacker
Level 8 volatility3.framework.automagic.stacker: Attempting to stack using MacIntelStacker
DEBUG volatility3.framework.automagic.mac: Identified banner: b'Darwin Kernel Version 21.6.0: Mon Aug 22 20:17:10 PDT 2022; root:xnu-8020.140.49~2/RELEASE_X86_64\x00'
INFO volatility3.schemas: Dependency for validation unavailable: jsonschema
DEBUG volatility3.schemas: All validations will report success, even with malformed input
Level 7 volatility3.framework.automagic.stacker: Exception during stacking: Symbol type not in MacintelStacker1 SymbolTable: vm_map_object
Level 6 volatility3.framework.automagic.stacker: Traceback (most recent call last):
File "/Users/test/volatility3/volatility3/framework/automagic/stacker.py", line 171, in stack_layer
new_layer = stacker.stack(context, initial_layer, progress_callback)
File "/Users/test/volatility3/volatility3/framework/automagic/mac.py", line 61, in stack
table = mac.MacKernelIntermedSymbols(context = context,
File "/Users/test/volatility3/volatility3/framework/symbols/mac/__init__.py", line 21, in __init__
self.set_type_class('vm_map_object', extensions.vm_map_object)
File "/Users/test/volatility3/volatility3/framework/symbols/intermed.py", line 54, in _delegate_function
return getattr(self._delegate, name)(*args, **kwargs)
File "/Users/test/volatility3/volatility3/framework/symbols/intermed.py", line 362, in set_type_class
raise ValueError(f"Symbol type not in {self.name} SymbolTable: {name}")
ValueError: Symbol type not in MacintelStacker1 SymbolTable: vm_map_object
Level 9 volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Bash.kernel.layer_name
Level 9 volatility3.framework.configuration.requirements: TypeError - Layer is not the required Architecture: FileLayer
DEBUG volatility3.framework.automagic.stacker: Stacked layers: ['FileLayer']
INFO volatility3.framework.automagic: Running automagic: SymbolFinder
INFO volatility3.framework.automagic: Running automagic: MacSymbolFinder
Level 9 volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Bash.kernel.symbol_table_name
INFO volatility3.framework.automagic: Running automagic: KernelModule
Level 9 volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Bash.kernel.layer_name
Level 9 volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Bash.kernel.symbol_table_name
Level 9 volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Bash.kernel.layer_name
Level 9 volatility3.framework.configuration.requirements: IndexError - No configuration provided: plugins.Bash.kernel.layer_name
Level 9 volatility3.framework.configuration.requirements: Symbol table requirement not yet fulfilled: plugins.Bash.kernel.symbol_table_name
Unsatisfied requirement plugins.Bash.kernel.layer_name:
Unsatisfied requirement plugins.Bash.kernel.symbol_table_name:
A translation layer requirement was not fulfilled. Please verify that:
A file was provided to create this layer (by -f, --single-location or by config)
The file exists and is readable
The file is a valid memory image and was acquired cleanly
A symbol table requirement was not fulfilled. Please verify that:
The associated translation layer requirement was fulfilled
You have the correct symbol file for the requirement
The symbol file is under the correct directory or zip file
The symbol file is named appropriately or contains the correct banner
Unable to validate the plugin requirements: ['plugins.Bash.kernel.layer_name', 'plugins.Bash.kernel.symbol_table_name']
Activity
ikelos commentedon Oct 26, 2022
So it did identify the correct banner, but we then immediately try to use a structure that wasn't present:
It's not clear if this structure's name has changed, or it was removed (or possibly if the symbol table was generated incorrectly, but it doesn't look like it). We'll need to do some investigation to figure out what the problem is. The
vm_map_object
type is one that we override with a custom handler, and it appears the custom handler isn't finding the original definition in the JSON. I've asked @atcuno to see whether thevm_map_object
structure was renamed or removed from recent mac kernels...github-actions commentedon Aug 18, 2023
This issue is stale because it has been open for 200 days with no activity.
ikelos commentedon Aug 19, 2023
Ping @atcuno, before this times out in a couple months, could you please check about the
vm_map_object
in the mac symbol tables?github-actions commentedon Mar 8, 2024
This issue is stale because it has been open for 200 days with no activity.
ikelos commentedon Mar 12, 2024
@atcuno The stale ticket just got added, which means it's been a couple of months since I asked. Have you had a chance to check out what's going on with the symbol tables and the
vm_map_object
symbol?Abyss-W4tcher commentedon Mar 18, 2024
This structure was removed from the kernel, as well as many related
vm_map
ones.The old/new versions are here :
Here is an article brieflly talking about it :
I was planning to update the framework, but It's gonna need more time and analysis to fix it. It mostly impacts
mac.malfind
.ikelos commentedon Mar 20, 2024
@Abyss-W4tcher thanks for the analysis! Perhaps @atcuno or @gcmoreira can help out now we know what it is?
Abyss-W4tcher commentedon Mar 20, 2024
Hi, I will propose a patch in a PR soon, it's only in my fork right now.
I inform any dev here, to avoid potentially duplicating the same work 😃
github-actions commentedon Oct 7, 2024
This issue is stale because it has been open for 200 days with no activity.
ikelos commentedon Oct 7, 2024
Definitely not stale, just tricky to fix... 5:S Sorry I've been so slow on it, I'm finding it hard to get dedicated time to fix big issues like this. It is still on my list though.
Abyss-W4tcher commentedon Oct 7, 2024
Hi, this should have been fixed in the two macOS PRs that were released a few months ago. However due to the current calendar I completely understand the lack of time to allocate on this subject 👍.
ikelos commentedon Oct 7, 2024
Yeah, I thought I was producing a MacSymbolTable that was a facade for the two different mac tables and gave you the right table (with the right shift) based on which symbol you asked for? Did we still need that, I don't remember how we left it sadly... 5:S
Abyss-W4tcher commentedon Oct 7, 2024
When you are ready, just comment on any of the PRs and I will provide you with a quick resume.
But basically, this issue is related to malfind (fixed by the "plugins" PR) and the "double module" aspect you mentioned relates to the automagic update PR supporting a new self-contained MACHO kernel in macOS.
ikelos commentedon Oct 7, 2024
Ok thanks, as I say it hasn't slipped off my list but it needs me to find some time to sit and concentrate on it. Thanks for your understanding, it's really appreciated!