Capabilities plugin breaks linux support for kernels without kernel_cap_struct

[ikelos]

We've had a report from a user that their linux plugins were failing with the following vebose output:

Level 8  volatility3.framework.automagic.stacker: Attempting to stack using LinuxIntelStacker
DEBUG    volatility3.framework.automagic.linux: Identified banner: b'Linux version 6.3.12-200.fc38.x86_64 (mockbuild@ad82227541134b7bb9538baac1587d1c) (gcc (GCC) 13.1.1 20230614 (Red Hat 13.1.1-4), GNU ld version 2.39-9.fc38) #1 SMP PREEMPT_DYNAMIC Thu Jul  6 04:05:18 UTC 2023\n\x00'
INFO     volatility3.schemas: Dependency for validation unavailable: jsonschema
DEBUG    volatility3.schemas: All validations will report success, even with malformed input
Level 7  volatility3.framework.automagic.stacker: Exception during stacking: Symbol type not in LintelStacker1 SymbolTable: kernel_cap_struct
Level 6  volatility3.framework.automagic.stacker: Traceback (most recent call last):
  File "/home/xatom32/Documenti/volatility3/volatility3/volatility3/framework/automagic/stacker.py", line 213, in stack_layer
    new_layer = stacker.stack(context, initial_layer, progress_callback)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/xatom32/Documenti/volatility3/volatility3/volatility3/framework/automagic/linux.py", line 72, in stack
    table = linux.LinuxKernelIntermedSymbols(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/xatom32/Documenti/volatility3/volatility3/volatility3/framework/symbols/linux/__init__.py", line 32, in __init__
    self.set_type_class("kernel_cap_struct", extensions.kernel_cap_struct)
  File "/home/xatom32/Documenti/volatility3/volatility3/volatility3/framework/symbols/intermed.py", line 60, in _delegate_function
    return getattr(self._delegate, name)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/xatom32/Documenti/volatility3/volatility3/volatility3/framework/symbols/intermed.py", line 425, in set_type_class
    raise ValueError(f"Symbol type not in {self.name} SymbolTable: {name}")
ValueError: Symbol type not in LintelStacker1 SymbolTable: kernel_cap_struct

This seems to be because the kernel_cap_struct isn't present in the symbol table. At the moment we're not sure whether the symbol table was made correctly, or if the symbol really isn't there. However, it might be prudent to allow for kernels that are too old or don't have the symbol, to continue loading by marking that symbol structures as optional. The line in question is here and could be changed to optional_set_type_class but would then require checking in the plugin that the structure exists and graceful error message if it doesn't.

@gcmoreira, any chance you could look into this please?

[ikelos]

The reporter on slack was treu if we need to follow up... 5:)

[gcmoreira]

Redhat kernel 6.3 that's interesting. That was exactly what I was looking for. Sure, leave it with me.

[gcmoreira]

The problem is that in 6.3 kernel_cap_struct became an anonymous struct with a kernel_cap_t typedef.

…

On Thu, 20 July 2023, 11:13 am ikelos, ***@***.***> wrote: The reporter on slack was treu if we need to follow up... 5:) — Reply to this email directly, view it on GitHub <#985 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACNRMVGXV54ONWWQJJV3VULXRDZFPANCNFSM6AAAAAA2RDSXX4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[gcmoreira]

Ok, I've installed Fedora 38 with 6.3.12-200.fc38.x86_64 and built the ISF.

As of kernel 6.3, there is no more kernel_cap_struct , it became an anonymous type.
Before kernels 6.3:

typedef struct kernel_cap_struct {
    __u32 cap[_KERNEL_CAPABILITY_U32S];
} kernel_cap_t;

After kernels 6.3:

typedef struct { u64 val; } kernel_cap_t;

Even if this type were added using optional_set_type_class(), when we do cap.get_capabilities() we will get:

AttributeError: StructType has no attribute: symbol_table_name1!unnamed_350bf69f5aefed8e.get_capabilities

... which makes sense.

I wonder why this is not already an issue with other types using the same style of declaration. I may be wrong but I don't think we are taking the typedef information from DWARF.

$ xzgrep "kernel_cap_t" linux-6.3.12-200.fc38.x86_64.json.xz | wc -l
0

In order to be able to work with typedef'ed anonymous types I think we should associate (or maybe just rename) the unnamed type internal name to the typedef name.
In the above example, kernel_cap_t = unnamed_350bf69f5aefed8e so that to fix this issue we can do something like this:

self.optional_set_type_class("kernel_cap_struct", extensions.kernel_cap_struct)  # For kernels < 6.3
self.optional_set_type_class("kernel_cap_t", extensions.kernel_cap_struct)  # For kernels >= 6.3

[gcmoreira]

@ikelos As soon as dwarf2json is updated with the changes mentioned above, I will test it again and create the pull request with the following changes:
develop...gcmoreira:volatility3:issue_985
I already talked with the dwarf2json maintainers, they are working on it.

[ikelos]

Hmmm, thanks for getting to the bottom of it and getting a solution for dwarf2json! We may also need to find a way to let plugins request a specific version of an ISF file, which might be trickier, but I'll start having a think about it (technically, they should all be versioned)... 5:S

[gcmoreira]

No worries. They already had a solution for it. But it's basically what I explained above.

[mkonshie]

I created a PR in dwarf2json to address this issue. @gcmoreira Can you take a look and confirm if this commit solves the issue?

[gcmoreira]

@mkonshie the fix in dwarf2json works for me. Tested using Fedora Core 38 with kernel 6.3.12 .... thanks

[mkonshie]

Thanks for confirming. The PR has been merged.

[gcmoreira]

Awesome, thanks @mkonshie .
@ikelos in #997 I've added the changes to fix this issue in vol3. Do you want to let the user know about it? so that he/she can also test it.
As the changes in dwarf2json are now merged into master. The user has to generate the ISF again using that code and then use the vol3 changes included in PR #997

[ikelos]

Hiya, I've let treu know on slack. There's still a bit of work about detecting when things may be broken from a previous version of dwarf2json, but that's not a high priority at the moment.