Complete rewrite of `pcustom` command #794

hugsy · 2022-01-18T01:54:10Z

Description/Motivation/Screenshots

This PR is a complete rewrite of the pcustom command (and all its subcommands). It creates and uses a new ExternalStructureLoader class that efficiently allows to manipulate custom ctypes.Structures located in gef.config["pcustom.struct_path"].

On top of being way more Pythonic, the great advantage of this approach is that now any command/function can use those new imported types, which is much more preferable to the old approach.

This is a first step towards what was described in #778

How Has This Been Tested?

Architecture	Yes/No	Comments
x86-32	✔️
x86-64	✔️
ARM	✔️
AARCH64	✔️
MIPS	✖️
POWERPC	✖️
SPARC	✖️
RISC-V	✖️
`make test`	✔️

Checklist

My PR was done against the dev branch, not master.
My code follows the code style of this project.
My change includes a change to the documentation, if required.
My change adds tests as appropriate.
I have read and agree to the CONTRIBUTING document.

…ther plugins (see #778)

Grazfather

Done first pass.

gef.py

- added cache

gef.py

Grazfather · 2022-01-19T15:51:18Z

gef.py

@@ -5467,8 +5475,9 @@ def do_invoke(self, argv: List) -> None:
        info(f"Listing custom structures from '{manager.path}'")
        struct_color = gef.config["pcustom.structure_type"]
        filename_color = gef.config["pcustom.structure_name"]
-        for module in manager.modules:
-            __modules = ", ".join([Color.colorify(structure.name, struct_color) for structure in module.structures])
+        for module_name in manager.modules:


for module in manager.modules.values():

~~not a dict~~ now changed to a subclass of dict

gef.py

Grazfather · 2022-01-19T15:57:00Z

gef.py

+            _class = getattr(module, self.name)
+            return _class
+
+        def apply_at(self, address: int, depth: int = 0, max_depth: int = -1) -> None:


But I mean that the check below: if depth >= max_depth: will never pass with the default max_depth, unless depth is negative, which doesn't make sense.

If you want it to be a mandatory arg, you can make it a normal arg, or if you want it a mandatory kwarg use the snippet I posted above.

gef.py

Grazfather · 2022-01-19T16:04:06Z

gef.py

+            for structure_name in module.structures:
+                yield module, module.structures[structure_name]


Suggested change

for structure_name in module.structures:

yield module, module.structures[structure_name]

for structure in module.structures.values():

yield module, structure

This one should work

gef.py

Co-authored-by: Grazfather <grazfather@gmail.com>

This reverts commit 3948328.

gef.py

…d iteration from it

theguy147

The code logic looks good. Just a few remarks about the function signatures...

Also from typing import Set is no longer used as a consequence of this PR and should therefore be removed.

gef.py

daniellimws

In the old version of pcustom, this error will show when the struct doesn't exist

gef➤  dt blah
[!] Invalid structure name 'blah'

Now it doesn't give any error when ran with a struct that doesn't exist.

gef➤  dt blah
[*] 'get_gef_setting' is deprecated and will be removed in a feature release. Use `gef.config[key]`
gef➤

daniellimws · 2022-01-24T16:46:58Z

gef.py

            gdb.execute("pcustom list")
            return

-        modname, structname = self.get_modulename_structname_from_arg(argv[0])
+        _, structname = self.explode_type(args.type)


Currently we don't provide any error message when the struct does not exist. It will be helpful to have one that tells the user how to define a struct.

Suggested change

_, structname = self.explode_type(args.type)

_, structname = self.explode_type(args.type)

manager = ExternalStructureManager()

struct_info = manager.find(structname)

if not struct_info:

path = pathlib.Path(gef.config["pcustom.struct_path"]).expanduser()

struct_color = gef.config["pcustom.structure_type"]

filename_color = gef.config["pcustom.structure_name"]

err(f"No struct {Color.colorify(structname, struct_color)} found. "

f"Use {Color.yellowify('pcustom edit')} or create a file in {Color.colorify(path, filename_color)} "

"to define its fields.")

return

Actually no, this should be done in ExternalStructureManger.find so that it applies for the other pcustom subcommands as well. Ignore this suggestion.

I think you were right the first time: the caller should handle the error (a-la str.find() )

Fixed as well, it is the caller's job to check the return value for ESM.find($structure_name) so now we have:

gef➤ pcustom [+] Listing custom structures from '/tmp/gef/structs' → /tmp/gef/structs/goo.py (foo_t, goo_t) gef➤ pcustom meh [!] No structure named 'goo' found

daniellimws · 2022-01-24T16:47:39Z

gef.py

+        address = parse_address(args.address)
+        result = manager.find(structname)
+        if result:
+            _, structure = result
+            structure.apply_at(address, self["max_depth"])


manager.find(structname) moved up as per suggestion above

Suggested change

address = parse_address(args.address)

result = manager.find(structname)

if result:

_, structure = result

structure.apply_at(address, self["max_depth"])

address = parse_address(args.address)

_, structure = struct_info

structure.apply_at(address, self["max_depth"])

return

@hugsy You only added the check to PCustomShowCommand, and there's no else branch here. So when running dt <invalid struct> <address>, there's still no error message.

gef.py

daniellimws

Sorry about the separate reviews, this is the last one.

That's all from me.

gef.py

Co-authored-by: theguy147 <37738506+theguy147@users.noreply.github.com>

gef.py

- passing a reference to `ExternalStructureLoader` to modules and structures for nested structure to work properly

Co-authored-by: Grazfather <grazfather@gmail.com>

daniellimws

All looks good. Just missing out writing the error message in PCustomCommand.

daniellimws · 2022-01-25T02:27:39Z

gef.py

+        address = parse_address(args.address)
+        result = manager.find(structname)
+        if result:
+            _, structure = result
+            structure.apply_at(address, self["max_depth"])


@hugsy You only added the check to PCustomShowCommand, and there's no else branch here. So when running dt <invalid struct> <address>, there's still no error message.

- added CI tests for error messages

…custom_command

hugsy · 2022-01-25T05:19:29Z

Another good catch, so I've also added some CI checks for those error messages.

Entirely rewrote the pcustom functionality to make it usable from o…

f8aecd7

…ther plugins (see #778)

hugsy added enhancement new feature code-quality labels Jan 18, 2022

hugsy added this to the Release: next milestone Jan 18, 2022

Grazfather reviewed Jan 18, 2022

View reviewed changes

- applied PR comments

07c979e

- added cache

Grazfather reviewed Jan 19, 2022

View reviewed changes

hugsy and others added 5 commits January 19, 2022 09:21

Apply suggestions from @Grazfather

3948328

Co-authored-by: Grazfather <grazfather@gmail.com>

Changed signature of apply_at

54044e3

Revert "Apply suggestions from @Grazfather"

bfb22b0

This reverts commit 3948328.

linting + formatting

eda147c

last fix

e5527ac

Grazfather reviewed Jan 19, 2022

View reviewed changes

gef.py Show resolved Hide resolved

hugsy added 2 commits January 19, 2022 14:03

making Modules and Module inherit dict so we can use getitem an…

1f67cc8

…d iteration from it

removed unused var in PCustomListCommand

085151f

Grazfather approved these changes Jan 19, 2022

View reviewed changes

Merge branch 'dev' into fix_pcustom_command

3f3c72d

hugsy requested review from daniellimws and theguy147 January 21, 2022 18:01

theguy147 requested changes Jan 24, 2022

View reviewed changes

gef.py Outdated Show resolved Hide resolved

gef.py Outdated Show resolved Hide resolved

gef.py Outdated Show resolved Hide resolved

gef.py Outdated Show resolved Hide resolved

gef.py Outdated Show resolved Hide resolved

daniellimws requested changes Jan 24, 2022

View reviewed changes

gef.py Outdated Show resolved Hide resolved

daniellimws requested changes Jan 24, 2022

View reviewed changes

daniellimws reviewed Jan 24, 2022

View reviewed changes

gef.py Show resolved Hide resolved

daniellimws requested changes Jan 24, 2022

View reviewed changes

gef.py Outdated Show resolved Hide resolved

Apply suggestions from code review

c253b86

Co-authored-by: theguy147 <37738506+theguy147@users.noreply.github.com>

Grazfather reviewed Jan 24, 2022

View reviewed changes

gef.py Outdated Show resolved Hide resolved

- added error message in pcustom show when not found

abd05e2

- passing a reference to `ExternalStructureLoader` to modules and structures for nested structure to work properly

hugsy added 2 commits January 24, 2022 14:01

- added pcustom list test

e08f87d

- added test for pcustom show with and without address

f4b1f37

hugsy requested review from daniellimws and theguy147 January 24, 2022 23:27

theguy147 approved these changes Jan 25, 2022

View reviewed changes

Update gef.py

b7afb6a

Co-authored-by: Grazfather <grazfather@gmail.com>

daniellimws requested changes Jan 25, 2022

View reviewed changes

hugsy added 2 commits January 24, 2022 21:18

- pcustom prints error message if struct name not found

9664a06

- added CI tests for error messages

Merge branch 'fix_pcustom_command' of github.com:hugsy/gef into fix_p…

d77e121

…custom_command

daniellimws approved these changes Jan 25, 2022

View reviewed changes

hugsy merged commit 779369f into dev Jan 25, 2022

hugsy deleted the fix_pcustom_command branch January 25, 2022 17:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Complete rewrite of `pcustom` command #794

Complete rewrite of `pcustom` command #794

hugsy commented Jan 18, 2022

Grazfather left a comment

Grazfather Jan 19, 2022

hugsy Jan 19, 2022 •

edited

Loading

Grazfather Jan 19, 2022

Grazfather Jan 19, 2022

Grazfather Jan 19, 2022

Grazfather Jan 19, 2022

theguy147 left a comment

daniellimws left a comment

daniellimws Jan 24, 2022

daniellimws Jan 24, 2022

hugsy Jan 24, 2022

hugsy Jan 24, 2022

daniellimws Jan 24, 2022

daniellimws Jan 25, 2022

daniellimws left a comment

daniellimws left a comment

daniellimws Jan 25, 2022

hugsy commented Jan 25, 2022

		for structure_name in module.structures:
		yield module, module.structures[structure_name]

-        _, structname = self.explode_type(args.type)
+        _, structname = self.explode_type(args.type)
+        manager = ExternalStructureManager()
+        struct_info = manager.find(structname)
+        if not struct_info:
+            path = pathlib.Path(gef.config["pcustom.struct_path"]).expanduser()
+            struct_color = gef.config["pcustom.structure_type"]
+            filename_color = gef.config["pcustom.structure_name"]
+            err(f"No struct {Color.colorify(structname, struct_color)} found. "
+                f"Use {Color.yellowify('pcustom edit')} or create a file in {Color.colorify(path, filename_color)} "
+                 "to define its fields.")
+            return

Complete rewrite of pcustom command #794

Complete rewrite of pcustom command #794

Conversation

hugsy commented Jan 18, 2022

Description/Motivation/Screenshots

How Has This Been Tested?

Checklist

Grazfather left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hugsy Jan 19, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

theguy147 left a comment

Choose a reason for hiding this comment

daniellimws left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

daniellimws left a comment

Choose a reason for hiding this comment

daniellimws left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hugsy commented Jan 25, 2022

Complete rewrite of `pcustom` command #794

Complete rewrite of `pcustom` command #794

hugsy Jan 19, 2022 •

edited

Loading