Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Heap bin tcache might be showing the wrong size for bins #641

Closed
daniellimws opened this issue Apr 18, 2021 · 10 comments
Closed

Heap bin tcache might be showing the wrong size for bins #641

daniellimws opened this issue Apr 18, 2021 · 10 comments

Comments

@daniellimws
Copy link
Collaborator

daniellimws commented Apr 18, 2021

Problem

As mentioned in #639 (comment), the chunk size of tcachebins do not seem to match what is observed in the glibc source code, so the displayed chunk size of the command output is different from the actual chunk size.

Steps to reproduce

Run heap bins tcache all on the following test cases.

Minimalist test case

Either heap-non-main or heap-tcache from our test suite.

Observed vs Expected Results

image

As seen in the screenshot above, tcachebin with idx=1 displays a chunk size of 0x18 but the actual chunk size of the residing chunks is 0x20. Same goes for the other bins.

Update:

At the point of 83b4934, this issue is fixed with 64-bit binaries, but not on 32-bit yet. Refer to #641 (comment)

(Also copied here for easy reference)

The chunks report that they have size 0x20, but the tcachebin says it stores chunks of size 0x18.

@hugsy
Copy link
Owner

hugsy commented Oct 22, 2021

@daniellimws I think this bug is fixed now (there was another in 32b too), can you retry?

@daniellimws
Copy link
Collaborator Author

Ok I'll test it again.

@irontigran
Copy link
Contributor

irontigran commented Oct 23, 2021

As far as I can tell, this is still a problem for 32-bit binaries on current dev (83b4934).

cd tests/binaries
make TARGET=i686 all
gdb -ex 'run' -ex 'heap bins tcache all' /tmp/heap-tcache.out

gives the following output:

32bit-tcache-size-mismatch

Where the chunks report that they have size 0x20, but the tcachebin says it stores chunks of size 0x18.

@daniellimws
Copy link
Collaborator Author

daniellimws commented Oct 25, 2021

Updated the issue description to reflect this ☝️
At least it works on 64-bit (just tested) 😅

@Grazfather
Copy link
Collaborator

Are you able to test on 32 bit again?

@stale
Copy link

stale bot commented Jan 25, 2022

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. You can reopen it by adding a comment to this issue.

@stale stale bot added the stale label Jan 25, 2022
@hugsy
Copy link
Owner

hugsy commented Jan 27, 2022

I could confirm the bug still exists as of 58f2a2e
We'll address this after the release is out. Also I'm flagging it for regression so we add a regression test for it. Better unit tests will do here, currently way too lax.

hugsy pushed a commit that referenced this issue Jan 28, 2022
- fixed calculation of tcache in gef
- added tests for `gef.heap`
- improved tests for `heap` command
- fixes #641
@hugsy
Copy link
Owner

hugsy commented Jan 28, 2022

Should be fixed by 2f3f5af (branch better_tests). The proper way of calculating the offset (tidx2size and csize2tidx) was added with proper CI tests. @daniellimws can you take a look?

@daniellimws
Copy link
Collaborator Author

Ok I'll take a look at it later

hugsy pushed a commit that referenced this issue Jan 30, 2022
- fixed calculation of tcache in gef
- added tests for `gef.heap`
- improved tests for `heap` command
- fixes #641
hugsy pushed a commit that referenced this issue Jan 30, 2022
- fixed calculation of tcache in gef
- added tests for `gef.heap`
- improved tests for `heap` command
- fixes #641
hugsy pushed a commit that referenced this issue Jan 30, 2022
- fixed calculation of tcache in gef
- added tests for `gef.heap`
- improved tests for `heap` command
- fixes #641
@daniellimws
Copy link
Collaborator Author

Yep it now shows the correct sizes for both 32-bit and 64-bit. We can close this issue now.

hugsy pushed a commit that referenced this issue Jan 31, 2022
- fixed calculation of tcache in gef
- added tests for `gef.heap`
- improved tests for `heap` command
- fixes #641
@hugsy hugsy closed this as completed Jan 31, 2022
hugsy added a commit that referenced this issue Feb 14, 2022
* first shot at refactoring ci testing

* - Added skeleton for most missing command test modules
- Fixed a logic error in `NopCommand`

* [ci] added tests for `nop`

* [ci] extra test for nop for memory check

* added benchmarking capability, can be triggered directly from `pytest`

* - fixed `pcustom` command test for 32b
- fixed calculation of tcache in gef
- added tests for `gef.heap`
- improved tests for `heap` command
- fixes #641

* [ci] cmd/heap - adjusted tcachebins indexes for 32b

* damnit

* - fixed linting
- fixed test `highlight` for 32b
- fixed test `pattern_search` for x86

* last fixes for tonight

* fixed `pattern` and `heap` tests for good

* - add 3rd party module check for `capstone`, `keystone`, `unicorn` and `ropper`

* added `test_func_update_gef`

* `make test` doesn't execute benchmark

* - fixed errors in the `pie` subcommands
- added tests for `pie`

* `theme` added more tests

* - improved tests for `pattern` and `edit-flags`

* [ci] created cases for all arches for bin tests in `tests/heap.py`

* fixed `heap` tests for good

* - added ci test for `glibcarena`
- fixed `theme` missing comma (original PR #808 by @mrshu )
- added missing values for `theme`

* - added tests for deprecated API
- added test for smart eval
- more function tests

* started `gef` test module

* - added tests for `syscall-args` and `is-syscall`

* - fixed `syscall-args` to also get catchpoints + tests

* - test `show_last_exception`

* make sure `syscall-args` test collects the ABI files from `gef-extras`

* linting

* only enable `syscall-args` test for x86

* `syscall-args` fixed typo in i686 test

* Fix RISCV arch detection (#790)

* Add RISCV alias so arch can be determined by ELF
* Add ptrsize property to RISCV arch
* Allow riscv tests to run

* Update tests/api/gef_arch.py

Co-authored-by: Grazfather <grazfather@gmail.com>

* fix: make shebang lines portable (#814)

* fix: make shebang lines portable

* fix: SC2006, SC2086, SC2016, SC2059

* make `heap` tests work universally

* disabling capstone/keystone/unicorn for some arches for now

* - fixed tests for ppc64
- added static `ptrsize` for ppc & ppc64 in gef

* - `BIN_LS` -> `_target("default")`
- removed auto demangle for now

* - disable pytest `--pdb` from makefile
- added doc for testing

* Apply suggestions from code review

Co-authored-by: Grazfather <grazfather@gmail.com>

* fixing ci

* [tests] use camel case for format string helper test class

* [tests] added docstring to `GefFuncDeprecatedApi`

* [tests] `edit-flags` are only for known arches for now

* PR review changes

* PR review last batch

Co-authored-by: Grazfather <grazfather@gmail.com>
Co-authored-by: theguy147 <37738506+theguy147@users.noreply.github.com>
hugsy pushed a commit that referenced this issue Jun 26, 2022
commit 430d9d3
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Sun Jun 26 09:53:07 2022 -0700

    Allow new file format (#849)

    * New abstraction class for fileformat, but defaults to GEF. This allows PE/MachO fileformat to be supported via gef-extras

    * 🐛 fixes
    - moved prompt init after gef initial loading
    - added a `__str__` for `FileFormat`
    - `FileFormat` also requires a section member

    * [CI] Fixed missing import

commit d594eb3
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Sun Jun 26 09:43:05 2022 -0700

    Fix 850 (#851)

    * Make sure GefSetting `on_write` hooks are called even on creation

    * Removed `SettingHookType`, not useful

    * in `GefSetting` replaced  `dict` with `super`

    * Added loading counter

commit a2a4bfa
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Sun Jun 26 09:42:25 2022 -0700

    Minor update to unittest of `print-format` after #843 (#852)

commit 9834252
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Sat Jun 25 14:08:13 2022 -0700

    Better `gef-remote` - part 2: Remote Qemu (#846)

    * - rewrite of `gef-remote` to properly manager remote session
    - removed unused functions (& tests)

    * fixes gef's got function fails in remote debug sessions #806

    * fully restore `qemu-user` + `test`

    * added more `__str__` for clarity

    * better __str__

    * better __str__

    * better __str__

    * last changes to `__str__`

    * add `qemu_user` support

    * stupid typo

    * stupid typo

    * qemu-system works too

    * [gef-remote] Updated the docs

    * Apply suggestions from code review

    * Update docs/commands/gef-remote.md

commit 546f4b1
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Sat Jun 25 09:11:28 2022 -0700

    Better `gef-remote` : rewrite and add `remote` full support (#830)

    * - rewrite of `gef-remote` to properly manager remote session

    * removed unused functions (& tests)

    * fixes gef's got function fails in remote debug sessions #806

    * fully restore `qemu-user` + `test`

    * added more `__str__` for clarity

    * better `__str__`  for more classes, making them more meaningful

commit 343cd23
Author: Dreg <dreg@fr33project.org>
Date:   Fri Jun 24 15:40:12 2022 +0200

    Fix typo in patch byte help (#848)

    `WORD` -> `BYTE`

commit 1e8f55f
Author: Dreg <dreg@fr33project.org>
Date:   Fri Jun 24 06:26:20 2022 +0200

    `print-format` and `patch byte` commands can work together (#843)

commit f890579
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Mon Jun 20 12:51:28 2022 -0700

    `master` -> `main` (#845)

commit 0fcd20a
Author: Dreg <dreg@fr33project.org>
Date:   Mon Jun 20 00:37:15 2022 +0200

    keep x86 and x86_64 FLAGS Register when calls to mprotect. Added pushfd, popfd, pushfq, popfq instructions (#844)

commit bf959a3
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Sun Jun 19 14:18:15 2022 -0700

    Type hinting fixes - part 2 (#839)

    * make `@deprecated` more informative on what must be changed

    * - `gef.binary` is being set only from the `newobjfile` handler
    - removed useless `get_elf_headers`
    - more pythonic gef.doc

    * simplified command/function registration

    * fixed `functions` command

    * [ci] roper was never executing correctly

    * Don't delete the `gef` object on `exit_handler` so we can still use it to get info even when the current debugging has exited

    * more type hinting fixes, now down to 75

    * - make all tests run in a tempdir (in tmp)
    - fixed gef install to use the tempdir config setting, not gef-extras dir

    * Formatting issues

    * add the possibility to specify read/write callbacks to `GefSetting`

    * [gef] updated unicode code strings to glyphs

    * allow directories inside `extra_plugin_dir` to act as python package

    * command examples can be provided as a list of strings

    * Allowing context disassembler to be monkey-patched

commit dbf2372
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Sat Jun 18 11:40:58 2022 -0700

    Remove `ida-interact` (#842) from GEF, entirely moved to GEF-Extras

    * removed all last parts of `ida-interact`, everything is moved to `gef-extras`

    * [docs] added a page to track command/function deprecation

commit 1c295fc
Author: hugsy <hugsy@blah.cat>
Date:   Fri Jun 17 09:40:45 2022 -0700

    restoring temporarily rtfd for redirection to the new docs

commit a78915d
Author: hugsy <hugsy@blah.cat>
Date:   Fri Jun 17 07:42:14 2022 -0700

    remove rtfd cfg file

commit f375efc
Merge: d463f7d 8ee281d
Author: hugsy <hugsy@blah.cat>
Date:   Thu Jun 16 14:19:09 2022 -0700

    Merge branch 'dev' of github.com:hugsy/gef into dev

commit 8ee281d
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Thu Jun 16 09:30:52 2022 -0700

    Update README.md

commit d463f7d
Author: hugsy <hugsy@blah.cat>
Date:   Tue Jun 14 11:02:06 2022 -0700

    [docs] hardcoded emojis instead of md

commit 37ddf86
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Tue Jun 14 10:29:20 2022 -0700

    Updated `gef-extras.sh` installation script

    Force script to pip-install the requirements of `gef-extras`

commit c671bed
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Tue Jun 14 10:10:09 2022 -0700

    Update install.md

commit ca67ced
Author: hugsy <hugsy@blah.cat>
Date:   Mon Jun 13 20:03:33 2022 -0700

    :bug: Last doc fixes, GEF API is now generated from Actions

commit 5b468d0
Author: hugsy <hugsy@blah.cat>
Date:   Mon Jun 13 19:16:39 2022 -0700

    :bug: forcing docs to run the same version than gdb

commit d836054
Author: hugsy <hugsy@blah.cat>
Date:   Mon Jun 13 19:07:36 2022 -0700

    Squashed commit of the following:

        [docs] make actions regenerate the api file

        [docs] regenerated gef api file

        [docs] cleanup

commit a7d8fa5
Author: hugsy <hugsy@blah.cat>
Date:   Mon Jun 13 14:50:35 2022 -0700

    Updated documentation appearance

commit dca3cb1
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Mon Jun 13 13:46:35 2022 -0700

    Update generate-docs.yml

commit d7d64a2
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Mon Jun 13 13:43:21 2022 -0700

    Update generate-docs.yml

commit a89f91c
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Mon Jun 13 13:37:37 2022 -0700

    Move 3rd party to extras (#841)

    * moved keystone, capstone, unicorn and ropper to gef-extras

    * moved test files to extras

    * moved docs

    * [docs] removed references to 3rd party packages

    * [actions] mlc -> lychee

    * updated PR & contrib templates and fixed actions command for lychee

    * moved requirements.txt to tests/ so people don't think there's a need to install any package

    * no need for readthedocs anymore

    * replaced references to rtfd to gh-pages

commit 7f45550
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Sun May 22 08:14:12 2022 -0700

    Type Hinting Fixes - Part 1 (#827)

    * better linting, by creating a proper .editorconfig & pylintrc
    * major drop of linting errors, from 786 errors to 145
    * using Py3.6 `__init_subclass__` allows to remove the use of `abc` and `@register_architecture`
    * using `__init_subclass__` to create base class for commands
    * using `__init_subclass__` to create base class for functions

commit e50af77
Author: Khaotic <6080590+khaoticdude@users.noreply.github.com>
Date:   Sat Apr 9 10:36:53 2022 -0500

    Improve identifying Stack Canaries (#833)

commit d540abd
Author: hugsy <hugsy@blah.cat>
Date:   Mon Mar 7 10:52:17 2022 -0800

    pushing urgent fix for #831
    (even tho `get_arch` will be deprecated after #827)

commit 93f3010
Merge: 7817bea 6e3cd5c
Author: hugsy <hugsy@blah.cat>
Date:   Fri Mar 4 09:35:13 2022 -0800

    Merge branch 'dev' of github.com:hugsy/gef into dev

commit 7817bea
Author: hugsy <hugsy@blah.cat>
Date:   Fri Mar 4 09:35:06 2022 -0800

    [docs] http -> https
    because apparently it's a big deal for debugging stuff

commit 6e3cd5c
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Sat Feb 26 12:29:58 2022 -0800

    Update gef.md

commit 0954a16
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Sat Feb 26 12:25:53 2022 -0800

    New subcommand `gef install` (#825)

    * Adding new command `gef install`

    * added doc

    * added tests

    * test: changing `remote` with `skel` because `remote` uses an external dependency (`rpyc`)

    * PR review additions

commit 18c40b6
Author: Boris-Chengbiao Zhou <bobo1239@web.de>
Date:   Tue Feb 22 05:18:33 2022 +0100

     Add mechanism to let architectures specify whether they support a gdb arch (#822)

    * Fix gdb arch parsing for auto-detected archs

    * Add mechanism to let architectures specify whether they support a gdb arch

    * Add documentation about adding architectures and supports_gdb_arch()

    * Address review comments

commit 2b7f315
Author: Ebubekir Türker <35612408+ebubekirtrkr@users.noreply.github.com>
Date:   Mon Feb 14 23:30:43 2022 +0300

    remove unnecessary warnings (#824)

    * remove unnecessary warnings

commit 18c7ba4
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Sun Feb 13 18:38:44 2022 -0800

    Better tests (#813)

    * first shot at refactoring ci testing

    * - Added skeleton for most missing command test modules
    - Fixed a logic error in `NopCommand`

    * [ci] added tests for `nop`

    * [ci] extra test for nop for memory check

    * added benchmarking capability, can be triggered directly from `pytest`

    * - fixed `pcustom` command test for 32b
    - fixed calculation of tcache in gef
    - added tests for `gef.heap`
    - improved tests for `heap` command
    - fixes #641

    * [ci] cmd/heap - adjusted tcachebins indexes for 32b

    * damnit

    * - fixed linting
    - fixed test `highlight` for 32b
    - fixed test `pattern_search` for x86

    * last fixes for tonight

    * fixed `pattern` and `heap` tests for good

    * - add 3rd party module check for `capstone`, `keystone`, `unicorn` and `ropper`

    * added `test_func_update_gef`

    * `make test` doesn't execute benchmark

    * - fixed errors in the `pie` subcommands
    - added tests for `pie`

    * `theme` added more tests

    * - improved tests for `pattern` and `edit-flags`

    * [ci] created cases for all arches for bin tests in `tests/heap.py`

    * fixed `heap` tests for good

    * - added ci test for `glibcarena`
    - fixed `theme` missing comma (original PR #808 by @mrshu )
    - added missing values for `theme`

    * - added tests for deprecated API
    - added test for smart eval
    - more function tests

    * started `gef` test module

    * - added tests for `syscall-args` and `is-syscall`

    * - fixed `syscall-args` to also get catchpoints + tests

    * - test `show_last_exception`

    * make sure `syscall-args` test collects the ABI files from `gef-extras`

    * linting

    * only enable `syscall-args` test for x86

    * `syscall-args` fixed typo in i686 test

    * Fix RISCV arch detection (#790)

    * Add RISCV alias so arch can be determined by ELF
    * Add ptrsize property to RISCV arch
    * Allow riscv tests to run

    * Update tests/api/gef_arch.py

    Co-authored-by: Grazfather <grazfather@gmail.com>

    * fix: make shebang lines portable (#814)

    * fix: make shebang lines portable

    * fix: SC2006, SC2086, SC2016, SC2059

    * make `heap` tests work universally

    * disabling capstone/keystone/unicorn for some arches for now

    * - fixed tests for ppc64
    - added static `ptrsize` for ppc & ppc64 in gef

    * - `BIN_LS` -> `_target("default")`
    - removed auto demangle for now

    * - disable pytest `--pdb` from makefile
    - added doc for testing

    * Apply suggestions from code review

    Co-authored-by: Grazfather <grazfather@gmail.com>

    * fixing ci

    * [tests] use camel case for format string helper test class

    * [tests] added docstring to `GefFuncDeprecatedApi`

    * [tests] `edit-flags` are only for known arches for now

    * PR review changes

    * PR review last batch

    Co-authored-by: Grazfather <grazfather@gmail.com>
    Co-authored-by: theguy147 <37738506+theguy147@users.noreply.github.com>

commit 37bb542
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Sun Feb 13 10:54:28 2022 -0800

    Add class factory support for `pcustom` (#819)

    * [pcustom] add class factory support
    instead of creating a static class, class factories allow to generate a `ctypes.Structure` class with information from the runtime, which can drastically simplify (and unify) classes declaration (from on libc version, architecture, ptrsize, etc.)

    * [pcustom] added doc for class factory

    * linting

    * [pcustom] completed the documentation for class factory

    * Better filter of external attribute in the `Structure.__init__` to catch both classes and class factory methods

    * Apply suggestions from code review

    Co-authored-by: Grazfather <grazfather@gmail.com>

commit 82b2570
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Sun Feb 13 09:52:18 2022 -0800

    Remove `ida-interact` (#817)

    * remove `ida-interact` from gef, replaced by a better version in `gef-extras`

    * Fix double loading of external plugins (#816)

    `register_external_command` was receiving an instance of a class for each new external script. This lead to a double initialization when calling `gef.gdb.load(cls)`. Fixed by registering directly a class (just like `register_command`)

    * [lint] removed `xmlrpclib` unused import

    * restored doctstring of `IdaInteractCommand`

    * restoring python path insertion for python plugins for extra pacakges

commit d86e7a0
Author: hugsy <hugsy@blah.cat>
Date:   Wed Feb 9 13:04:48 2022 -0800

    Minor follow-up of #821

commit 8c0f625
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Tue Feb 8 17:59:08 2022 -0800

    - create `gef.ui.libc_args_table` (#821)

    - make the old  `libc_args_definitions` dict point to `gef.ui.libc_args_table`

commit 4365d9c
Author: hugsy <hugsy@users.noreply.github.com>
Date:   Fri Feb 4 09:55:08 2022 -0800

    Fix double loading of external plugins (#816)

    `register_external_command` was receiving an instance of a class for each new external script. This lead to a double initialization when calling `gef.gdb.load(cls)`. Fixed by registering directly a class (just like `register_command`)

commit d1fa00f
Author: theguy147 <37738506+theguy147@users.noreply.github.com>
Date:   Tue Feb 1 19:28:30 2022 +0100

    fix: make shebang lines portable (#814)

    * fix: make shebang lines portable

    * fix: SC2006, SC2086, SC2016, SC2059

commit c078733
Author: Grazfather <grazfather@gmail.com>
Date:   Tue Feb 1 10:57:08 2022 -0500

    Fix RISCV arch detection (#790)

    * Add RISCV alias so arch can be determined by ELF
    * Add ptrsize property to RISCV arch
    * Allow riscv tests to run

commit 48d39bb
Author: code-byter <10854537+code-byter@users.noreply.github.com>
Date:   Mon Jan 31 05:36:52 2022 +0100

    Update gef config parameters of gef-extras installation script (#807)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants