[Support] help getting unmapped memory address when error occurs

[GTP95]

Hello,

I'm trying to emulate a binary, but unfortunately it fails with the following error message: unicorn.unicorn.UcError: Invalid memory write (UC_ERR_WRITE_UNMAPPED). When such an error occurs, I would like to be able to get the offending memory address, so that I can check if my memory mappings are correct. My idea would be to add a hook via ql.hook_mem_unmapped(), but it is not clear to me how I could get the memory address from there. What I had in mind, is something like this:

def warn_unmapped(mem_addr):
    print("Warning: attempted access to unmapped memory at address " + mem_addr)

ql.hook_mem_unmapped(warn_unmapped(mem_addr))

But I don't know where to get the value of mem_addr from. How can I get this to work? Or is there a better way to do this?
Thank you for your help,

GTP95

[elicn]

This is supported with the latest Unicorn library.
You can do the following:

try:
    # do some stuff that might fail due to an invalid memory access ...
except UcError as ex:
    addr, size = ex.args

    print(f'failed to access address {addr:#010x} (access size: {size:d})')

[GTP95]

Thank you for your suggestion. Unfortunately, if I do that I get a ValueError as ex.args has only one value instead of two, if I print it I get (7,). What does this mean?

EDIT: I also directly modified Unicorn's code to print the address when the exception is raised, and the memory address in decimal is 268435456, which has nothing to do with 7.

[GTP95]

Nevermind, the problem was that I had an older version of Unicorn, now I have installed unicorn-2.1.4. But now, instead of printing the offending memory address, I get a different error:

Traceback (most recent call last):
  File "/home/giacomo/git/armchair/main.py", line 79, in <module>
    session.run_session(target_profile=tp)
  File "/home/giacomo/git/armchair/armchair/components/armchair_session.py", line 142, in run_session
    session.run_session()
  File "/home/giacomo/git/armchair/armchair/components/armchair_session_runner.py", line 114, in run_session
    self.process_row(row=self.target_data[0], index=[0])
  File "/home/giacomo/git/armchair/armchair/components/armchair_session_runner.py", line 99, in process_row
    raise e
  File "/home/giacomo/git/armchair/armchair/components/armchair_session_runner.py", line 57, in process_row
    ql: Qiling = initialize_qiling(
                 ^^^^^^^^^^^^^^^^^^
  File "/home/giacomo/git/armchair/armchair/utils/helpers.py", line 647, in initialize_qiling
    ql = Qiling(
         ^^^^^^^
  File "/home/giacomo/git/armchair/.venv/lib/python3.12/site-packages/qiling/core.py", line 190, in __init__
    self.loader.run()
  File "/home/giacomo/git/armchair/.venv/lib/python3.12/site-packages/qiling/loader/mcu.py", line 133, in run
    self.load_env()
  File "/home/giacomo/git/armchair/.venv/lib/python3.12/site-packages/qiling/loader/mcu.py", line 115, in load_env
    self.ql.mem.map(base, size, info=f'[{name}]')
  File "/home/giacomo/git/armchair/.venv/lib/python3.12/site-packages/qiling/os/memory.py", line 612, in map
    self.ql.uc.mem_map(addr, size, perms)
  File "/home/giacomo/git/armchair/.venv/lib/python3.12/site-packages/unicorn/unicorn_py3/unicorn.py", line 831, in mem_map
    status = uclib.uc_mem_map(self._uch, address, size, perms)
                              ^^^^^^^^^
AttributeError: 'UcAArch32' object has no attribute '_uch'

Any idea why Unicorn is now raising an AttributeError?

[GTP95]

Turned out that the AttributeError was caused by using the latest version of Unicorn, which isn't compatible with the latest version of Qiling. Fixed by pinning the correct version.

[GTP95]

Sorry, I spoke too soon: with the latest Qiling-supported version of Unicorn ( 2.0.1.post1), it still doesn't work:

 try:
            # init traces object in memory (of the instructions are too many the hook for recording instructions could be used to write directly to csv instead)
            traces: list = []
            cache: dict = {}

            # initialize Qiling object
            ql: Qiling = initialize_qiling(
                elf=self.elf_path,
                sym_parser=self.sym_parser,
                profile=self.target_profile,
                traces=traces,
                cache=cache,
                json_path=self.json_path
            )

            self.target_profile.init_uart(ql=ql, input_format=self.input_format)

            output_path = f"{TRACESPATH}-{self.target_profile.get_algorithm_name()}"

            os.makedirs(output_path, 511, True)

            output_path: str = os.path.join(
                output_path,
                f"traces_{index[0] + 1}.csv",
            )

            self.target_profile.hook_cmds(
                ql=ql, sym_parser=self.sym_parser, target_data=row
            )

            ql.log.info("Starting Qiling..")
            ql.run()

            if logging.getLogger().isEnabledFor(logging.DEBUG):
                logger.debug("Writing traces to CSV...")

            # Write traces using csv module
            with open(output_path, mode="w", newline="") as file:
                csv_writer = csv.writer(file)
                csv_writer.writerow(headerList)  # Write the headers
                csv_writer.writerows(traces)  # Write the rows of data
        except UcError as ex:   # catch memory errors, so that we get the offending address
            addr, size = ex.args
            self.logger.error(f'failed to access address {addr:#0x10} (access size: {size:d})')
        except Exception as e:
            logger.error(f"\nError processing data {row}: {e}")
            raise e

Traceback (most recent call last):
  File "/IdeaProjects/armchair/armchair/components/armchair_session_runner.py", line 57, in process_row
    ql: Qiling = initialize_qiling(
                 ^^^^^^^^^^^^^^^^^^
  File "/IdeaProjects/armchair/armchair/utils/helpers.py", line 653, in initialize_qiling
    ql = Qiling(
         ^^^^^^^
  File "/IdeaProjects/armchair/.venv/lib/python3.12/site-packages/qiling/core.py", line 190, in __init__
    self.loader.run()
  File "/IdeaProjects/armchair/.venv/lib/python3.12/site-packages/qiling/loader/mcu.py", line 138, in run
    self.reset()
  File "/IdeaProjects/armchair/.venv/lib/python3.12/site-packages/qiling/loader/mcu.py", line 90, in reset
    self.ql.mem.write(segment['p_paddr'], segment.data())
  File "/IdeaProjects/armchair/.venv/lib/python3.12/site-packages/qiling/os/memory.py", line 353, in write
    self.ql.uc.mem_write(addr, data)
  File "/IdeaProjects/armchair/.venv/lib/python3.12/site-packages/unicorn/unicorn.py", line 586, in mem_write
    raise UcError(status)
unicorn.unicorn.UcError: Invalid memory write (UC_ERR_WRITE_UNMAPPED)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/IdeaProjects/armchair/ARMChair.py", line 79, in <module>
    session.run_session(target_profile=tp)
  File "/IdeaProjects/armchair/armchair/components/armchair_session.py", line 142, in run_session
    session.run_session()
  File "/IdeaProjects/armchair/armchair/components/armchair_session_runner.py", line 112, in run_session
    self.process_row(row=self.target_data[0], index=[0])
  File "/IdeaProjects/armchair/armchair/components/armchair_session_runner.py", line 93, in process_row
    addr, size = ex.args
    ^^^^^^^^^^
ValueError: not enough values to unpack (expected 2, got 1)

So I guess that the latest version of Unicorn allows this, but since it is not supported by Qiling I'm out of luck for now. Is this correct?

[elicn]

You should go back to Unicorn version 2.1.4.
The AttributeError you encountered there has nothing to do with compatibility to Qiling. I'll look into it and let you know.

[elicn]

I ran a few MCU tests and they all run fine.
Is there a repro script you can share?

[GTP95]

Sure, here's a minimum reproducible example. I'm running this using Python 3.12, Qiling 1.4.6, and Unicorn 2.1.4. The executable I'm emulating is the ELF from the RP2350 Hacking Challenge 2 that you will need to build, then pass the path to it in the appropriate place inside the following script:

from qiling import Qiling
from qiling.const import QL_ARCH, QL_OS, QL_VERBOSE
from qiling.extensions.mcu.stm32f1 import stm32f103

if __name__ == '__main__':
    ql = Qiling(
            argv=["here_goes_your_path_to_rp2350_hacking_challenge_2.elf"],
            archtype=QL_ARCH.CORTEX_M,
            ostype=QL_OS.MCU,
            env=stm32f103,
            verbose=QL_VERBOSE.DEBUG
        )

    ql.hw.create("usart1")

    ql.run()

It's still entirely possible that I'm doing something wrong, as I'm not familiar with Qiling. Let me know if that's the case.

[elicn]

Let's start by aligning the versions: Qiling 1.4.6 is more than 2 years old. Try again after updating it to 1.4.7, or better yet, pip install Qiling's dev branch as it is most up to date.

[GTP95]

Thank you for pointing this out, I assumed to have the latest version as this is the current version on PyPI. I installed the development branch with pip install https://github.com/qilingframework/qiling/archive/dev.zip. It did install Qiling, but it also slightly downgraded Unicorn (to version 2.1.3) in the process.

It now indeed stopped complaining about the missing attribute, and got back to it complaining about unmapped memory. Now your suggested way of capturing the exception works, thank you! The only little detail, for future reference, is that it says ValueError: Invalid format specifier '#0x10' for object of type 'int', but removing that format specifier works.
Thanks again!