w3m-27 trace misses valid sections (maybe library)

[Airtnp]

gdb ./w3m
b __interceptor___getdelim
r -T text/html -dump '../ID-27/crash.html'
bt

In my OS
#0 0x00007ffff6e98630 in __interceptor___getdelim () from /lib64/libasan.so.4
#1 0x00007ffff2e019ac in selinuxfs_exists () from /lib64/libselinux.so.1
#2 0x00007ffff2df9c28 in init_lib () from /lib64/libselinux.so.1
#3 0x00007ffff7dea903 in _dl_init_internal () from /lib64/ld-linux-x86-64.so.2
#4 0x00007ffff7ddc15a in _dl_start_user () from /lib64/ld-linux-x86-64.so.2

In hase, the section containing 0x00007ffff2e019ac is missing (not exactly this address, just the function calling __interceptor__getdelim)

I cannot have exact function frame in NixOS for #1

Multithreading issues?

[Mic92]

Is this related to?

hase replay recordings/w3m-19-02ba3d6.tar.gz failed
Traceback (most recent call last):
  File "./replay.py", line 54, in process_trace
    except Exception as e:
  File "/local/incoop/hase/hase/__init__.py", line 14, in main
    return args.func(args)
  File "/local/incoop/hase/hase/cli.py", line 56, in lazy_import_replay_command
    return replay_command(args)
  File "/local/incoop/hase/hase/replay.py", line 137, in replay_command
    states, constraints = rt.run()
  File "/local/incoop/hase/hase/replay.py", line 122, in run
    states = self.tracer.run()
  File "/local/incoop/hase/hase/symbex/tracer.py", line 514, in run
    instruction.ip
AssertionError

[Airtnp]

Yes. Which means the IP is not contained in the project. Is this related to asan?

[Mic92]

I will have a look. The mapping must be valid in the context of processor trace otherwise we would not see it in the trace. vdso maybe?

[Mic92]

libraries are loaded incorrect:

Angr:

<ELF Object w3m, maps [0x400000:0x7f9af7]>
<ELF Object libcrypto.so.1.0.0, maps [0x1000000:0x146663f]>
<ELF Object libpthread-2.27.so, maps [0x2000000:0x221e24f]>
<ELF Object libc-2.27.so, maps [0x3000000:0x33b399f]>
<ELF Object libgcc_s.so.1, maps [0x4000000:0x42172cf]>
<ELFTLSObject Object cle##tls, maps [0x5000000:0x5015010]>
<ExternObject Object cle##externs, maps [0x6000000:0x6008000]>
<KernelObject Object cle##kernel, maps [0x7000000:0x7008000]>
<ELF Object libstdc++.so.6.0.24, maps [0x7fad74671000:0x7fad749f761f]>
<ELF Object librt-2.27.so, maps [0x7fad78ba9000:0x7fad78db09ff]>
<ELF Object libdl-2.27.so, maps [0x7fad78db1000:0x7fad78fb408f]>
<ELF Object libssl.so.1.0.0, maps [0x7fad797d0000:0x7fad79a4362f]>
<ELF Object libgc.so.1.3.4, maps [0x7fad79a44000:0x7fad79cae79f]>
<ELF Object libm-2.27.so, maps [0x7fad79caf000:0x7fad7a043017]>
<ELF Object libncursesw.so.6.1, maps [0x7fad7a044000:0x7fad7a2b2d57]>
<ELF Object libasan.so.4.0.0, maps [0x7fad7a2b3000:0x7fad7b2666a7]>
<ELF Object ld-2.27.so, maps [0x7fad7b267000:0x7fad7b48d10f]>
<ELF Object vdso, maps [0x7ffe4efd4000:0x7ffe4efd508a]>

core dump:

400000-516000 r-xp 116000 /tmp/tmpom6y6vti/binaries/home/joerg/git/hase/bug-db/w3m/02ba3d6/w3m
716000-717000 r--p 1000 /tmp/tmpom6y6vti/binaries/home/joerg/git/hase/bug-db/w3m/02ba3d6/w3m
717000-7f3000 rw-p dc000 /tmp/tmpom6y6vti/binaries/home/joerg/git/hase/bug-db/w3m/02ba3d6/w3m
7fad73a37000-7fad73d00000 r--p 2c9000 /tmp/tmpom6y6vti/binaries/home/joerg/git/hase/bug-db/w3m/02ba3d6/w3m
7fad74671000-7fad746b3000 r--p 42000 /tmp/tmpom6y6vti/binaries/nix/store/llnpd8fw5ymyv57jyxjh4v6sb92n1wff-gcc-7.3.0-lib/lib/libstdc++.so.6.0.24
7fad74848000-7fad74864000 r--p 1c000 /tmp/tmpom6y6vti/binaries/nix/store/bsgd325bvcns5lj8jkaapjyayvwlc6r5-openssl-1.0.2p/lib/libcrypto.so.1.0.0
7fad74870000-7fad74875000 r--p 5000 /tmp/tmpom6y6vti/binaries/nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libpthread-2.27.so
7fad74875000-7fad74889000 r--p 14000 /tmp/tmpom6y6vti/binaries/nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc-2.27.so
7fad74894000-7fad74896000 r--p 2000 /tmp/tmpom6y6vti/binaries/nix/store/llnpd8fw5ymyv57jyxjh4v6sb92n1wff-gcc-7.3.0-lib/lib/libgcc_s.so.1
7fad783eb000-7fad78401000 r-xp 16000 /tmp/tmpom6y6vti/binaries/nix/store/llnpd8fw5ymyv57jyxjh4v6sb92n1wff-gcc-7.3.0-lib/lib/libgcc_s.so.1
7fad78401000-7fad78601000 ---p 200000 /tmp/tmpom6y6vti/binaries/nix/store/llnpd8fw5ymyv57jyxjh4v6sb92n1wff-gcc-7.3.0-lib/lib/libgcc_s.so.1
7fad78601000-7fad78602000 r--p 1000 /tmp/tmpom6y6vti/binaries/nix/store/llnpd8fw5ymyv57jyxjh4v6sb92n1wff-gcc-7.3.0-lib/lib/libgcc_s.so.1
7fad78602000-7fad78603000 rw-p 1000 /tmp/tmpom6y6vti/binaries/nix/store/llnpd8fw5ymyv57jyxjh4v6sb92n1wff-gcc-7.3.0-lib/lib/libgcc_s.so.1
7fad78603000-7fad7877b000 r-xp 178000 /tmp/tmpom6y6vti/binaries/nix/store/llnpd8fw5ymyv57jyxjh4v6sb92n1wff-gcc-7.3.0-lib/lib/libstdc++.so.6.0.24
7fad7877b000-7fad7897a000 ---p 1ff000 /tmp/tmpom6y6vti/binaries/nix/store/llnpd8fw5ymyv57jyxjh4v6sb92n1wff-gcc-7.3.0-lib/lib/libstdc++.so.6.0.24
7fad7897a000-7fad78986000 r--p c000 /tmp/tmpom6y6vti/binaries/nix/store/llnpd8fw5ymyv57jyxjh4v6sb92n1wff-gcc-7.3.0-lib/lib/libstdc++.so.6.0.24
7fad78986000-7fad78987000 rw-p 1000 /tmp/tmpom6y6vti/binaries/nix/store/llnpd8fw5ymyv57jyxjh4v6sb92n1wff-gcc-7.3.0-lib/lib/libstdc++.so.6.0.24
7fad7898a000-7fad789a3000 r-xp 19000 /tmp/tmpom6y6vti/binaries/nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libpthread-2.27.so
7fad789a3000-7fad78ba3000 ---p 200000 /tmp/tmpom6y6vti/binaries/nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libpthread-2.27.so
7fad78ba3000-7fad78ba4000 r--p 1000 /tmp/tmpom6y6vti/binaries/nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libpthread-2.27.so
7fad78ba4000-7fad78ba5000 rw-p 1000 /tmp/tmpom6y6vti/binaries/nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libpthread-2.27.so
7fad78ba9000-7fad78bb0000 r-xp 7000 /tmp/tmpom6y6vti/binaries/nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/librt-2.27.so
7fad78bb0000-7fad78daf000 ---p 1ff000 /tmp/tmpom6y6vti/binaries/nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/librt-2.27.so
7fad78daf000-7fad78db0000 r--p 1000 /tmp/tmpom6y6vti/binaries/nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/librt-2.27.so
7fad78db0000-7fad78db1000 rw-p 1000 /tmp/tmpom6y6vti/binaries/nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/librt-2.27.so
7fad78db1000-7fad78db4000 r-xp 3000 /tmp/tmpom6y6vti/binaries/nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libdl-2.27.so
7fad78db4000-7fad78fb3000 ---p 1ff000 /tmp/tmpom6y6vti/binaries/nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libdl-2.27.so
7fad78fb3000-7fad78fb4000 r--p 1000 /tmp/tmpom6y6vti/binaries/nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libdl-2.27.so
7fad78fb4000-7fad78fb5000 rw-p 1000 /tmp/tmpom6y6vti/binaries/nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libdl-2.27.so
7fad78fb5000-7fad7915f000 r-xp 1aa000 /tmp/tmpom6y6vti/binaries/nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc-2.27.so
7fad7915f000-7fad7935f000 ---p 200000 /tmp/tmpom6y6vti/binaries/nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc-2.27.so
7fad7935f000-7fad79363000 r--p 4000 /tmp/tmpom6y6vti/binaries/nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc-2.27.so
7fad79363000-7fad79365000 rw-p 2000 /tmp/tmpom6y6vti/binaries/nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libc-2.27.so
7fad79369000-7fad795a6000 r-xp 23d000 /tmp/tmpom6y6vti/binaries/nix/store/bsgd325bvcns5lj8jkaapjyayvwlc6r5-openssl-1.0.2p/lib/libcrypto.so.1.0.0
7fad795a6000-7fad797a5000 ---p 1ff000 /tmp/tmpom6y6vti/binaries/nix/store/bsgd325bvcns5lj8jkaapjyayvwlc6r5-openssl-1.0.2p/lib/libcrypto.so.1.0.0
7fad797a5000-7fad797c1000 r--p 1c000 /tmp/tmpom6y6vti/binaries/nix/store/bsgd325bvcns5lj8jkaapjyayvwlc6r5-openssl-1.0.2p/lib/libcrypto.so.1.0.0
7fad797c1000-7fad797cc000 rw-p b000 /tmp/tmpom6y6vti/binaries/nix/store/bsgd325bvcns5lj8jkaapjyayvwlc6r5-openssl-1.0.2p/lib/libcrypto.so.1.0.0
7fad797d0000-7fad79839000 r-xp 69000 /tmp/tmpom6y6vti/binaries/nix/store/bsgd325bvcns5lj8jkaapjyayvwlc6r5-openssl-1.0.2p/lib/libssl.so.1.0.0
7fad79839000-7fad79a39000 ---p 200000 /tmp/tmpom6y6vti/binaries/nix/store/bsgd325bvcns5lj8jkaapjyayvwlc6r5-openssl-1.0.2p/lib/libssl.so.1.0.0
7fad79a39000-7fad79a3e000 r--p 5000 /tmp/tmpom6y6vti/binaries/nix/store/bsgd325bvcns5lj8jkaapjyayvwlc6r5-openssl-1.0.2p/lib/libssl.so.1.0.0
7fad79a3e000-7fad79a44000 rw-p 6000 /tmp/tmpom6y6vti/binaries/nix/store/bsgd325bvcns5lj8jkaapjyayvwlc6r5-openssl-1.0.2p/lib/libssl.so.1.0.0
7fad79a44000-7fad79a6b000 r-xp 27000 /tmp/tmpom6y6vti/binaries/nix/store/xg0ilj83pz2h47mlnwzf2l63xi60xy93-boehm-gc-7.6.8/lib/libgc.so.1.3.4
7fad79a6b000-7fad79c6b000 ---p 200000 /tmp/tmpom6y6vti/binaries/nix/store/xg0ilj83pz2h47mlnwzf2l63xi60xy93-boehm-gc-7.6.8/lib/libgc.so.1.3.4
7fad79c6b000-7fad79c6c000 r--p 1000 /tmp/tmpom6y6vti/binaries/nix/store/xg0ilj83pz2h47mlnwzf2l63xi60xy93-boehm-gc-7.6.8/lib/libgc.so.1.3.4
7fad79c6c000-7fad79c6d000 rw-p 1000 /tmp/tmpom6y6vti/binaries/nix/store/xg0ilj83pz2h47mlnwzf2l63xi60xy93-boehm-gc-7.6.8/lib/libgc.so.1.3.4
7fad79caf000-7fad79e42000 r-xp 193000 /tmp/tmpom6y6vti/binaries/nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libm-2.27.so
7fad79e42000-7fad7a042000 ---p 200000 /tmp/tmpom6y6vti/binaries/nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libm-2.27.so
7fad7a042000-7fad7a043000 r--p 1000 /tmp/tmpom6y6vti/binaries/nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libm-2.27.so
7fad7a043000-7fad7a044000 rw-p 1000 /tmp/tmpom6y6vti/binaries/nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libm-2.27.so
7fad7a044000-7fad7a0ae000 r-xp 6a000 /tmp/tmpom6y6vti/binaries/nix/store/hgxybgxcmcn13fl25l6lmd5smdhdn6g4-ncurses-6.1/lib/libncursesw.so.6.1
7fad7a0ae000-7fad7a2ad000 ---p 1ff000 /tmp/tmpom6y6vti/binaries/nix/store/hgxybgxcmcn13fl25l6lmd5smdhdn6g4-ncurses-6.1/lib/libncursesw.so.6.1
7fad7a2ad000-7fad7a2b2000 r--p 5000 /tmp/tmpom6y6vti/binaries/nix/store/hgxybgxcmcn13fl25l6lmd5smdhdn6g4-ncurses-6.1/lib/libncursesw.so.6.1
7fad7a2b2000-7fad7a2b3000 rw-p 1000 /tmp/tmpom6y6vti/binaries/nix/store/hgxybgxcmcn13fl25l6lmd5smdhdn6g4-ncurses-6.1/lib/libncursesw.so.6.1
7fad7a2b3000-7fad7a3fc000 r-xp 149000 /tmp/tmpom6y6vti/binaries/nix/store/llnpd8fw5ymyv57jyxjh4v6sb92n1wff-gcc-7.3.0-lib/lib/libasan.so.4.0.0
7fad7a3fc000-7fad7a5fc000 ---p 200000 /tmp/tmpom6y6vti/binaries/nix/store/llnpd8fw5ymyv57jyxjh4v6sb92n1wff-gcc-7.3.0-lib/lib/libasan.so.4.0.0
7fad7a5fc000-7fad7a5ff000 r--p 3000 /tmp/tmpom6y6vti/binaries/nix/store/llnpd8fw5ymyv57jyxjh4v6sb92n1wff-gcc-7.3.0-lib/lib/libasan.so.4.0.0
7fad7a5ff000-7fad7a602000 rw-p 3000 /tmp/tmpom6y6vti/binaries/nix/store/llnpd8fw5ymyv57jyxjh4v6sb92n1wff-gcc-7.3.0-lib/lib/libasan.so.4.0.0
7fad7b267000-7fad7b28c000 r-xp 25000 /tmp/tmpom6y6vti/binaries/nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/ld-2.27.so
7fad7b28d000-7fad7b28f000 r--p 2000 /tmp/tmpom6y6vti/binaries/nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/librt-2.27.so
7fad7b2a4000-7fad7b2a7000 r--p 3000 /tmp/tmpom6y6vti/binaries/nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/ld-2.27.so
7fad7b2ac000-7fad7b2b1000 r--p 5000 /tmp/tmpom6y6vti/binaries/nix/store/xg0ilj83pz2h47mlnwzf2l63xi60xy93-boehm-gc-7.6.8/lib/libgc.so.1.3.4
7fad7b2b1000-7fad7b2b7000 r--p 6000 /tmp/tmpom6y6vti/binaries/nix/store/bsgd325bvcns5lj8jkaapjyayvwlc6r5-openssl-1.0.2p/lib/libssl.so.1.0.0
7fad7b2b7000-7fad7b2b8000 r--p 1000 /tmp/tmpom6y6vti/binaries/nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libdl-2.27.so
7fad7b2c9000-7fad7b2d4000 r--p b000 /tmp/tmpom6y6vti/binaries/nix/store/llnpd8fw5ymyv57jyxjh4v6sb92n1wff-gcc-7.3.0-lib/lib/libasan.so.4.0.0
7fad7b2d4000-7fad7b2d8000 r--p 4000 /tmp/tmpom6y6vti/binaries/nix/store/hgxybgxcmcn13fl25l6lmd5smdhdn6g4-ncurses-6.1/lib/libncursesw.so.6.1
7fad7b2d8000-7fad7b2e0000 r--p 8000 /tmp/tmpom6y6vti/binaries/nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libm-2.27.so
7fad7b327000-7fad7b331000 r--p a000 /tmp/tmpom6y6vti/binaries/home/joerg/git/hase/bug-db/w3m/02ba3d6/w3m
7fad7b48b000-7fad7b48c000 r--p 1000 /tmp/tmpom6y6vti/binaries/nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/ld-2.27.so
7fad7b48c000-7fad7b48d000 rw-p 1000 /tmp/tmpom6y6vti/binaries/nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/ld-2.27.so
7ffe4efd4000-7ffe4efd6000 r-xp 2000 /tmp/tmpom6y6vti/vdso

[Airtnp]

The loader should have a fix?
https://github.com/hase-project/hase/blob/master/hase/loader.py#L53
I don't know whether Angr supports a complex library mapping.

[Mic92]

I currently blame autoloading. I try to disable it.
Angr also does not support loading objects twice, in which case we would need to translate the offsets on the fly.

[Mic92]

We already depend indirectly on https://github.com/chaimleib/intervaltree for that purpose.

[Airtnp]

I think if we need loading fragmented objects, we need to modify Angr since we need to execute instructions. Otherwise, we must repeat what Angr has done from assembly -> VEX IR -> engines.

Also we need to change everything relevant to library decoding.

[Mic92]

Angr also allows to specify a custom loader rather then using load_options.

[Mic92]

I think libasan or something else does map the elf header, because those are read-only:

7fad74671000-7fad746b3000 r--p 42000 /tmp/tmpom6y6vti/binaries/nix/store/llnpd8fw5ymyv57jyxjh4v6sb92n1wff-gcc-7.3.0-lib/lib/libstdc++.so.6.0.24
7fad74848000-7fad74864000 r--p 1c000 /tmp/tmpom6y6vti/binaries/nix/store/bsgd325bvcns5lj8jkaapjyayvwlc6r5-openssl-1.0.2p/lib/libcrypto.so.1.0.0
7fad74870000-7fad74875000 r--p 5000 /tmp/tmpom6y6vti/binaries/nix/store/g2yk54hifqlsjiha3szr4q3ccmdzyrdv-glibc-2.27/lib/libpthread-2.27.so

I ignore those mapping for the time beeing

[Mic92]

That issue was solved.