New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP reflective loading #154

Open
wants to merge 12 commits into
base: master
from

Conversation

Projects
None yet
2 participants
@acammack-r7
Copy link
Contributor

acammack-r7 commented Dec 21, 2018

Ok, POC is now mostly refactored and reliable. Time for a proper TODO:

  • Convert to autoconf
  • Integrate into mettle build system
  • Add support in mettle for executing a TLV_TYPE_VALUE_DATA accompanying a stdapi sys.execute request
    • Execute the thing
    • Add process name imitation
    • Check executables for compatibility
    • Bounds checking?
  • Port to remaining mettle Linux architectures
    • x86_64
    • i486
    • aarch64
    • armv5l
    • armv5b
    • powerpc-e500v2
    • powerpc
    • powerpc64le
    • mips
    • mipsel
    • mips64
    • s390x
  • Implement non-architecture-dependent fallback, probably with memfd_create(2) and execveat(2)
  • Convert mettle plugins to load using libreflect

Verification

libreflect can be built by itself from the mettle root directory with make libreflect TARGET=X, similar to how mettle is built.

  • With one of the standalone examples (currently noexec), run a statically-linked, a dynamically-linked (interpreted), a position-independent, and a position-dependent executable (exes can be more than one of those) for each of the mettle Linux architectures:

    • x86_64
    • i486
    • aarch64
    • armv5l
    • armv5b
    • powerpc-e500v2
    • powerpc
    • powerpc64le
    • mips
    • mipsel
    • mips64
    • s390x
  • TODO: Once implemented, run mettle for each of the Linux architectures and use execute -m to upload and run an executable without touching disk:

    • x86_64
    • i486
    • aarch64
    • armv5l
    • armv5b
    • powerpc-e500v2
    • powerpc
    • powerpc64le
    • mips
    • mipsel
    • mips64
    • s390x

@acammack-r7 acammack-r7 added the delayed label Dec 21, 2018

@CaledoniaProject

This comment has been minimized.

Copy link

CaledoniaProject commented Jan 8, 2019

Segment fault on Ubuntu 16.04 + /bin/ls:

%> gcc exec.c -g
%> gdb ./a.out
Reading symbols from ./a.out...done.
(gdb) run /bin/ls
Starting program: /XXXX/a.out /bin/ls

Program received signal SIGSEGV, Segmentation fault.
__memset_avx2 () at ../sysdeps/x86_64/multiarch/memset-avx2.S:161
161	../sysdeps/x86_64/multiarch/memset-avx2.S: No such file or directory.
(gdb) bt
#0  __memset_avx2 () at ../sysdeps/x86_64/multiarch/memset-avx2.S:161
#1  0x0000000000400ad4 in map_elf (data=0x7ffff7fcd000 "\177ELF\002\001\001") at exec.c:65
#2  0x0000000000400da5 in main (argc=2, argv=0x7fffffffe588, env=0x7fffffffe5a0) at exec.c:127
(gdb)

The corresponding source code:

    61						if(mapping == MAP_FAILED) {
    62							printf("Failed to mmap(): %s\n", strerror(errno));
    63							exit(EXIT_FAILURE);
    64						}
    65						memset(mapping, 0, total_to_map);
    66						printf("data @ %p, mapping @ %p\n", data, mapping);
    67						if(phdr->p_vaddr == 0) virtual_offset = (size_t) mapping;
    68					}

@acammack-r7 acammack-r7 force-pushed the acammack-r7:wip-reflection branch from e79a1f7 to 3b801d7 Jan 8, 2019

@acammack-r7

This comment has been minimized.

Copy link
Contributor

acammack-r7 commented Jan 8, 2019

Sorry about that bug, @CaledoniaProject my post ended up going live before I had time to revise the POC. The original version had errors with non-PIE executables. This updated version no longer contains that bug, but does require GCC 8+ to build while I add in the autoconf support enable use of the toolchain we use for the rest of mettle.

acammack-r7 added some commits Jan 8, 2019

@acammack-r7

This comment has been minimized.

Copy link
Contributor

acammack-r7 commented Jan 9, 2019

@CaledoniaProject You should now be able to compile this with the tool chain the rest of mettle uses with make libreflect TARGET=x86_64-linux-musl from the base directory. ./build/x86_64-linux-musl/bin/noexec /bin/ls should then behave like you expect. You can also build compile on 18.04 (GCC 8) and use make libreflect to build ./build/linux.x86_64/bin/noexec.

acammack-r7 added some commits Jan 9, 2019

@CaledoniaProject

This comment has been minimized.

Copy link

CaledoniaProject commented Jan 10, 2019

Still getting segfault on Ubuntu 16.04. Haven't tried autoconf and Ubuntu 18.04, but is gcc 8.X required to compile this?

# gcc -DDEBUG examples/noexec.c src/*.c -o noexec -Iinclude -Isrc -Iarch/linux/x86_64/ -g
# gdb ./noexec
Reading symbols from ./noexec...done.
(gdb) run /bin/ls
Starting program: /tmp/mettle-wip-reflection/libreflect/noexec /bin/ls
total mapping is now 0041dae4 based on 0001dae4 seg at 0x400000
total mapping is now 0061f368 based on 00001568 seg at 0x61de00
data @ 0x7ffff7fcd000, mapping @ 0x7ffff77ee000
memcpy(0x400000, 0x7ffff7fcd000, 0001dae4)

Program received signal SIGSEGV, Segmentation fault.
__memcpy_avx_unaligned () at ../sysdeps/x86_64/multiarch/memcpy-avx-unaligned.S:238
238	../sysdeps/x86_64/multiarch/memcpy-avx-unaligned.S: No such file or directory.
(gdb) bt
#0  __memcpy_avx_unaligned () at ../sysdeps/x86_64/multiarch/memcpy-avx-unaligned.S:238
#1  0x000000000040106d in map_elf (data=0x7ffff7fcd000 "\177ELF\002\001\001", obj=0x7fffffffe360) at src/map_elf.c:65
#2  0x0000000000400b39 in main (argc=2, argv=0x7fffffffe528, env=0x7fffffffe540) at examples/noexec.c:50
(gdb)

Line 64-66:

			dprint("memcpy(%p, %p, %08zx)\n", (void *)dest, source, len);
			memcpy((void *)dest, source, len);

@CaledoniaProject

This comment has been minimized.

Copy link

CaledoniaProject commented Jan 10, 2019

Multiple issues

1. autoconf does not work on Ubuntu 16.04

I got error: possibly undefined macro: AC_MSG_WARN,

# make libreflect TARGET=x86_64-linux-musl
Downloading toolchain
curl -f -O https://s3.amazonaws.com/muslcross/musl-cross-linux-6.tar.xz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  346M  100  346M    0     0  2347k      0  0:02:31  0:02:31 --:--:-- 3051k
Unpacking toolchain
configure.ac:26: error: possibly undefined macro: dnl
      If this token and others are legitimate, please use m4_pattern_allow.
      See the Autoconf documentation.
configure.ac:47: error: possibly undefined macro: AC_MSG_WARN
autoreconf: /usr/bin/autoconf failed with exit status: 1
make/Makefile.libreflect:2: recipe for target '/home/tmp/mettle-wip-reflection/libreflect/configure' failed
make: *** [/home/tmp/mettle-wip-reflection/libreflect/configure] Error 1

2. More test results. It works on Ubuntu 18.04 only

I first did a static compile on Ubuntu 16.04:

gcc -DDEBUG examples/noexec.c src/*.c -o /tmp/noexec -Iinclude -Isrc -Iarch/linux/x86_64/ -g -static

Execute on Ubuntu 18.04

root@ubuntu:~# ./noexec /bin/ls -lh .
total mapping is now 0001e6e8 based on 0001e6e8 seg at (nil)
total mapping is now 00221560 based on 00002570 seg at 0x21eff0
data @ 0x7f19aff49000, mapping @ 0x7f19af739000
memcpy(0x7f19af739000, 0x7f19aff49000, 0001e6e8)
memcpy(0x7f19af957ff0, 0x7f19aff67ff0, 00001278)
Mapped ELF interp file in: /lib64/ld-linux-x86-64.so.2
total mapping is now 00026c24 based on 00026c24 seg at (nil)
total mapping is now 00229170 based on 00001af0 seg at 0x227680
data @ 0x7f19aff40000, mapping @ 0x7f19af50f000
memcpy(0x7f19af50f000, 0x7f19aff40000, 00026c24)
memcpy(0x7f19af736680, 0x7f19aff67680, 00001958)
total 20K
-rwxr-xr-x 1 root root 20K Jan 10 04:18 noexec

Execute on CentOS 6

# ./noexec /bin/ls
total mapping is now 0041851c based on 0001851c seg at 0x400000
total mapping is now 0061af60 based on 00001f60 seg at 0x619000
data @ 0x7f628bd88000, mapping @ 0x7f628bb6d000
memcpy(0x400000, 0x7f628bd88000, 0001851c)
Segmentation fault

Execute on CentOS 7

# ./noexec /bin/ls
total mapping is now 004193fc based on 000193fc seg at 0x400000
total mapping is now 0061c320 based on 00002000 seg at 0x61a320
data @ 0x7fbe6e41a000, mapping @ 0x7fbe6e1fd000
memcpy(0x400000, 0x7fbe6e41a000, 000193fc)
Segmentation fault
@acammack-r7

This comment has been minimized.

Copy link
Contributor

acammack-r7 commented Jan 10, 2019

@CaledoniaProject The example as written needs to be compiled as a PIE to work, and on 18.04 that is the default; adding -fPIE to the other systems should get you what you need. You can get it building by with the cross toolchain by making sure the autoconf, libtool, and autoconf-archive packages are properly installed. The autoconf-archive is required to check if the compiler flags we need are supported. It is a new dependency for mettle, sorry to not call it out. Thanks for taking the time to try this out!

@CaledoniaProject

This comment has been minimized.

Copy link

CaledoniaProject commented Jan 11, 2019

Recompiled with PIE enabled,

gcc -DDEBUG examples/noexec.c src/*.c -o noexec -Iinclude -Isrc -Iarch/linux/x86_64/ -g -fPIE -pie

And it worked 👍

acammack-r7 added some commits Jan 18, 2019

@CaledoniaProject

This comment has been minimized.

Copy link

CaledoniaProject commented Jan 22, 2019

libreflect can't handle UPX compressed files, can you add some support?

@acammack-r7

This comment has been minimized.

Copy link
Contributor

acammack-r7 commented Jan 22, 2019

Hmm, if they don't work out of the box it'll take some digging to figure out why and how to make them work. I probably won't have time to include them running them in this PR, but I'll take a look after all the basics are done.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment