ARM Assembly Cheat
Almost all the content in this repository has been moved to: https://github.com/cirosantilli/linux-kernel-module-cheat#userland-assembly
Only the core infrastructure of this repo was left here. No major new features are intended to be added here.
Notable advantages of LKMC repository include:
a single unified cross arch setup for ARM and x86_64, with cross arch concepts all nicely factored out
gem5 support. This is because we have integration of QEMU / gem5 / buildroot setups already done there
parallel testing. Mostly because the build system there is Python, which is more flexible.
other stuff I can’t remember right now. That setup just has a ton of features, and will continue to get more and more ;-)
There is only one use case left for this repository: since this is more minimal, it is easier to upgrate to the latest binutils-gdb here without breaking unrelated stuff, in order to get very latest instructions.
The initial motivation for that was SVE although that specific case will soon be covered on LKMC as well.
I might however start taking some risks on LKMC and upgrading binutils-gdb to master when needed anyways, or just build the latest binutils-gdb myself over there for userland only.
Here is the old README with only infrastructure sections left.
ARMv7 and [ARMv8] assembly userland minimal examples tutorial. Runnable asserts on x86 hosts with QEMU user mode or natively on ARM targets. Nice GDB step debug setup. Tested on Ubuntu 18.04 host and Raspberry Pi 2 and 3 targets. Baremetal setup at: https://github.com/cirosantilli/linux-kernel-module-cheat#baremetal-setup x86 cheat at: https://github.com/cirosantilli/x86-assembly-cheat
- 1. Old README
- 2. Getting started
- 2.1. Asserts
- 2.2. Getting started on non-Ubuntu operating systems
- 2.3. GDB step debug
- 2.4. Disassemble
- 2.5. crosstool-NG toolchain
- 2.6. Build the documentation
- 2.7. Custom build flags
- 2.8. SVE
- 3. Theory
- 4. CONTRIBUTING
On Ubuntu, clone, configure, build QEMU and Binutils from source, run all ARMv7 and ARMv8 examples through QEMU user, and assert that they exit with status 0:
git clone --recursive https://github.com/cirosantilli/arm-assembly-cheat cd arm-assembly-cheat ./download-dependencies make test echo $?
Expected outcome: the exit status is successful:
For other operating systems, see: Getting started on non-Ubuntu operating systems.
We compile our own Binutils and QEMU to be able to use the newest ISA features. Those projects build pretty fast (~10 minutes), so it is fine. The cleanest thing would be to also compile GCC with crosstool-NG toolchain.
The armv7 examples are all located under the v7 directory. Run all of them:
cd v7 make test echo $?
Run just one of them:
cd v7 make test-<basename-no-extension> echo $?
will run userland/arch/arm/add.S.
This just tests some assertions, but does not output anything. See: Asserts.
Alternatively, to help with tab completion, the following shortcuts all do the same thing as
./t add ./t add. ./t add.out
cd v8 make test-movk
Just build the examples without running:
Clean the examples:
This does not clean QEMU builds themselves. To do that run:
Almost all example don’t output anything, they just assert that the computations are as expected and exit 0 is that was the case.
Failures however output clear error messages.
Try messing with the examples to see them fail, e.g. modify userland/arch/arm/add.S to contain:
mov r0, #1 add r1, r0, #2 ASSERT_EQ(r1, 4)
and then watch it fail:
cd v7 make test-add
error 1 at line 12 Makefile:138: recipe for target 'test-add' failed error 1 at line 12
1 + 2 tends to equal
3 and not
So look how nice we are: we even gave you the line number
12 of the failing assert!
If you are not on an Ubuntu host machine, here are some ways in which you can use this repo.
For other Linux distros, you can either:
have a look at what
download-dependenciesdoes and adapt it to your distro. It should be easy, then proceed normally.
Might fail due to some incompatibility, but likely won’t.
run this repo with docker. Requires you to know some Docker boilerplate, but cannot (?) fail.
sudo apt install docker sudo docker create -it --name arm-assembly-cheat -w "/host/$(pwd)" -v "/:/host" ubuntu:18.04 sudo docker exec -it arm-assembly-cheat /bin/bash
Then inside Docker just add the
--docker flag to
./download-dependencies and proceed otherwise normally:
./download-dependencies --docker make test
download-dependencies takes a while because
build-dep binutils is large.
We share the repository between Docker and host, so you can just edit the files on host with your favorite text editor, and then just run them from inside Docker.
TODO: GDB TUI GUI is broken inside Docker due to terminal quirks. Forwarding the port and connecting from host will likely work, but I’m lazy to try it out now.
For non-Linux systems, the easiest thing to do is to use an Ubuntu virtual machine such as VirtualBox: https://askubuntu.com/questions/142549/how-to-install-ubuntu-on-virtualbox.
Porting is not however impossible because we use the C standard library for portability, see: Architecture of this repo. Pull requests are welcome.
Yay! Let’s see if this actually works on real hardware, or if it is just an emulation pipe dream?
Tested on Raspbian Lite 2018-11-13 with this repo at commit bcddf29c8e00b30afe7b3643558b25f22a64405b.
For now, we will just compile natively, since I’m not in the mood for cross compilation hell today.
Therefore we will only be able to run
v7 examples on that board.
First connect to your Pi through SSH as explained at: https://stackoverflow.com/revisions/39086537/10
Then inside the Pi:
sudo apt-get update sudo apt-get install git make gcc gdb git clone https://github.com/cirosantilli/arm-assembly-cheat cd arm-assembly-cheat/v7 make NATIVE=y test make NATIVE=y gdb-add
GDB TUI is slightly buggier on the ancient 4.9 toolchain (current line gets different indentation, does not break on the right instruction after
asm_main_after_prologue, cannot leave TUI), but it might still be usable
The Pi 0 and 1 however have a BCM2835 SoC, which has an ARM1176JZF-S core, which implements the ARMv6Z ISA, which we don’t support yet on this repo.
However, as of July 2018, there is no official [armv8] image for the Pi 3, the same ARMv7 image is provided for both: https://raspberrypi.stackexchange.com/questions/43921/raspbian-moving-to-64-bit-mode
Then we look at the following threads:
which lead us to this 64-bit Debian based distro for the Pi: https://github.com/bamarni/pi64
So first we flash pi64’s 2017-07-31 release, and then do exactly the same as for the Raspberry Pi 2, except that you must go into the
v8 directory instead of
TODO: can we run the
v7 folder in ARMv8? First I can’t even compile it. Related: https://stackoverflow.com/questions/21716800/does-gcc-arm-linux-gnueabi-build-for-a-64-bit-target For runtime: https://stackoverflow.com/questions/22460589/armv8-running-legacy-32-bit-applications-on-64-bit-os
Debug one example with GDB:
./t -g add
This leaves us right at the end of the prologue of
asm_main in GDB TUI mode, which is at the start of the assembly code in the
Stop on a different symbol instead:
make GDB_BREAK=main gdb-add
./t -b main -g add
It is not possible to restart the running program from GDB as in
gdbserver --multi unfortunately: https://stackoverflow.com/questions/51357124/how-to-restart-qemu-user-mode-programs-from-the-gdb-stub-as-in-gdbserver-multi
Quick GDB tips:
print a register:
i r r0
print floating point registers:
print an array of 4 32-bit integers in hex:
print the address of a variable:
The default setup is opinionated and assumes that your are a newb: it ignores your
.gdbinit and puts you in TUI mode.
However, you will sooner or later notice that TUI is crappy print on break Python scripts are the path of light, e.g. GDB dashboard.
In order to prevent our opinionated defaults get in the way of your perfect setup, use:
make GDB_EXPERT=y gdb-add
or the shortcut:
./t -G add
Even though GDB step debug can already disassemble instructions for us, it is sometimes useful to have the disassembly in a text file for further examination.
Disassemble all examples:
make -j `nproc` objdump
Disassemble one example:
Examine one disassembly:
less -p asm_main add.objdump
This jumps directly to
asm_main, which is what you likely want to see.
Disassembly is still useful even though we are writing assembly because the assembler can do some non-obvious magic that we want to understand.
Currently we build just Binutils from source, but use the host GCC to save time.
This could lead to incompatibilities, although we haven’t observed any so far.
crosstool-NG is a set of scripts that makes it easy to obtain a cross compiled GCC. Ideally we should track it here as a submodule and automate from there.
You can build the toolchain with crosstool-NG as explained at: https://stackoverflow.com/revisions/51310756/6
Then run this repo with:
make \ CTNG=crosstool-ng/.build/ct_prefix \ PREFIX=arm-cortex_a15-linux-gnueabihf \ test \ ;
If you don’t like reading on GitHub, the HTML documentation can be generated from the README with:
make doc xdg-open out/README.html
E.g., to pass
-static for an emulator that does not support dynamically linked executables like gem5:
Scalable Vector Extension.
aarch64 only, newer than [neon].
It is called Scalable because it does not specify the vector width! Therefore we don’t have to worry about new vector width instructions every few years! Hurray!
The instructions then allow implicitly tracking the loop index without knowing the actual vector length.
Added to QEMU use mode in 3.0.0.
TODO announcement date. Possibly 2017: https://alastairreid.github.io/papers/sve-ieee-micro-2017.pdf There is also a 2016 mention: https://community.arm.com/tools/hpc/b/hpc/posts/technology-update-the-scalable-vector-extension-sve-for-the-armv8-a-architecture
The Linux kernel shows
/proc/cpuinfo compatibility as
https://www.rico.cat/files/ICS18-gem5-sve-tutorial.pdf step by step of a complete code execution examples, the best initial tutorial so far
https://alastairreid.github.io/papers/sve-ieee-micro-2017.pdf paper with some nice few concrete examples, illustrations and rationale
[armarm8] A1.7 "ARMv8 architecture extensions" says:
SVE is an optional extension to ARMv8.2. That is, SVE requires the implementation of ARMv8.2.
A1.7.8 "The Scalable Vector Extension (SVE)": then says that only changes to the existing registers are described in that manual, and that you should look instead at the "ARM Architecture Reference Manual Supplement, The Scalable Vector Extension (SVE), for ARMv8-A."
We then download the zip from: https://developer.arm.com/docs/ddi0584/latest/arm-architecture-reference-manual-supplement-the-scalable-vector-extension-sve-for-armv8-a and it contains the PDF:
DDI0584A_d_SVE_supp_armv8A.pdf which we use here.
That document then describes the SVE instructions and registers.
qemu-arm-static is used for emulation on x86 hosts. It translates ARM to x86, and forwards system calls to the host kernel.
OS portability is achieved with the C standard library which makes system calls for us: this would in theory work in operating systems other than Linux if you port the build system to them.
Using the standard library also allows us to use its convenient functionality such as
printf formatting and
memcpy to check memory.
Non-OS portable examples will be clearly labeled with their OS.
These examples show how our infrastructure works:
We link all examples against a C program: main.c. Sample simplified commands:
arm-linux-gnueabihf-gcc -c -o 'main.o' 'main.c' arm-linux-gnueabihf-gcc -c -o 'sub.o' 'sub.S' arm-linux-gnueabihf-gcc -o 'sub.out' 'sub.o' main.o
The C driver then just calls
asm_main, which each
.S example implements.
This allows us to easily use the C standard library portably: from the point of view of GCC, everything looks like a regular C program, which does the required glibc initialization before
git -C qemu pull make -B -C v7 qemu make -B -C v8 qemu
If the build fails due to drastic QEMU changes, first do:
Then make sure that the tests still pass:
This tutorial only covers userland concepts.
However, certain instructions can only be used in higher privilege levels from an operating system itself.
Here is a base setup ARM programming without an operating system, also known as "Bare Metal Programming": https://github.com/cirosantilli/linux-kernel-module-cheat/tree/7d6f8c3884a4b4170aa274b986caae55b1bebaaf#baremetal-setup
clean crosstool-NG build for GCC
C standard library powevered by Newlib
works on both QEMU and gem5
Here are further links:
-M virtUART bare metal hello world, nothing else, just works
https://github.com/bravegnu/gnu-eprog Not tested.
https://github.com/cirosantilli/raspberry-pi-bare-metal-blinker minimal, but not very QEMU friendly however because hard to observe LED: https://raspberrypi.stackexchange.com/questions/56373/is-it-possible-to-get-the-state-of-the-leds-and-gpios-in-a-qemu-emulation-like-t
https://github.com/tukl-msd/gem5.bare-metal bare metal UART example. Tested with: https://stackoverflow.com/questions/43682311/uart-communication-in-gem5-with-arm-bare-metal/50983650#50983650
x86 bare metal tutorial at: https://github.com/cirosantilli/x86-bare-metal-examples