Open Mesh lock down exploit
From the very beginning of this project we were focused on finding easy way* to bypass firmware lock down introduced in dual-band Open Mesh devices and thus allow their users to use open source software. We confirmed, also based on analyze of U-Boot sources provided by Open Mesh, that removing RSA public key stored in FLASH is enough to fully disable firmware lock down.
This can be actually done easily, directly in the vendor's firmware as the related mtd partition is writable. For users of devices located in USA or Canada this is still possible but slightly more difficult to achieve as it requires VPN service with IP outside these countries.
Fortunately, thanks to the vulnerability we found in the update related code in bootloader, we were able to develop some kind of ready exploit which unlocks affected Open Mesh devices (at the moment, only OM5P-AC v2). Below we try to explain how it works.
* at least without the requirement of opening the device and using some special tools like JTAG, FLASH programmer, etc.
Before we dive into details, there is one important thing you should be aware of. Almost all Qualcomm/Atheros Wireless SoCs (including the QCA955x series which Open Mesh dual-band devices use) based devices use very old and deeply modified U-Boot 1.1.4 version, released around the end of 2005. The reason for this is that SoC vendor SDK is based on this version and has never been updated to something more up to date - you can find the code in CodeAurora.
On QC/A WiSoC based devices, which boot directly from SPI NOR FLASH like the Open Mesh OM5P-AC, we can divide booting process into two parts. First one, often called low level, prepares things like clocks, caches, external RAM - this part is executed from FLASH. Then, when external memory is ready, U-Boot relocates itself to RAM, setups few things, including new stack and continues execution from there.
QC/A WiSoCs physical addressing space includes several separate elements, which among others include 256 MB of DDR range: 0x0000.0000~0x0FFF.FFFF which is translated into two virtual address space ranges: 0x8000.0000~0x8FFF.FFFF (cached) and 0xA000.0000~0xAFFF.FFFF (uncached).
On some devices, running U-Boot compiled with debug enabled, you can get detailed information about addresses used on top of the available memory on serial console:
U-Boot 1.1.4 (Sep 25 2016 - 12:59:29)
ap152 - Dragonfly 1.0
DRAM: 128 MB
Top of RAM usable for U-Boot at: 88000000
Reserving 200k for U-Boot at: 87fcc000
Reserving 192k for malloc() at: 87f9c000
Reserving 44 Bytes for Board Info at: 87f9bfd4
Reserving 36 Bytes for Global Data at: 87f9bfb0
Reserving 128k for boot params() at: 87f7bfb0
Stack Pointer at: 87f7bf98
Now running in RAM - U-Boot at: 87fcc000
[...]
Two most important facts here, which we are going to use later, are:
- U-Boot relocates itself to the top of available memory, leaving low address space usable for e.g. loading kernel image.
- The new stack is placed below and grows down (from higher to lower addresses).
Open Mesh devices uses custom flashit function in U-Boot:
[...]
preboot=flashit 0x80100000 fwupgrade.cfg
[...]
bootcmd=test -n "${preboot}" && run preboot; test -n "${bootseq}" || bootseq=1,2; run set_bootargs_1; run set_bootargs_2; boot "${bootseq}"
[...]
This function first loads file with name provided in second argument (fwupgrade.cfg here) into RAM at address (0x80100000 here) from first argument, parses it, downloads parts of firmware provided in downloaded configuration file and writes them to FLASH.
In locked down devices, before actually parsing the file, this function also tries to download and verify signature (fwupgrade.cfg.sig). Obviously, without access to private key we are not able to properly sign fwupgrade.cfg file.
Files are downloaded over TFTP mechanizm, same as in well known tftpboot command, we can see it in tftp_get helper function:
static int tftp_get (ulong loadaddr, char *filename)
{
int rcode = 0;
load_addr = loadaddr;
copy_filename (BootFile, filename, sizeof(BootFile));
if ((rcode = NetLoop (TFTPGET)) > 0) {
flush_cache (load_addr, rcode);
}
return rcode;
}Lets see now how does the RAM address map with marked important things look like:
We have here stack placed somewhere at the upper part of RAM and growing down. On the other side, parts of the upgrade data (fwupgrade.cfg, fwupgrade.cfg.sig, kernel and rootfs) are uploaded one by one, starting from address 0x80100000 and... growing up, as in real example (in fact, there is a mistake in the code and first part of the firmware, the kernel, is loaded 1 byte after end of fwupgrade.cfg, you can see it below):
[...]
Hit any key to stop autoboot: 2 1 0
Speed is 1000T
dup 1 speed 1000
Using eth0 device
TFTP from server 192.168.100.8; our IP address is 192.168.100.9
Filename 'fwupgrade.cfg'.
Load address: 0x80100000
Loading: *#
done
Bytes transferred = 640 (280 hex)
Speed is 1000T
Using eth0 device
TFTP from server 192.168.100.8; our IP address is 192.168.100.9
Filename 'fwupgrade.cfg.sig'.
Load address: 0x80100280
Loading: *#
done
Bytes transferred = 528 (210 hex)
Speed is 1000T
Using eth0 device
TFTP from server 192.168.100.8; our IP address is 192.168.100.9
Filename 'kernel'.
Load address: 0x80100281
Loading: *#################################################################
#################################################################
#################################################################
########################
done
Bytes transferred = 1117582 (110d8e hex)
Speed is 1000T
Using eth0 device
TFTP from server 192.168.100.8; our IP address is 192.168.100.9
Filename 'rootfs'.
Load address: 0x8021100f
Loading: *#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
#################################################################
##
done
Bytes transferred = 3670020 (380004 hex)
[...]
As some of you might have already guessed, the problem here is that the size of uploaded data over TFTP is not limited in any way. In simple words, if you send enough data over TFTP, at some point it will start overwriting the stack.
In best case scenario this will just trigger some CPU exception and it will hang. But in worst case (for the vendor of locked down device, not for us - the attackers, of course), with specially crafted data, you can make use of existing code and for example... jump to previously uploaded, custom version of U-Boot and do with device whatever you want. And this is exactly what we did.
In fact this is not a bug in Open Mesh code but rather a feature of U-Boot - even the mainline version does not include any way of limiting size of uploaded data over TFTP (and this is probably true also for other loading functions). In practice, this means that any board which contains some kind of automatic firmware upgrade mechanizm, built on top of tftpboot command, is vulnerable for the attack type we are showing.
After we confirmed that it is possible to upload any amount of data over TFTP and the router will accept it, idea for automatic unlocker was ready. The plan was to use custom U-Boot image which can be loaded and started directly from RAM and then perform the unlocking inside it: check if the RSA key is installed, backup it on TFTP server and at the end, remove it from FLASH.
Based on the sources for MR1750 we got from Open Mesh, we prepared a custom, RAM-loadable version of U-Boot for OM5P-AC v2, with a several changes in the code. For load address we chose the same one which fwupgrade.cfg is loaded at, during the ap51-flash driven upgrade: 0x80100000.
The only thing we were still missing at that point, was some way to switch from the vendor's U-Boot to our custom one. More precisely, we were looking for exact sequence of data which, after uploaded to the router, could allow us to execute a jump to 0x80100000 address instruction (as we were going to put our custom RAM-loadable image there).
Below you can find a list of major hardware and software tools used during the hacking part of the development of the exploit:
- 8devices Rambutan Development Kit
- JTAG-lock-pick Tiny 2 adapter
- EZP2010 USB SPI FLASH programmer
- DediProg SOK-SPI-8W SMT socket adapter
- OpenOCD 0.8.0
- Radare2, IDA
- Other (TFTP server, HEX editor, etc.)
We selected 8devices Rambutan board because from hardware point of view it is very similar to the OM5P-AC, offers easy access to many I/O signals, including (E)JTAG interface and happily, one of the available Ethernet interfaces was properly configured and worked with original Open Mesh U-Boot image. Moreover, development board offers also selection of booting device, including SPI NOR FLASH.
We did not need low level initialization, like clocks and RAM setup done with JTAG. All we needed was access to CPU registers and this can be achieved with just a basic, target configuration file:
set CHIPNAME qca955x
set TARGETNAME $CHIPNAME.cpu
jtag newtap $CHIPNAME cpu -irlen 5 -ircapture 0x1 -irmask 0x1f -expected-id 1
target create $TARGETNAME mips_m4k -endian big -chain-position $TARGETNAME
reset_config trst_and_srst
To be able to analyze locally code of running U-Boot we needed also relocation address, where U-Boot moves itself after low level initialization. With analyze of board_init_f (lib_mips/board.c) and relocate_code (cpu/mips/start.S) functions and a little bit of poking around in RAM, we found out that relocation address is 0x87fc8000.
At this point we knew and had ready:
- RAM-loadable (load address 0x80100000), custom U-Boot image with built-in unlocking code
- RAM address where factory U-Boot relocates itself: 0x87fc8000
- Original OM5P-AC v2 U-Boot binary dump
- 8devices Rambutan Development Kit with SPI NOR FLASH installed (with dump taken from OM5P-AC v2)
- Configured and connected JTAG, running TFTP server (at 192.168.100.8/24)
We decided to store our custom U-Boot image inside fake fwupgrade.cfg file, with its size set to 0x30000 (192 KB) and rest of the exploit inside fake fwupgrade.cfg.sig file.
In the result, our fwupgrade.cfg.sig was loaded by factory U-Boot at address 0x80130000 (0x80100000 + 0x30000) which gave around 126 MB (0x87fc8000 - 0x80130000) of gap between start of the file and factory U-Boot code. And somewhere in this gap, just below the U-Boot code was the stack content we wanted to overwrite.
We did not try to disassemble factory U-Boot image as most likely it would take us a lot of time and effort (U-Boot images are not that simple like for example ELF format, they usually contain parts written both in assembler and C, plus they are also stripped down as much as possible). This also means that we did not try to find other possible vulnerabilities (e.g. stack buffer overflow) in update related code. Probably more experienced vulnerabilities searchers would suggest a different and maybe a better approach.
So how did we actually find exact sequence of data needed to be uploaded over TFTP to make our exploit working? We used some kind of brute force approach and started with a 132743168 bytes fwupgrade.cfg.sig file, filled at the end (exactly last 1 MB) with random data (range 0x01~0xff) and with 0x0 in remaining area, hoping to trigger an exception in the CPU... And it worked - the factory U-Boot stalled after receiving exactly 132412928 bytes.
Lets take a look first at the value stored in EPC (Exception Program Counter, CP0 Register 14, Select 0) register:
> mips32 cp0 14 0
cp0 reg 14, select 0: 87fdf73c
And now the corresponding disassembled code near the 0x87fdf73c address:
[...]
87fdf738 lw $gp, 0x10($sp)
87fdf73c lw $v1, 0x118($gp) # <- here!
87fdf740 lw $v0, 0($v1)
87fdf744 sltu $v0, $s1
87fdf748 bnezl $v0, loc_87fdf750
87fdf74c sw $s1, 0($v1)
loc_87fdf750:
87fdf750 lw $t9, 0x210($gp)
87fdf754 addiu $t9, 0x6f80
87fdf758 jalr $t9 # <- target jump instruction
87fdf75c nop
[...]The $gp and $badvaddr registers values were:
> reg gp
gp (/32): 0xc01f5679
> reg badvaddr
badvaddr (/32): 0xc01f5791
With the marked instruction lw $v1, 0x118($gp) CPU was trying to load a word value from address stored in $gp + 0x118 (0xc01f5679 + 0x118 is 0xc01f5791, the value of $badvaddr) into $v1 register. Outside kernel mode, address from this range is not accessible.
As you can also see in the code above, we were lucky - target jump instruction (jalr $t9) we were looking for was just a few instruction down. All we needed to do was to prepare few values at exact offsets inside the fwupgrade.cfg.sig file which would lead to store value 0x80100000 (address where we uploaded our custom U-Boot image) inside $t9 register.
Lets analyze now the code starting from lw (load word) instruction at 0x87fdf738 (nothing before is important for us):
87fdf738 lw $gp, 0x10($sp) # $gp = value stored at address in $sp + 0x10
# $gp = value from 0x7e47768 offset in fwupgrade.cfg.sig
# $gp = 0x80130000We already knew that $gp value is taken from value at offset 0x7e47768 in our fwupgrade.cfg.sig file, so we did:
- put value 0x80130000 at 0x7e47768 offset in fwupgrade.cfg.sig
- put value 0x80130000 at 0x118 offset in fwupgrade.cfg.sig
- put value 0x800f9080 at 0x210 offset in fwupgrade.cfg.sig
- shrink the fwupgrade.cfg.sig file size to 132413292 bytes
Just a side note, the $sp (stack pointer) value during execution of above code is 0x87f77758. You can calculate it based on address of fwupgrade.cfg.sig in RAM, offset of $gp value in the file and above line of code: 0x80130000 + 0x7e47768 - 0x10 = 0x87f77758.
Now, rest of the code:
87fdf73c lw $v1, 0x118($gp) # $v1 = value stored at address in $gp + 0x118 (0x80130118)
# $v1 = value from 0x118 offset in fwupgrade.cfg.sig
# $v1 = 0x80130000
87fdf740 lw $v0, 0($v1) # $v0 = value stored at address in $v1 (0x80130000)
# $v0 = value from 0x0 offset in fwupgrade.cfg.sig
# $v0 = 0x0
87fdf744 sltu $v0, $s1 # $v0 < $s1 ? $v0 = 1 : $v0 = 0;
# $s1 at this point was 0x07e47800, so $v0 < $s1
# $v0 = 1
87fdf748 bnezl $v0, loc_87fdf750 # branch to 0x87fdf750 if $v != 0
87fdf74c sw $s1, 0($v1)Value inside $t9 is then calculated as below:
87fdf750 lw $t9, 0x210($gp) # $t9 = value stored at address in $gp + 0x210 (0x80130210)
# $t9 = value from 0x210 offset in fwupgrade.cfg.sig
# $t9 = 0x800f9080
87fdf754 addiu $t9, 0x6f80 # $t9 = $t9 + 0x6f80
# $t9 = 0x800f9080 + 0x6f80 = 0x80100000
87fdf758 jalr $t9 # jump to address stored in $t9 (0x80100000)And that was all, we made factory U-Boot jump to our custom one!