-
Notifications
You must be signed in to change notification settings - Fork 0
mm notes
Since v4.20:
System RAM is limited to 512TB due to limitations of the percpu allocator.
However the linear mapping can extend up to 2PB, to support nvdimm.
See commit 4ffe713b7587 ("powerpc/mm: Increase the max addressable memory to 2PB")
for more detail.
MMU | Page size | RAM Limit | Limit | Notes |
---|---|---|---|---|
Radix | 64K | 512TB | 2PB | #define R_MAX_PHYSMEM_BITS 51 |
Radix | 4K | 512TB | 2PB | #define R_MAX_PHYSMEM_BITS 51 |
Hash | 64K | 512TB | 2PB | #define H_MAX_PHYSMEM_BITS 51 |
Hash | 4K | 61TB | 61TB | See 7746406baa3b ("powerpc/book3s64/hash/4k: Support large linear mapping range with 4K")
|
Note that on some systems memory on the second NUMA node will appear at a high address.
For example on Power9 memory on the second node begins at 32TB. So a system with 4GB on each node would have memory at 0-4GB and 32768GB-32772GB.
Because RAM is mapped 1:1 in the linear mapping that gap will be reflected in the linear mapping, meaning kernel code will see addresses 32TB above PAGE_OFFSET
.
Start Address | End Address | Size | Use | Notes |
---|---|---|---|---|
0x0000000000000000 |
0x00007fffffffffff |
128TB | Default mmap space | DEFAULT_MAP_WINDOW_USER64 |
0x0000000000000000 |
0x000fffffffffffff |
4PB | Userspace | TASK_SIZE_4PB |
0x5deadbeef0000000 |
±~5XB | Illegal pointer value | CONFIG_ILLEGAL_POINTER_VALUE |
|
0xc000000000000000 |
0xc001ffffffffffff |
512TB | linear mapping of RAM | 1:1, PAGE_OFFSET
|
0xc002000000000000 |
0xc007ffffffffffff |
1.5PB | linear mapping non-RAM | 1:1 |
0xc008000000000000 |
0xc009ffffffffffff |
512TB | vmalloc, modules | Dynamic, H_KERN_VIRT_START , RADIX_KERN_VIRT_START , RADIX_VMALLOC_START
|
0xc00a000000000000 |
0xc00bffffffffffff |
512TB | ioremap | Dynamic, RADIX_KERN_IO_START , H_KERN_IO_START
|
0xc00c000000000000 |
0xc00dffffffffffff |
512TB | vmemmap | Dynamic, RADIX_VMEMMAP_START , H_VMEMMAP_START
|
0xc00e000000000000 |
0xffffffffffffffff |
not used | out of page table range |
See:
-
0034d395f89d ("powerpc/mm/hash64: Map all the kernel regions in the same 0xc range")
. arch/powerpc/include/asm/book3s/64/radix.h
arch/powerpc/include/asm/book3s/64/hash-64k.h
Start Address | End Address | Size | Use | Notes |
---|---|---|---|---|
0x0000000000000000 |
0x00003fffffffffff |
64TB | Userspace | TASK_SIZE_64TB |
... as for 64K |
Start Address | End Address | Size | Use | Notes |
---|---|---|---|---|
0x0000000000000000 |
0x00003fffffffffff |
64TB | Userspace | TASK_SIZE_64TB |
0x5deadbeef0000000 |
±~5XB | Illegal pointer value | CONFIG_ILLEGAL_POINTER_VALUE |
|
0xc000000000000000 |
0xc0003cffffffffff |
61TB | linear mapping of RAM | 1:1, PAGE_OFFSET
|
0xc0003d0000000000 |
0xc0003dffffffffff |
1TB | vmalloc, modules | Dynamic, H_KERN_VIRT_START
|
0xc0003e0000000000 |
0xc0003effffffffff |
1TB | ioremap | Dynamic, H_KERN_IO_START
|
0xc0003f0000000000 |
0xc0003fffffffffff |
1TB | vmemmap | Dynamic, H_VMEMMAP_START
|
0xc000400000000000 |
0xffffffffffffffff |
not used |
See:
-
7746406baa3b ("powerpc/book3s64/hash/4k: Support large linear mapping range with 4K")
. arch/powerpc/include/asm/book3s/64/hash-4k.h
Start Address | End Address | Size | Use | Notes |
---|---|---|---|---|
0x0000000000000000 |
0x00003fffffffffff |
64TB | Userspace | TASK_SIZE_64TB |
0x5deadbeef0000000 |
~ -5XB, +2XB | Illegal pointer value | CONFIG_ILLEGAL_POINTER_VALUE |
|
0x8000000000000000 |
0x800003ffffffffff |
4TB | vmalloc, modules | KERN_VIRT_START |
0x8000040000000000 |
0x800007ffffffffff |
4TB | vmemmap | |
0x8000080000000000 |
0x80000bfffdffffff |
4TB-32MB | ioremap | |
0x80000bfffe000000 |
0x80000bffffffffff |
32MB | fixmap | |
0x80000c0000000000 |
0xbfffffffffffffff |
~3XB | not used | |
0xc000000000000000 |
0xc0000fffffffffff |
16TB | linear mapping of RAM | 1:1, PAGE_OFFSET
|
0xc000100000000000 |
0xffffffffffffffff |
not used |
See:
arch/powerpc/include/asm/nohash/64/pgtable.h
arch/powerpc/include/asm/nohash/mmu-book3e.h
This defines the size of the user virtual address space.
#define H_PTE_INDEX_SIZE 8 // size: 8B << 8 = 2KB, maps 2^8 x 64KB = 16MB
#define H_PMD_INDEX_SIZE 10 // size: 8B << 10 = 8KB, maps 2^10 x 16MB = 16GB
#define H_PUD_INDEX_SIZE 10 // size: 8B << 10 = 8KB, maps 2^10 x 16GB = 16TB
#define H_PGD_INDEX_SIZE 8 // size: 8B << 8 = 2KB, maps 2^8 x 16TB = 4PB
#define H_PTE_INDEX_SIZE 9 // size: 8B << 9 = 4KB, maps: 2^9 x 4KB = 2MB
#define H_PMD_INDEX_SIZE 7 // size: 8B << 7 = 1KB, maps: 2^7 x 2MB = 256MB
#define H_PUD_INDEX_SIZE 9 // size: 8B << 9 = 4KB, maps: 2^9 x 256MB = 128GB
#define H_PGD_INDEX_SIZE 9 // size: 8B << 9 = 4KB, maps: 2^9 x 128GB = 64TB
#define RADIX_PTE_INDEX_SIZE 5 // size: 8B << 5 = 256B, maps 2^5 x 64K = 2MB
#define RADIX_PMD_INDEX_SIZE 9 // size: 8B << 9 = 4KB, maps 2^9 x 2MB = 1GB
#define RADIX_PUD_INDEX_SIZE 9 // size: 8B << 9 = 4KB, maps 2^9 x 1GB = 512GB
#define RADIX_PGD_INDEX_SIZE 13 // size: 8B << 13 = 64KB, maps 2^13 x 512GB = 4PB
#define RADIX_PTE_INDEX_SIZE 9 // size: 8B << 9 = 4KB, maps 2^9 x 4K = 2MB
#define RADIX_PMD_INDEX_SIZE 9 // size: 8B << 9 = 4KB, maps 2^9 x 2MB = 1GB
#define RADIX_PUD_INDEX_SIZE 9 // size: 8B << 9 = 4KB, maps 2^9 x 1GB = 512GB
#define RADIX_PGD_INDEX_SIZE 13 // size: 8B << 13 = 64KB, maps 2^13 x 512GB = 4PB
Some details from Christophe: https://lore.kernel.org/linuxppc-dev/badfcf58-9fcb-6189-c9db-e8429f88799e@c-s.fr/
Christophe on 8xx page table layout: https://lore.kernel.org/linuxppc-dev/16ad8cab-08e2-27a7-6803-baadc6b8721b@csgroup.eu
See this email from Aneesh, which refers to this commit.