Skip to content

mm notes

Jump to bottom
Michael Ellerman edited this page Aug 26, 2022 · 8 revisions

Real Memory Limits

64-bit

Book3S

Since v4.20:

System RAM is limited to 512TB due to limitations of the percpu allocator.

However the linear mapping can extend up to 2PB, to support nvdimm.

See commit 4ffe713b7587 ("powerpc/mm: Increase the max addressable memory to 2PB") for more detail.

MMU Page size RAM Limit Limit Notes
Radix 64K 512TB 2PB #define R_MAX_PHYSMEM_BITS 51
Radix 4K 512TB 2PB #define R_MAX_PHYSMEM_BITS 51
Hash 64K 512TB 2PB #define H_MAX_PHYSMEM_BITS 51
Hash 4K 61TB 61TB See 7746406baa3b ("powerpc/book3s64/hash/4k: Support large linear mapping range with 4K")

Note that on some systems memory on the second NUMA node will appear at a high address.

For example on Power9 memory on the second node begins at 32TB. So a system with 4GB on each node would have memory at 0-4GB and 32768GB-32772GB.

Because RAM is mapped 1:1 in the linear mapping that gap will be reflected in the linear mapping, meaning kernel code will see addresses 32TB above PAGE_OFFSET.

Address Ranges

64-bit

Book3S

Radix and Hash 64K

Start Address End Address Size Use Notes
0x0000000000000000 0x00007fffffffffff 128TB Default mmap space DEFAULT_MAP_WINDOW_USER64
0x0000000000000000 0x000fffffffffffff 4PB Userspace TASK_SIZE_4PB
0x5deadbeef0000000 ±~5XB Illegal pointer value CONFIG_ILLEGAL_POINTER_VALUE
0xc000000000000000 0xc001ffffffffffff 512TB linear mapping of RAM 1:1, PAGE_OFFSET
0xc002000000000000 0xc007ffffffffffff 1.5PB linear mapping non-RAM 1:1
0xc008000000000000 0xc009ffffffffffff 512TB vmalloc, modules Dynamic, H_KERN_VIRT_START, RADIX_KERN_VIRT_START, RADIX_VMALLOC_START
0xc00a000000000000 0xc00bffffffffffff 512TB ioremap Dynamic, RADIX_KERN_IO_START, H_KERN_IO_START
0xc00c000000000000 0xc00dffffffffffff 512TB vmemmap Dynamic, RADIX_VMEMMAP_START, H_VMEMMAP_START
0xc00e000000000000 0xffffffffffffffff not used out of page table range

See:

Radix 4K

Start Address End Address Size Use Notes
0x0000000000000000 0x00003fffffffffff 64TB Userspace TASK_SIZE_64TB
... as for 64K

Hash 4K

Start Address End Address Size Use Notes
0x0000000000000000 0x00003fffffffffff 64TB Userspace TASK_SIZE_64TB
0x5deadbeef0000000 ±~5XB Illegal pointer value CONFIG_ILLEGAL_POINTER_VALUE
0xc000000000000000 0xc0003cffffffffff 61TB linear mapping of RAM 1:1, PAGE_OFFSET
0xc0003d0000000000 0xc0003dffffffffff 1TB vmalloc, modules Dynamic, H_KERN_VIRT_START
0xc0003e0000000000 0xc0003effffffffff 1TB ioremap Dynamic, H_KERN_IO_START
0xc0003f0000000000 0xc0003fffffffffff 1TB vmemmap Dynamic, H_VMEMMAP_START
0xc000400000000000 0xffffffffffffffff not used

See:

Book3E

Start Address End Address Size Use Notes
0x0000000000000000 0x00003fffffffffff 64TB Userspace TASK_SIZE_64TB
0x5deadbeef0000000 ~ -5XB, +2XB Illegal pointer value CONFIG_ILLEGAL_POINTER_VALUE
0x8000000000000000 0x800003ffffffffff 4TB vmalloc, modules KERN_VIRT_START
0x8000040000000000 0x800007ffffffffff 4TB vmemmap
0x8000080000000000 0x80000bfffdffffff 4TB-32MB ioremap
0x80000bfffe000000 0x80000bffffffffff 32MB fixmap
0x80000c0000000000 0xbfffffffffffffff ~3XB not used
0xc000000000000000 0xc0000fffffffffff 16TB linear mapping of RAM 1:1, PAGE_OFFSET
0xc000100000000000 0xffffffffffffffff not used

See:

  • arch/powerpc/include/asm/nohash/64/pgtable.h
  • arch/powerpc/include/asm/nohash/mmu-book3e.h

Page Table Geometry

This defines the size of the user virtual address space.

64-bit

Book3S

See also MM History.

Hash

64K
#define H_PTE_INDEX_SIZE   8  // size: 8B <<  8 = 2KB, maps 2^8  x 64KB = 16MB
#define H_PMD_INDEX_SIZE  10  // size: 8B << 10 = 8KB, maps 2^10 x 16MB = 16GB
#define H_PUD_INDEX_SIZE  10  // size: 8B << 10 = 8KB, maps 2^10 x 16GB = 16TB
#define H_PGD_INDEX_SIZE   8  // size: 8B <<  8 = 2KB, maps 2^8  x 16TB =  4PB
4K
#define H_PTE_INDEX_SIZE  9  // size: 8B << 9 = 4KB, maps: 2^9 x   4KB =   2MB
#define H_PMD_INDEX_SIZE  7  // size: 8B << 7 = 1KB, maps: 2^7 x   2MB = 256MB
#define H_PUD_INDEX_SIZE  9  // size: 8B << 9 = 4KB, maps: 2^9 x 256MB = 128GB
#define H_PGD_INDEX_SIZE  9  // size: 8B << 9 = 4KB, maps: 2^9 x 128GB =  64TB

Radix

64K
#define RADIX_PTE_INDEX_SIZE   5  // size: 8B <<  5 = 256B, maps 2^5  x   64K =   2MB
#define RADIX_PMD_INDEX_SIZE   9  // size: 8B <<  9 =  4KB, maps 2^9  x   2MB =   1GB
#define RADIX_PUD_INDEX_SIZE   9  // size: 8B <<  9 =  4KB, maps 2^9  x   1GB = 512GB
#define RADIX_PGD_INDEX_SIZE  13  // size: 8B << 13 = 64KB, maps 2^13 x 512GB =   4PB
4K
#define RADIX_PTE_INDEX_SIZE   9  // size: 8B <<  9 =  4KB, maps 2^9  x    4K =   2MB
#define RADIX_PMD_INDEX_SIZE   9  // size: 8B <<  9 =  4KB, maps 2^9  x   2MB =   1GB
#define RADIX_PUD_INDEX_SIZE   9  // size: 8B <<  9 =  4KB, maps 2^9  x   1GB = 512GB
#define RADIX_PGD_INDEX_SIZE  13  // size: 8B << 13 = 64KB, maps 2^13 x 512GB =   4PB

8xx

Some details from Christophe: https://lore.kernel.org/linuxppc-dev/badfcf58-9fcb-6189-c9db-e8429f88799e@c-s.fr/

Christophe on 8xx page table layout: https://lore.kernel.org/linuxppc-dev/16ad8cab-08e2-27a7-6803-baadc6b8721b@csgroup.eu

Hash MMU voodoo

See this email from Aneesh, which refers to this commit.

Clone this wiki locally