Set CONFIG_VMSPLIT_3G=y in kernel configuration will make the initramfs data lose

[brobwind]

Hi, I am trying to make the newest Brillo system run on the Raspberry Pi 2 Model B, In the Brillo source code master branch, it suggests add CONFIG_VMSPLIT_3G=y in the kernel configuration(https://android.googlesource.com/device/generic/brillo/+/5b23ce3ee5fecc9f40dad82b03990499237ba686%5E%21/#F0), but while I am trying to do this, the initramfs data in the main memory will lose(I need a initramfs file to boot):
The kernel have such messages:

[    0.000000] Booting Linux on physical CPU 0xf00
[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Initializing cgroup subsys cpuacct
[    0.000000] Linux version 4.4.21-v7+ (hzak@B85RPI) (gcc version 4.9.x 20150123 (prerelease) (GCC) ) #2 SMP Sat Sep 17 16:07:49 CST 2016
[    0.000000] CPU: ARMv7 Processor [410fc075] revision 5 (ARMv7), cr=10c5387d
[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
[    0.000000] Machine model: Raspberry Pi 2 Model B Rev 1.1
[    0.000000] Truncating RAM at 0x00000000-0x3b000000 to -0x30000000
[    0.000000] Consider using a HIGHMEM enabled kernel.
[    0.000000] INITRD: 0x3aed7000+0x00118f60 is not a memory region - disabling initrd
[    0.000000] cma: Reserved 8 MiB at 0x2f400000
[    0.000000] Memory policy: Data cache writealloc

From these logs, I known:

• The RAM is truncated to 0x30000000
• The initrd has been disabled due to locate in 0x3aed700+0x00118f60
• To enable HIGHMEM configuration

But while I trying to enable CONFIG_HIGHMEM in kernel, it will cause another kernel crash:

[    0.000000] Booting Linux on physical CPU 0xf00
[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Initializing cgroup subsys cpuacct
[    0.000000] Linux version 4.4.21-v7+ (hzak@B85RPI) (gcc version 4.9.x 20150123 (prerelease) (GCC) ) #2 SMP Sun Sep 18 14:11:00 CST 2016
[    0.000000] CPU: ARMv7 Processor [410fc075] revision 5 (ARMv7), cr=10c5387d
[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
[    0.000000] Machine model: Raspberry Pi 2 Model B Rev 1.1
[    0.000000] cma: Reserved 8 MiB at 0x3a400000
[    0.000000] Memory policy: Data cache writealloc
[    0.000000] [bcm2709_smp_init_cpus] enter (101620->f3003010)
[    0.000000] [bcm2709_smp_init_cpus] ncores=4
[    0.000000] PERCPU: Embedded 11 pages/cpu @ef768000 s23104 r0 d21952 u45056
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 239936
[    0.000000] Kernel command line: dma.dmachans=0x7f35 bcm2708_fb.fbwidth=656 bcm2708_fb.fbheight=416 bcm2709.boardrev=0xa21041 bcm2709.serial=0x35d1198a smsc95xx.macaddr=B8:27:EB:D1:19:8A bcm2708_fb.fbswap=1 bcm2709.uart_clock=48000000 bcm2709.disk_led_gpio=47 bcm2709.disk_led_active_low=0 vc_mem.mem_base=0x3dc00000 vc_mem.mem_size=0x3f000000  dwc_otg.lpm_enable=0 console=ttyAMA0,115200 rootwait noinitrd init=/init elevator=deadline androidboot.hardware=rpi androidboot.selinux=permissive androidboot.disk.boot=/dev/block/mmcblk0p1 androidboot.disk.system=/dev/null androidboot.disk.data=/dev/null androidboot.mode=recovery initcall_debug=1
[    0.000000] PID hash table entries: 4096 (order: 2, 16384 bytes)
[    0.000000] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
[    0.000000] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
[    0.000000] Memory: 933952K/966656K available (9865K kernel code, 369K rwdata, 1368K rodata, 1024K init, 906K bss, 24512K reserved, 8192K cma-reserved, 171104K highmem)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
[    0.000000]     fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
[    0.000000]     vmalloc : 0xf0800000 - 0xff800000   ( 240 MB)
[    0.000000]     lowmem  : 0xc0000000 - 0xf0000000   ( 768 MB)
[    0.000000]     pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
[    0.000000]       .text : 0xc0008000 - 0xc0bf86c8   (12226 kB)
[    0.000000]       .init : 0xc0c00000 - 0xc0d00000   (1024 kB)
[    0.000000]       .data : 0xc0d00000 - 0xc0d5c450   ( 370 kB)
[    0.000000]        .bss : 0xc0dc70dc - 0xc0ea99d4   ( 907 kB)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000]  Build-time adjustment of leaf fanout to 32.
[    0.000000] NR_IRQS:16 nr_irqs:16 16
[    0.000000] Architected cp15 timer(s) running at 19.20MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x46d987e47, max_idle_ns: 440795202767 ns
[    0.000012] sched_clock: 56 bits at 19MHz, resolution 52ns, wraps every 4398046511078ns
[    0.000032] Switching to timer-based delay loop, resolution 52ns
[    0.000362] Console: colour dummy device 80x30
[    0.000412] Calibrating delay loop (skipped), value calculated using timer frequency.. 38.40 BogoMIPS (lpj=192000)
[    0.000440] pid_max: default: 32768 minimum: 301
[    0.000600] Security Framework initialized
[    0.000622] SELinux:  Initializing.
[    0.000983] Mount-cache hash table entries: 2048 (order: 1, 8192 bytes)
[    0.001006] Mountpoint-cache hash table entries: 2048 (order: 1, 8192 bytes)
[    0.002471] Disabling cpuset control group subsystem
[    0.002525] Initializing cgroup subsys io
[    0.002564] Initializing cgroup subsys memory
[    0.002630] Initializing cgroup subsys devices
[    0.002660] Initializing cgroup subsys freezer
[    0.002688] Initializing cgroup subsys debug
[    0.002770] CPU: Testing write buffer coherency: ok
[    0.002868] ftrace: allocating 27560 entries in 81 pages
[    0.080171] CPU0: update cpu_capacity 1024
[    0.080215] CPU0: thread -1, cpu 0, socket 15, mpidr 80000f00
[    0.080234] [bcm2709_smp_prepare_cpus] enter
[    0.080516] Setting up static identity map for 0x100000 - 0x100058
[    0.083765] [bcm2709_boot_secondary] cpu:1 started (0) 18
[    0.084130] [bcm2709_secondary_init] enter cpu:1
[    0.084195] CPU1: update cpu_capacity 1024
[    0.084204] CPU1: thread -1, cpu 1, socket 15, mpidr 80000f01
[    0.084917] [bcm2709_boot_secondary] cpu:2 started (0) 18
[    0.085203] [bcm2709_secondary_init] enter cpu:2
[    0.085238] CPU2: update cpu_capacity 1024
[    0.085247] CPU2: thread -1, cpu 2, socket 15, mpidr 80000f02
[    0.085916] [bcm2709_boot_secondary] cpu:3 started (0) 18
[    0.086169] [bcm2709_secondary_init] enter cpu:3
[    0.086203] CPU3: update cpu_capacity 1024
[    0.086211] CPU3: thread -1, cpu 3, socket 15, mpidr 80000f03
[    0.086311] Brought up 4 CPUs
[    0.086343] SMP: Total of 4 processors activated (153.60 BogoMIPS).
[    0.086354] CPU: All CPU(s) started in HYP mode.
[    0.086364] CPU: Virtualization extensions available.
[    0.087582] devtmpfs: initialized
[    0.103277] VFP support v0.3: implementor 41 architecture 2 part 30 variant 7 rev 5
[    0.104199] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.106044] pinctrl core: initialized pinctrl subsystem
[    0.107094] NET: Registered protocol family 16
[    0.115069] DMA: preallocated 4096 KiB pool for atomic coherent allocations
[    0.125160] hw-breakpoint: found 5 (+1 reserved) breakpoint and 4 watchpoint registers.
[    0.125173] hw-breakpoint: maximum watchpoint size is 8 bytes.
[    0.125591] Serial: AMBA PL011 UART driver
[    0.125927] 3f201000.uart: ttyAMA0 at MMIO 0x3f201000 (irq = 87, base_baud = 0) is a PL011 rev2
[    0.648163] console [ttyAMA0] enabled
[    0.652678] bcm2835-mbox 3f00b880.mailbox: mailbox enabled
[    0.751414] SCSI subsystem initialized
[    0.755689] usbcore: registered new interface driver usbfs
[    0.761314] usbcore: registered new interface driver hub
[    0.766773] usbcore: registered new device driver usb
[    0.781935] raspberrypi-firmware soc:firmware: Attached to firmware from 2016-09-14 19:56
[    0.816084] Advanced Linux Sound Architecture Driver Initialized.
[    0.826651] clocksource: Switched to clocksource arch_sys_counter
[    0.919404] FS-Cache: Loaded
[    0.922763] CacheFiles: Loaded
[    0.945075] NET: Registered protocol family 2
[    0.950555] TCP established hash table entries: 8192 (order: 3, 32768 bytes)
[    0.957792] TCP bind hash table entries: 8192 (order: 4, 65536 bytes)
[    0.964418] TCP: Hash tables configured (established 8192 bind 8192)
[    0.970938] UDP hash table entries: 512 (order: 2, 16384 bytes)
[    0.976948] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes)
[    0.983755] NET: Registered protocol family 1
[    0.988846] RPC: Registered named UNIX socket transport module.
[    0.994779] RPC: Registered udp transport module.
[    0.999524] RPC: Registered tcp transport module.
[    1.004230] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    1.011187] Trying to unpack rootfs image as initramfs...
[    1.016610] Unable to handle kernel paging request at virtual address faf08000
[    1.023833] pgd = c0004000
[    1.026539] [faf08000] *pgd=00000000
[    1.030132] Internal error: Oops: 5 [#1] SMP ARM
[    1.034759] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.4.21-v7+ #2
[    1.041025] Hardware name: BCM2709
[    1.044429] task: ef0b8000 ti: ef0a6000 task.ti: ef0a6000
[    1.049835] PC is at unpack_to_rootfs+0x17c/0x3cc
[    1.054538] LR is at 0x7
[    1.057076] pc : [<c0c033d0>]    lr : [<00000007>]    psr: 20000013
[    1.057076] sp : ef0a7d88  ip : 00000000  fp : ef0a7e14
[    1.068557] r10: c0c768c8  r9 : 00000000  r8 : 2d8bc2ae
[    1.073784] r7 : c0c768c8  r6 : 000e74ee  r5 : faf08000  r4 : c0c768c8
[    1.080312] r3 : 00000001  r2 : 00000000  r1 : c0c76954  r0 : ef338000
[    1.086843] Flags: nzCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
[    1.093980] Control: 10c5387d  Table: 0000406a  DAC: 00000051
[    1.099728] Process swapper/0 (pid: 1, stack limit = 0xef0a6210)
[    1.105736] Stack: (0xef0a7d88 to 0xef0a8000)
[    1.110100] 7d80:                   00000000 c0dc7154 00000000 00000000 00000000 00000000
[    1.118288] 7da0: c0ac7c80 c0c02cf4 c0dc7154 c0c04088 c0ac7c94 c0ac7cbc c0ac7cbc c0c768c8
[    1.126475] 7dc0: c0dc7154 c0c768c8 c0d034cc c0c76954 c0ac7cf8 ef0a7e0c ef0a7e04 ef0a7de8
[    1.134662] 7de0: c021eebc dc8ba615 ef0a7e0c c0dc7150 c0dc714c c0100000 00000000 2d8bc2ae
[    1.142849] 7e00: 00000000 00000000 ef0a7ea4 ef0a7e18 c0c037d8 c0c03260 ffffffff dc8ba615
[    1.151036] 7e20: c0c9ee70 c0c9ede8 c0d034cc 00000006 00000010 c0d034cc ef0a7e54 ef0a7e48
[    1.159223] 7e40: c06f2594 c06f2250 ef0a7ea4 ef0a7e58 c019655c c06f257c c01788a8 c0ded7c0
[    1.167411] 7e60: ffee6c00 002d4cad 00000000 00000000 ffffffff dc8ba615 ef0a7e9c ef2d1d80
[    1.175598] 7e80: c0c0373c ef0a7eb8 c0c76010 2d8bc2ae 00000000 00000000 ef0a7f34 ef0a7ea8
[    1.183785] 7ea0: c0c010c4 c0c03748 0000556f 00000000 ef0a7edc c0d034cc 2d8bc2ae 00000000
[    1.191972] 7ec0: c0141810 c0141700 efffb862 00000000 ef0a7f34 ef0a7ee0 c0141a24 c01417ec
[    1.200158] 7ee0: c0ad2670 c0acd340 c0acd320 c0acd36c c0acd2c4 00000005 00000005 00000000
[    1.208345] 7f00: c0b538e8 dc8ba615 00000001 c0c9e984 00000005 c0c76430 c0b538e8 00000000
[    1.216533] 7f20: c0c00680 c0dc7100 ef0a7f8c ef0a7f38 c0c01470 c0c00fd8 00000005 00000005
[    1.224720] 7f40: 00000000 c0c00680 00000156 c0c76000 c0dc7120 c0d034cc aaaaaaaa dc8ba615
[    1.232906] 7f60: aaaaaaaa ff7460f8 c09b9edc 00000000 00000000 00000000 00000000 00000000
[    1.241093] 7f80: ef0a7fac ef0a7f90 c09b9ef8 c0c0121c ef0a6000 00000000 c09b9edc 00000000
[    1.249279] 7fa0: 00000000 ef0a7fb0 c0108390 c09b9ee8 00000000 00000000 00000000 00000000
[    1.257465] 7fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[    1.265651] 7fe0: 00000000 00000000 00000000 00000000 00000013 00000000 aaaaaaaa aaaaaaaa
[    1.273853] [<c0c033d0>] (unpack_to_rootfs) from [<c0c037d8>] (populate_rootfs+0x9c/0x32c)
[    1.282130] [<c0c037d8>] (populate_rootfs) from [<c0c010c4>] (do_one_initcall+0xf8/0x230)
[    1.290319] [<c0c010c4>] (do_one_initcall) from [<c0c01470>] (kernel_init_freeable+0x260/0x36c)
[    1.299030] [<c0c01470>] (kernel_init_freeable) from [<c09b9ef8>] (kernel_init+0x1c/0x184)
[    1.307311] [<c09b9ef8>] (kernel_init) from [<c0108390>] (ret_from_fork+0x14/0x24)
[    1.314891] Code: e3520000 13a03000 e3530000 0a000056 (e5d53000) 
[    1.321018] ---[ end trace 47a32b28cd16ae4c ]---
[    1.325637] Kernel panic - not syncing: Fatal exception
[    1.330872] CPU3: stopping
[    1.333591] CPU: 3 PID: 0 Comm: swapper/3 Tainted: G      D         4.4.21-v7+ #2
[    1.341073] Hardware name: BCM2709
[    1.344495] [<c0111f80>] (unwind_backtrace) from [<c010d174>] (show_stack+0x20/0x24)
[    1.352254] [<c010d174>] (show_stack) from [<c04a385c>] (dump_stack+0xc8/0x120)
[    1.359576] [<c04a385c>] (dump_stack) from [<c010fe5c>] (handle_IPI+0x2c8/0x31c)
[    1.366984] [<c010fe5c>] (handle_IPI) from [<c01015c4>] (bcm2836_arm_irqchip_handle_irq+0x7c/0xbc)
[    1.375954] [<c01015c4>] (bcm2836_arm_irqchip_handle_irq) from [<c010df38>] (__irq_svc+0x58/0x78)
[    1.384827] Exception stack(0xef13df30 to 0xef13df78)
[    1.389883] df20:                                     00000000 ef789388 c0d044b4 00000000
[    1.398070] df40: c0d0359c c0100000 c0d586e4 c0d9bb48 c0d0359c c0c9f364 00000000 ef13df8c
[    1.406254] df60: ef13df80 ef13df80 c0108e80 c0108e84 60000013 ffffffff
[    1.412880] [<c010df38>] (__irq_svc) from [<c0108e84>] (arch_cpu_idle+0x34/0x54)
[    1.420292] [<c0108e84>] (arch_cpu_idle) from [<c0168aec>] (default_idle_call+0x34/0x48)
[    1.428398] [<c0168aec>] (default_idle_call) from [<c0168d8c>] (cpu_startup_entry+0x28c/0x338)
[    1.437023] [<c0168d8c>] (cpu_startup_entry) from [<c010f88c>] (secondary_start_kernel+0x16c/0x194)
[    1.446078] [<c010f88c>] (secondary_start_kernel) from [<001016ac>] (0x1016ac)
[    1.453303] CPU1: stopping
[    1.456020] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G      D         4.4.21-v7+ #2
[    1.463502] Hardware name: BCM2709
[    1.466921] [<c0111f80>] (unwind_backtrace) from [<c010d174>] (show_stack+0x20/0x24)
[    1.474678] [<c010d174>] (show_stack) from [<c04a385c>] (dump_stack+0xc8/0x120)
[    1.481999] [<c04a385c>] (dump_stack) from [<c010fe5c>] (handle_IPI+0x2c8/0x31c)
[    1.489407] [<c010fe5c>] (handle_IPI) from [<c01015c4>] (bcm2836_arm_irqchip_handle_irq+0x7c/0xbc)
[    1.498376] [<c01015c4>] (bcm2836_arm_irqchip_handle_irq) from [<c010df38>] (__irq_svc+0x58/0x78)
[    1.507249] Exception stack(0xef139f30 to 0xef139f78)
[    1.512305] 9f20:                                     00000000 ef773388 c0d044b4 00000000
[    1.520492] 9f40: c0d0359c c0100000 c0d586e4 c0d9bb48 c0d0359c c0c9f364 00000000 ef139f8c
[    1.528677] 9f60: ef139f80 ef139f80 c0108e80 c0108e84 60000013 ffffffff
[    1.535302] [<c010df38>] (__irq_svc) from [<c0108e84>] (arch_cpu_idle+0x34/0x54)
[    1.542711] [<c0108e84>] (arch_cpu_idle) from [<c0168aec>] (default_idle_call+0x34/0x48)
[    1.550816] [<c0168aec>] (default_idle_call) from [<c0168d8c>] (cpu_startup_entry+0x28c/0x338)
[    1.559441] [<c0168d8c>] (cpu_startup_entry) from [<c010f88c>] (secondary_start_kernel+0x16c/0x194)
[    1.568495] [<c010f88c>] (secondary_start_kernel) from [<001016ac>] (0x1016ac)
[    1.575719] CPU2: stopping
[    1.578436] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G      D         4.4.21-v7+ #2
[    1.585917] Hardware name: BCM2709
[    1.589337] [<c0111f80>] (unwind_backtrace) from [<c010d174>] (show_stack+0x20/0x24)
[    1.597093] [<c010d174>] (show_stack) from [<c04a385c>] (dump_stack+0xc8/0x120)
[    1.604413] [<c04a385c>] (dump_stack) from [<c010fe5c>] (handle_IPI+0x2c8/0x31c)
[    1.611821] [<c010fe5c>] (handle_IPI) from [<c01015c4>] (bcm2836_arm_irqchip_handle_irq+0x7c/0xbc)
[    1.620790] [<c01015c4>] (bcm2836_arm_irqchip_handle_irq) from [<c010df38>] (__irq_svc+0x58/0x78)
[    1.629663] Exception stack(0xef13bf30 to 0xef13bf78)
[    1.634720] bf20:                                     00000000 ef77e388 c0d044b4 00000000
[    1.642907] bf40: c0d0359c c0100000 c0d586e4 c0d9bb48 c0d0359c c0c9f364 00000000 ef13bf8c
[    1.651091] bf60: ef13bf80 ef13bf80 c0108e80 c0108e84 60000013 ffffffff
[    1.657716] [<c010df38>] (__irq_svc) from [<c0108e84>] (arch_cpu_idle+0x34/0x54)
[    1.665126] [<c0108e84>] (arch_cpu_idle) from [<c0168aec>] (default_idle_call+0x34/0x48)
[    1.673230] [<c0168aec>] (default_idle_call) from [<c0168d8c>] (cpu_startup_entry+0x28c/0x338)
[    1.681854] [<c0168d8c>] (cpu_startup_entry) from [<c010f88c>] (secondary_start_kernel+0x16c/0x194)
[    1.690908] [<c010f88c>] (secondary_start_kernel) from [<001016ac>] (0x1016ac)hzak@B85RPI:~$

NOTE:

• firmware version: 8979042
• kernel version: 2d31cd5
• initramfs configured in boot partition's config.txt

Thanks!

[pelwell]

The firmware is expecting the kernel to be configured with a 2+2 split so that it can address all available RAM without using HIGHMEM (which we couldn't get to work). Even with HIGHMEM enabled I'm not sure it will work early enough in the boot for initramfs.

Try using the ramfsaddr config.txt setting to load the initramfs file to a lower address - 0x2f000000 or lower ought to work, e.g. ramfsaddr=0x2f000000, or even lower ramfsaddr=0x10000000.

[brobwind]

Yes, Without using HIGHMEM, it does work when set ramfsaddr=0x2f000000 or ramfsaddr=0x10000000, or using initramfs ramdisk7.img 0x2f000000. I think it's a little dangerous since I don't known the kernel loading address(loading rule: the default loading address) and the reserved video memory address(the RAM layout seems not document yet).
I just hope this problem can be solved.
Thanks

[pelwell]

The kernel is always loaded to 0x8000, so setting the address as you have done is safe. However, it is a bit hacky.

As it turns out, we've already restricted the load address for the device tree blob to below 0x30000000 (unless hard coded by the user or the loader stub), and I don't see any reason why we can't do that for the initramfs blob as well - it's just that nobody's asked before.

[pelwell]

Can you try this test start.elf for me? It is hard-coded to have gpu_mem set to 128K, but that should be OK for testing.

To use it, make a copy of your existing /boot/start.elf then copy this one in its place and reboot, having first ensured that you haven't set start_x=1 or start_debug=1. If that boots OK for you, try removing the explicit initramfs address (use followkernel instead), then report back.

Assuming there are no problems I'll get the relevant patch into the next firmware release.

[brobwind]

Just let you known, configure kernel with VMSPLIT_3G and without HIGHMEM, and set initramfs to ramdisk7.img followkernel, the initramfs still locate above 0x30000000(with followkernel, the initramfs address should locate at the end of kernel?), here is the kernel message:

[    0.000000] Booting Linux on physical CPU 0xf00
[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Initializing cgroup subsys cpuacct
[    0.000000] Linux version 4.4.21-v7+ (hzak@B85RPI) (gcc version 4.9.x 20150123 (prerelease) (GCC) ) #1 SMP Tue Sep 20 22:16:55 CST 2016
[    0.000000] CPU: ARMv7 Processor [410fc075] revision 5 (ARMv7), cr=10c5387d
[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
[    0.000000] Machine model: Raspberry Pi 2 Model B Rev 1.1
**[    0.000000] Truncating RAM at 0x00000000-0x3b000000 to -0x30000000**
[    0.000000] Consider using a HIGHMEM enabled kernel.
**[    0.000000] INITRD: 0x3af08000+0x000e74ee is not a memory region - disabling initrd**
[    0.000000] cma: Reserved 8 MiB at 0x2f400000
[    0.000000] Memory policy: Data cache writealloc
[    0.000000] [bcm2709_smp_init_cpus] enter (101620->f3003010)
[    0.000000] [bcm2709_smp_init_cpus] ncores=4
[    0.000000] PERCPU: Embedded 11 pages/cpu @effb2000 s23104 r0 d21952 u45056
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 194880
[    0.000000] Kernel command line: dma.dmachans=0x7f35 bcm2708_fb.fbwidth=656 bcm2708_fb.fbheight=416 bcm2709.boardrev=0xa21041 bcm2709.serial=0x35d1198a smsc95xx.macaddr=B8:27:EB:D1:19:8A bcm2708_fb.fbswap=1 bcm2709.uart_clock=48000000 bcm2709.disk_led_gpio=47 bcm2709.disk_led_active_low=0 vc_mem.mem_base=0x3dc00000 vc_mem.mem_size=0x3f000000  dwc_otg.lpm_enable=0 console=ttyAMA0,115200 rootwait noinitrd init=/init elevator=deadline androidboot.hardware=rpi androidboot.selinux=permissive androidboot.disk.boot=/dev/block/mmcblk0p1 androidboot.disk.system=/dev/null androidboot.disk.data=/dev/null androidboot.mode=recovery initcall_debug=1
[    0.000000] PID hash table entries: 4096 (order: 2, 16384 bytes)
[    0.000000] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
[    0.000000] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
[    0.000000] Memory: 756248K/786432K available (9865K kernel code, 367K rwdata, 1368K rodata, 1024K init, 888K bss, 21992K reserved, 8192K cma-reserved)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
[    0.000000]     fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
[    0.000000]     vmalloc : 0xf0800000 - 0xff800000   ( 240 MB)
[    0.000000]     lowmem  : 0xc0000000 - 0xf0000000   ( 768 MB)
[    0.000000]       .text : 0xc0008000 - 0xc0bf84d4   (12226 kB)
[    0.000000]       .init : 0xc0c00000 - 0xc0d00000   (1024 kB)
[    0.000000]       .data : 0xc0d00000 - 0xc0d5bf90   ( 368 kB)
[    0.000000]        .bss : 0xc0dc709c - 0xc0ea5114   ( 889 kB)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000]  Build-time adjustment of leaf fanout to 32.
[    0.000000] NR_IRQS:16 nr_irqs:16 16

I will test the new start.elf later.
Thanks

[brobwind]

After using the new start.elf with VMSPLIT_3G and without HIGHMEM, the kernel seems can not boot(there is not kernel message from the serial port), with follow configurations:

• initramfs ramdisk7.img
• initramfs ramdisk7.img 0x2f000000
• initramfs ramdisk7.img followkernel

NOTE: I always do not use the mkknlimg script just copy the arch/arm/boot/zImage.
Thanks

[brobwind]

All related files can download from https://github.com/brobwind/rpi2b-kernel-issue
firmware version: 8979042 ( 2016-09-15 14:23:54)
boot/kernel7.img and kernel7.img-3g are the same and can boot with this firmware
replace the test start.elf can not boot
Thanks

[pelwell]

Thanks - that was helpful. It seems that loading just below 0x30000000 causes problems, so I've updated the test start.elf to load below 0x2f000000 instead. With that I get serial port output, but I think it then stalls because the rest of my SD card isn't correct.

Try the updated start.elf yourself - I hope it will work for you.

[brobwind]

It seems these two test start.elf are the same(with the same link address). And have the same sha1sum after downloaded, please help to check again.
Thanks

[pelwell]

The original file had the sha1sum of 90a57a705bdd99f14d859d9f94d1a424e948862f. The new one has a sha1sum of 0a2b6b08bf2ad95e1bb3d81fb8178d0b86a9d800. The link didn't change because I'm using the Google Drive versioning system.

[brobwind]

Sorry, my mistake. But the new start.elf(0a2b6b0) still can not boot the kernel, no serial port output. After replace with the old one (firmware @ 8979042), it has the serial port output.
Test with following configurations:

• initramfs ramdisk7.img 0x2f000000
• initramfs ramdisk7.img 0x10000000
• initramfs ramdisk7.img followkernel

[pelwell]

I started with a clean Raspbian installation, scratched the boot partition and copied your files in then overwrote the start.elf. For some reason it wasn't booting, except when started from our debugger. I've done a clean build, copied it to the card and it boots (to a point, with serial output) from cold, followkernel and an explit 0x2f000000. That version I've now uploaded (direct from the SD card) to the same URL.

If that doesn't at least get you some serial output then I'm stumped.

[brobwind]

Great job, it can boot now with following configurations:

• initramfs ramdisk7.img: initrd locate at 0x2ef18000 ~ 0x2efff4ee
• initramfs ramdisk7.img followkernel: initrd also locate at 0x2ef18000 ~ 0x2efff4ee (I am really confused, as you mentioned previously default kernel image start address is 0x8000, the initrd start address should not stay such high).

I also test with Raspberry Pi 3 Model B v1.2, it seems I still need add gpu=240 and enable_uart=1 as you mentioned on #1394 (comment) to make it boot.

Thanks

[pelwell]

One of the first things the kernel does is to read and internalise the Device Tree blob, after which it no longer needs the memory it occupied. The same happens with initramfs a short while later:

[    0.312126] Trying to unpack rootfs image as initramfs...
[    0.383790] Freeing initrd memory: 928K (eef18000 - ef000000)

So I don't think it matters where we load the DTB and initramfs to, provided it isn't too high (as we've seen) or so low that it collides with the end of the kernel.

With a Pi3 you will need enable_uart=1 if you want serial port output, but you don't need to set the gpu_mem so high any more. The reason it used to help is that it only left 768MB for the kernel, all of which is addressable as LOWMEM. (Yes, 1024-240 is not 768, but the top 16MB is effectively unusable, so the calculation is really 1008 - 240 = 768).

Now that the automatic DTB+initramfs placement is limited to the first 768MB you can run with any gpu_mem size you like, or at least you will once we've made a proper firmware release containing this change.

[brobwind]

You are right. Thank you very much