Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cannot access XIP from sram build #412

Closed
nonchip opened this issue May 16, 2021 · 30 comments
Closed

cannot access XIP from sram build #412

nonchip opened this issue May 16, 2021 · 30 comments

Comments

@nonchip
Copy link

nonchip commented May 16, 2021

while hardware_flash provides functions to write/erase the flash when running from sram (type no_flash), i'd need to both read from and execute into XIP, but sram builds don't (seem to) provide a way of enabling boot_stage2 and the sdk doesn't (seem to) provide any other function to initialize the flash for XIP.

manually adding boot_stage2 as a target_link_library fails too because then the linker complains about there not being a libboot_stage2.so

could you please provide a "load into and run from sram" mode that doesn't disable flash access (= includes boot_stage2)?

EDIT: apparently there seems to be some weirdness with flash_enter_cmd_xip while using usb CDC, but also please just add a "run from sram but with boot2" step, because "run from sram" does not always have to mean "no_flash"

Update

figured out the correct sequence for enabling XIP via boot2 (located inside XIP) thanks to @Wren6991: #412 (comment)

also I provide a full working "blink" example containing a neat header+source file you can use over there: https://gitlab.com/nonchip/pi-pico-xipram

@lurch
Copy link
Contributor

lurch commented May 16, 2021

Is the _flash_enter_cmd_xip bootrom function (detailed in section 2.8.2.1.3. of the RP2040 datasheet ) what you're looking for? 🤷‍♂️

@kilograham
Copy link
Contributor

there is copy_to_ram as well as no_flash

@nonchip
Copy link
Author

nonchip commented May 16, 2021

@lurch i thought so, and i tried it like this (with pico_bootrom included, and added as a target link library):

int main() {
  stdio_init_all();
  // copied verbatim from https://github.com/raspberrypi/pico-sdk/blob/master/src/rp2_common/hardware_flash/flash.c#L51
  void (*flash_enter_cmd_xip)(void) = (void(*)(void))rom_func_lookup(rom_table_code('C', 'X'));
  assert(flash_enter_cmd_xip);
  flash_enter_cmd_xip();
  // rom_funcs below is *not* the bootrom stuff but a const struct defined in the XIP .rodata, the address is automatically set via a linker script:
  while (true) {
    printf("rom_funcs* %x\n",&rom_funcs);
    printf("rom_funcs: %x\n",rom_funcs);
    printf("     init: %x\n",rom_funcs.init);
    printf("       on: %x\n",rom_funcs.on);
    printf("      off: %x\n",rom_funcs.off);
    printf("\n");
    sleep_ms(1000);
  }
}

but either the assertion fails or the function errors out (i don't have a SWD probe that can debug sadly), because the CPU locks up and doesn't even respond to USB keepalive packets anymore. before adding the xip/bootrom stuff the loop would actually run and print something (but every actual access, so everything except &rom_funcs which has its address supplied by the linker, returned invalid data)

@kilograham i know and that would be the exact opposite of what i need, given i don't want to overwrite the flash before accessing it.

@lurch
Copy link
Contributor

lurch commented May 16, 2021

(i don't have a SWD probe that can debug sadly)

You're probably already aware of this, but just in case you're not: You can use either a Raspberry Pi or a second Pico for this. Instructions in the Getting Started guide.

@nonchip
Copy link
Author

nonchip commented May 16, 2021

@lurch yeah i am but the only pi i have currently is hardwired into my 3d printer and i don't have a 2nd pico yet (currently stuck in shipping), so i was hoping anyone could tell me if i'm just calling that function wrong until it arrives

@nonchip
Copy link
Author

nonchip commented May 16, 2021

in case anyone wants to see the full code: https://gitlab.com/nonchip/pi-pico-xipram/-/blob/master/ram/ram.c is where i do the xip access, ../xip/xip.c is where things are declared, everything is supposed to be built using the top level cmakelists (kinda like the pico-examples), and if it worked it should be printing the addresses via usbcdc (and the commented out part would be the "cross-memory" equivalent of the official blink example) after loading both uf2s (the xip one resets into bootloader, so both after flashing it and after powering up it's ready for the ram one)

what actually happens is as soon as i load the ram uf2 it initializes cdc and hangs.

@kilograham
Copy link
Contributor

It seems like it should work, so I tried your code, and it works fine for me. I'm not sure what you're trying to do; the code in the bootrom will get you generic (slower speed) access to XIP.

The simplest thing to do might be to have a valid program in flash which just calls into RAM, then you can run boot stage2 by doing a watchdog reboot - note also it is possible to call the second stage boot as a function (you have to copy it into RAM at the right place first); i could swear there was an example that does this, but I cannot find it atm.

@kilograham
Copy link
Contributor

what actually happens is as soon as i load the ram uf2 it initializes cdc and hangs.

oh, i didn't read this - i have UART hooked up, and saw the right stuff there. might be worth trying it without a USB terminal hooked up to see if it makes a difference.

@nonchip
Copy link
Author

nonchip commented May 16, 2021

It seems like it should work, so I tried your code, and it works fine for me.

i just cleared my whole build folder and built it again just in case there was some old object not being built there, dmesg says:

[290103.139857] usb 1-12: device descriptor read/64, error -110   // first reboot after XIP flash
[290119.012858] usb 1-12: device descriptor read/64, error -110   // 2nd reboot after RAM loaded
[290119.236802] usb 1-12: new full-speed USB device number 64 using xhci_hcd // CDC initialized
[290134.883853] usb 1-12: device descriptor read/64, error -110  // immediately disconnects and stays dead

I'm not sure what you're trying to do; the code in the bootrom will get you generic (slower speed) access to XIP.

essentially what i was trying to do is have a bunch of low level common functionality i'll need for a "modular" kinda project in XIP, then load application specific code into RAM which accesses that to save on code duplication and flash destruction.

The simplest thing to do might be to have a valid program in flash which just calls into RAM

yeah that's what i thought first but then i'd have to implement the bootloader for the RAM code loading myself so i figured i'd rather have the program in flash reboot into the bootloader :/

@nonchip
Copy link
Author

nonchip commented May 16, 2021

might be worth trying it without a USB terminal hooked up to see if it makes a difference.

inspired by this i tried putting the stdio init below the XIP init stuff, now i get this:


[290537.027786] usb 1-12: new full-speed USB device number 70 using xhci_hcd
[290552.675829] usb 1-12: device descriptor read/64, error -110 // first reboot, XIP flashed
[290568.547832] usb 1-12: device descriptor read/64, error -110 // second reboot, RAM loaded
[290568.649835] usb usb1-port12: attempt power cycle // everything below this happened on its own
[290569.277786] usb 1-12: new full-speed USB device number 71 using xhci_hcd
[290574.103859] usb 1-12: Device not responding to setup address.
[290579.131986] usb 1-12: Device not responding to setup address.
[290579.337801] usb 1-12: device not accepting address 71, error -71
[290579.451800] usb 1-12: new full-speed USB device number 72 using xhci_hcd
[290584.278086] usb 1-12: Device not responding to setup address.
[290589.308189] usb 1-12: Device not responding to setup address.
[290589.513778] usb 1-12: device not accepting address 72, error -71
[290589.513832] usb usb1-port12: unable to enumerate USB device

note it's trying to load a driver, so apparently there's some USB traffic going on, but then just fails to do anything. without the XIP init code it simply prints fine

@kilograham
Copy link
Contributor

yeah using USB as a starting point is likely making life hard.... note for a start you are likely to be shooting yourself in the foot if your flash based "program" uses any static mutable data, as without extra work it is going to stomp on the data of the ram based program.

yeah that's what i thought first but then i'd have to implement the bootloader for the RAM code loading myself so i figured i'd rather have the program in flash reboot into the bootloader :/

Not sure what RAM bootloader you are talking about?

@nonchip
Copy link
Author

nonchip commented May 16, 2021

as without extra work it is going to stomp on the data of the ram based program

oh yeah that's something to keep in mind, i'd have to allocate any buffers in the ram program i guess

Not sure what RAM bootloader you are talking about?

well if i have code sitting in XIP which you suggested should jump to ram instead of just resetting to the bootloader like i'm currently doing, then i'd have to make that XIP code actually load the code to ram, because the builtin bootloader isn't gonna do that for me then.

i'd also probably have to write my own linker script and/or crt to produce a payload for that to load correctly

@nonchip
Copy link
Author

nonchip commented May 16, 2021

to elaborate on what i'm trying to do: build a "smart" adapter for a variety of different serial interfaces (Gameboy/GBA Link to be precise), where i'll put the low level "phy" layer stuff that never changes (and common game-agnostic features like the GBA wireless adapter emulation or MultiBoot) in XIP, and then have an application on the host (the USB host, aka PC, not the gameboy) that can load game-specific logic onto RAM which then interfaces that to the application over USB, because there's various games that require handshakes / ping timeouts (all implemented differently without any standard) faster than even USB can guarantee.

@nonchip
Copy link
Author

nonchip commented May 16, 2021

so i'll need to:

  • either boot to XIP and then somehow provide a way to load code onto RAM which gets called from the XIP code (which i'd kinda prefer but don't know how to do without having to reinvent the wheel that is bootloading and linking/crt)
  • or boot to RAM (which i'm currently doing, note my XIP main literally just resets into the bootloader) and call into functions in XIP; but which seems to fail somewhere along "using USB and XIP at the same time" (which makes no sense to me)

only thing i can imagine being the culprit there with USB and XIP is the "super slow XIP" access i'm using via the boot1 function literally timing out some USB interrupts/timing, maybe it might be worthwile investigating option 3:

  • boot into RAM, set up slow XIP or some kinda flash DMA stuff, copy boot2 to RAM, exit slow XIP/DMA, run boot2 from ram, then initialize USB and call into XIP?

but that sounds very much like a load of pain and boils down to "please gimme a target mode that runs in ram that isn't no_flash" again :P

@kilograham
Copy link
Contributor

but that sounds very much like a load of pain and boils down to "please gimme a target mode that runs in ram that isn't no_flash" again :P

well someone's pain, and given how much other stuff we have going on, probably gonna be yours :-)

well if i have code sitting in XIP which you suggested should jump to ram instead of just resetting to the bootloader like i'm currently doing, then i'd have to make that XIP code actually load the code to ram, because the builtin bootloader isn't gonna do that for me then.

I was suggesting you make a flash binary, which when run just jumps into a known address in the RAM binary (in fact just the beginning of binary is fine with a RAM binary). Your RAM binary when run checks one of the watchdog scratch registers to see what to do; (either do a watchdog reboot (into flash) having set a value in the scratch register for the next re-entry into the RAM binary), or continue executing and clear the special value.

@nonchip
Copy link
Author

nonchip commented May 16, 2021

I was suggesting you make a flash binary, which when run just jumps into a known address in the RAM binary

ooh that might actually work, especially given i think i can "flash" the ram binary using picotool and the magic baudrate without having to ever actually reset into the bootloader for the usb drive emulation.

good idea and sounds rather easy to achieve, will try :)

@kilograham
Copy link
Contributor

that works too; though i was just suggesting you run the RAM binary, and it reboots into flash, which then calls back into RAM

@nonchip
Copy link
Author

nonchip commented May 16, 2021

@kilograham ok that worked, ...somewhat.

// XIP:
int main() {
  if(watchdog_caused_reboot()){
    int (*ram_entry)() = (int(*)())SRAM_BASE;
    return ram_entry(); // jump to ram
  }
  reset_usb_boot(0, 0); // reboot into bootloader
}

// RAM:
int main() {
  if(!watchdog_caused_reboot()){
    int (*xip_entry)() = (int(*)())XIP_BASE; // yes the double cast is unnecessary but it works and is consistent with above
    watchdog_reboot((uintptr_t)xip_entry,SRAM_END,10);
    while (true); // let the watchdog kill us
  }
  stdio_init_all();
  /// my printf loop
}

problem is my output is now:

xip_funcs* 10001c20␍␊ // correct
xip_funcs: 0␍␊ // all wrong.
     init: 0␍␊
       on: 6␍␊
      off: 0␍␊

according to the map file those 3 should be function pointers to the addresses:

 .text.blinkoff
                0x0000000010000310        0xc CMakeFiles/xip.dir/xip.c.obj
                0x0000000010000310                blinkoff
 .text.blinkinit
                0x000000001000031c       0x14 CMakeFiles/xip.dir/xip.c.obj
                0x000000001000031c                blinkinit
 .text.blinkon  0x0000000010000330        0xc CMakeFiles/xip.dir/xip.c.obj
                0x0000000010000330                blinkon

could it be the ram entry point actually actively resets whatever boot2 did before?

yeah damn looks like runtime_init resets all the blocks. so i can't just jump to the start of SRAM
i guess i could "cheat" by writing the pointer to a different-than-main function somewhere when watchdog-rebooting to XIP and having it call that, but then i'll have the issue of all the stuff like initializing data not actually being done if i skip runtime_init

actually runtime_init seems to reset everything but PLL and QSPI (as to not screw up clocking and explicitely XIP). so no idea why it doesn't wanna read the correct values

i've pushed the current code again if you wanna give it a try

@kilograham
Copy link
Contributor

I'm sure you can debug on your own, but I will note that the ram pointer should be 0x20000001 so your probably aren't reaching there or it'd hard fault.

Also if you want to reboot into flash, you should pass 0 for the address

@nonchip
Copy link
Author

nonchip commented May 16, 2021

weird that it actually called into ram then, i suspect that's the compiler saving me (thumb calls are always odd). but good point for the rebooting to 0 there, trying...
...tried, yeah that does exactly the same as rebooting to XIP_BASE i'm afraid

@nonchip
Copy link
Author

nonchip commented May 17, 2021

ok yeah you were right @kilograham, something during ram initialization killed the "are we rebooting from watchdog" detection, i never actually reached the XIP code even, so my non-thumb-call didn't matter, and the struct was still borked because XIP never got initialized.

fixed that now by using different scratch registers, tried with both watchdog-rebooting to ROM_BASE and XIP_BASE (and both of them with and without +1 to force thumb mode), and with having rom call into both SRAM_BASE (with+without +1 again) and the address of the ram code's main (again even+odd), but somehow my code never reaches back to ram. guess i might have to wait for the picoprobe :/

@nonchip
Copy link
Author

nonchip commented May 17, 2021

what i still don't quite get is how rom_table_code('C', 'X') (enable slow XIP) messes with USB (like you said, during testing with a UART, it works just fine, but even initializing stdio after XIP was set up breaks CDC), maybe i can somehow reset the usb peripheral after that was enabled?

update on that front: it seems to actually crash the CPU and not just USBCDC when talking to XIP after "enabling" it using that bootrom call.
what I tried was a "two staged" approach for loading boot2 (via bootrom's "slow XIP") and calling it (to enable device specific "fast XIP") right from ram without the detour via the XIP watchdog reset:

void __no_inline_not_in_flash_func(enable_xip)(){
  void (*connect_internal_flash)(void) = (void(*)(void))rom_func_lookup(rom_table_code('I', 'F'));
  void (*flash_enter_cmd_xip)(void) = (void(*)(void))rom_func_lookup(rom_table_code('C', 'X'));
  void (*flash_exit_xip)(void) = (void(*)(void))rom_func_lookup(rom_table_code('E', 'X'));
  printf("connect_internal_flash: %x\n", connect_internal_flash);
  printf("flash_enter_cmd_xip: %x\n", flash_enter_cmd_xip);
  printf("flash_exit_xip: %x\n", flash_exit_xip);
  assert(connect_internal_flash && flash_enter_cmd_xip && flash_exit_xip);

  // enable super-slow XIP
  __compiler_memory_barrier();
  printf("connect_internal_flash: ");
  connect_internal_flash();
  printf("done\n");
  __compiler_memory_barrier();

  // enable super-slow XIP
  __compiler_memory_barrier();
  printf("flash_enter_cmd_xip: ");
  flash_enter_cmd_xip();
  printf("done\n");
  __compiler_memory_barrier();

  // we're in super-slow XIP now, copy boot2 to RAM
  __compiler_memory_barrier();
  static uint32_t boot2_copyout[64];
  printf("copying boot2 from %x to %x: ",XIP_BASE,boot2_copyout);
  for (int i = 0; i < 64; ++i){
    boot2_copyout[i] = ((uint32_t *)XIP_BASE)[i];
    printf(".");
  }
  printf("done\n");
  __compiler_memory_barrier();

  // we have the boot2 in RAM now, disable super-slow XIP
  __compiler_memory_barrier();
  printf("flash_exit_xip: ");
  flash_exit_xip();
  printf("done\n");
  __compiler_memory_barrier();

  // call boot2 to set up fast XIP
  __compiler_memory_barrier();
  printf("calling boot2: ");
  ((void (*)(void))boot2_copyout+1)();
  printf("done\n");
  __compiler_memory_barrier();
}

note this is literally the same as what hardware_flash/flash.c does to implement setting up XIP for write/erase access, except that it only does one of the two depending on the target (because unlike my code it can't expect boot2 to be available for copying on flash when running from ram).

the output i get from that (via UART):

connect_internal_flash: 24a1␍␊
flash_enter_cmd_xip: 2331␍␊
flash_exit_xip: 23f5␍␊
connect_internal_flash: done␍␊
flash_enter_cmd_xip: done␍␊
copying boot2 from 10000000 to 200075e0: 

and then it just hangs there like that, not doing anything else.

@Wren6991
Copy link
Contributor

Wren6991 commented May 17, 2021

The call to flash_exit_xip() after connect_internal_flash(), which you will see in the SDK flash code, is not optional. The function flash_init_spi() here is folded into that call (bit of a poor API, a victim of size hacking to fit into ROM space). flash_enter_cmd_xip() was intended to be used to reenable XIP following some flash programming sequence, so it assumes the calls are paired, and doesn't completely configure the SSI on its own (in particular the write ssi->ser = 1; is important)

Likewise flash_flush_cache() is slightly misnamed, and touches a couple of other registers to get the SSI into a good state following flash programming. In particular, the flash chip select is bitbanged during flash programming so that it says asserted when the processor is heavily interrupted, and this function restores SSI control of the chip select pin.

To get the flash from zero to slow-XIP access the sequence of calls is

  • connect_internal_flash()
  • flash_exit_xip()
  • flash_flush_cache()
  • flash_enter_cmd_xip()

which matches what you see in the SDK, minus some programming operation taking place in between the second and third call.

@lurch
Copy link
Contributor

lurch commented May 17, 2021

It's an entirely separate repo, but this reminds me of raspberrypi/openocd#29

EDIT: I meant in terms of the enter/exit XIP stuff

@Wren6991
Copy link
Contributor

It's an entirely separate repo, but this reminds me of raspberrypi/openocd#29

Seems unrelated, this issue doesn't involve flash erase, flash write, or openocd

@nonchip
Copy link
Author

nonchip commented May 17, 2021

@Wren6991 omg thank you so much, i would never have guessed i need to actively exit and flush before entering, it works perfectly now!

will clean up the code a bit and put it here for future reference :)

@Wren6991
Copy link
Contributor

Great! The flash_exit_xip() call is useful anyway, because after rebooting RP2040, the external flash device may be left in a state where it won't accept serial commands like the 03h used by slow-XIP

@nonchip
Copy link
Author

nonchip commented May 17, 2021

So here we go, this is the code for enabling "fast" XIP via a boot2 inside XIP:

#include "pico/stdlib.h"
#include "pico/bootrom.h"

// call this if you want "slow XIP":
void __no_inline_not_in_flash_func(enable_xip_via_bootrom)(){
  void (*connect_internal_flash)(void) = (void(*)(void))rom_func_lookup(rom_table_code('I', 'F'));
  void (*flash_exit_xip)(void) = (void(*)(void))rom_func_lookup(rom_table_code('E', 'X'));
  void (*flash_flush_cache)(void) = (void(*)(void))rom_func_lookup(rom_table_code('F', 'C'));
  void (*flash_enter_cmd_xip)(void) = (void(*)(void))rom_func_lookup(rom_table_code('C', 'X'));
  assert(connect_internal_flash && flash_enter_cmd_xip && flash_exit_xip && flash_flush_cache);

  __compiler_memory_barrier();
  connect_internal_flash();
  __compiler_memory_barrier();
  flash_exit_xip();
  __compiler_memory_barrier();
  flash_flush_cache();
  __compiler_memory_barrier();
  flash_enter_cmd_xip();
  __compiler_memory_barrier();
}

// call this if you want "fast XIP" and have boot2 residing in XIP:
void __no_inline_not_in_flash_func(enable_xip_via_boot2_in_xip)(){
  void (*flash_exit_xip)(void) = (void(*)(void))rom_func_lookup(rom_table_code('E', 'X'));
  assert(flash_exit_xip);

  enable_xip_via_bootrom();

  static uint32_t boot2_copyout[64];
  for (int i = 0; i < 64; ++i){
    boot2_copyout[i] = ((uint32_t *)XIP_BASE)[i];
  }

  __compiler_memory_barrier();
  flash_exit_xip();
  __compiler_memory_barrier();
  ((void (*)(void))boot2_copyout+1)();
  __compiler_memory_barrier();
}

@Wren6991
Copy link
Contributor

Wren6991 commented May 17, 2021

void blinkoff() {
  gpio_put(LED_PIN, 1);
}

void blinkon() {
  gpio_put(LED_PIN, 1);
}

Hmmm :)
edit: guess you found it

@nonchip
Copy link
Author

nonchip commented May 17, 2021

void blinkoff() {
  gpio_put(LED_PIN, 1);
}

void blinkon() {
  gpio_put(LED_PIN, 1);
}

Hmmm :)

yeeaaaah i feel dumb, did a copypaste derp, noticed when i saw the watchdog never killing me :'D
guess this is all fixed now then :P

@nonchip nonchip closed this as completed May 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants