Implement booting from USB storage #5

FooDeas · 2016-08-05T07:44:33Z

The installer should implement the new feature to install and boot from USB storage as described here: PI 3 BOOTING PART I: USB MASS STORAGE BOOT BETA

FooDeas · 2016-08-08T16:12:40Z

Important info from Raspberry Pi Forum:

..."program_usb_timeout" is non reversible written to OTP...

...
Yes it's permanent otherwise it couldn't effect the bootrom, it readable in the otp_dump it's bit 24 up near the top of the range...
...
new empty file called TIMEOUT on the sdcard (this triggers the same effect of setting the above extra delay without setting the OTP bit)

So if booting from MSD, the installer should create this file and stay away from other OTP bits than usb_boot_mode.

FooDeas · 2016-08-12T23:58:43Z

Functionality is given with 305196c. This should be working as soon as the firmware package is updated by package maintainers.

Documentation is not completed yet.

thijstriemstra · 2016-11-29T00:35:39Z

Is there any documentation on how to do this?

FooDeas · 2016-11-29T00:49:08Z

You have to replace the bootcode.bin before and after the installation and set usbboot=1 in installer-config.txt.
Have an extra look at the installer output belonging to the msd boot bit you might have to program once!

thijstriemstra · 2016-12-05T01:31:51Z

Ordered a PiDrive and will test this. Also, support for raspberrypi-ua-netinst in https://github.com/maxnet/berryboot would be nice. It uses a Qt front-end, maybe I can hack it a little. This page also mentions a 4GB swap can be created and used as RAM which sounds awesome.

FooDeas · 2016-12-05T09:44:07Z

You know, that you can use a simple USB flash drive, too?

thijstriemstra · 2016-12-05T14:36:03Z

True, but they're much slower and more expensive (this disk was 30 euros for 250GB). Benchmarks here: http://www.legitreviews.com/wd-labs-pidrive-314gb-hard-drive-review_180301/4

FooDeas · 2016-12-05T22:27:46Z

👍

thijstriemstra · 2016-12-29T16:45:54Z

This should be working as soon as the firmware package is updated by package maintainers.

@FooDeas has it been updated, where to check? I just flashed my PiDrive with a raspberrypi-ua-netinst build but it's not booting without an SD-card..

FooDeas · 2016-12-29T16:58:45Z

It's still in beta. I'm testing if it is moved to stable from time to time.
I'm pretty sure, it will be posted on the RPi Blog when the time comes.

thijstriemstra · 2016-12-29T17:40:00Z

Alright, too bad.

thijstriemstra · 2017-02-16T18:21:04Z

So according to this page the following steps are required:

replace the default start.elf and bootcode.bin files with the alternatives from the next branch (using apt-get update but I'd rather wget it?)
enable usb boot mode and reboot:

echo program_usb_boot_mode=1 | sudo tee -a /boot/config.txt

verify that OTP is representative of the address 0x3020000a (such as 17:3020000a):

vcgencmd otp_dump | grep 17:

Which I'll test now.

thijstriemstra · 2017-02-16T18:57:51Z

Rebooted and checked the output of OTP but this didn't return the expected result (17:3020000a):

$ vcgencmd otp_dump | grep 17:
17:1020000a

So I did the recommended branch update instead:

$ sudo BRANCH=next rpi-update
 *** Updating firmware
 *** Updating kernel modules
 *** depmod 4.9.9-v7+
 *** depmod 4.9.9+
 *** Updating VideoCore libraries
 *** Using HardFP libraries
 *** Updating SDK
 *** Running ldconfig
 *** Storing current firmware revision
 *** Deleting downloaded files
 *** Syncing changes to disk
 *** If no errors appeared, your firmware was successfully updated to 719eb18d040b53903f7083cef2ee9f92300a90ab
 *** A reboot is needed to activate the new firmware

And rebooted but still same result. Maybe it's because I'm using a RPi2 (the tutorial says it's for RPi3)?

Tried it on a RPi3 (with master branch, not next) and gives the same 17:1020000a.

According to this post: "however, I didn’t receive the correct output as stated (0x3020000a) but I decided to continue rather than investigate and as it turned out it made no difference to the outcome".

So I'm moving to the next step.

FooDeas · 2017-02-16T20:07:51Z

The first byte (0x10) has to contain the 1 on 3rd position (0b00100000). So you're right with 0x30....

The config.txt has to include program_usb_boot_mode=1 (this is needed only once). At this time, the bootcode.bin has to be already replaced from here. Do that by yourself.
During the next boot, the OTP will get programmed.

Now vcgencmd otp_dump | grep 17: should return the 0x30....

thijstriemstra · 2017-02-16T20:10:49Z

completed the tutorial and my machine isn't booting from USB; most likely because of this otp_dump issue. Maybe I should hold off until it's officially supported..

FooDeas · 2017-02-16T20:14:42Z

It definitely works! Have you get the OTP setup working with my comment from above now?
(I'm unsure, what you've meant with "tutorial"...)

It's a strong requirement - and works with RPi 3 only!

thijstriemstra · 2017-02-16T20:17:45Z

I meant this tutorial, I haven't used raspberrypi-ua-netinst for this yet; first I'd like to get it working without but this OTP returns the wrong nr. Will try with a raspberrypi 3 vs rpi2

FooDeas · 2017-02-16T20:21:13Z

Forget the RPi 2 - it only works with RPi 3:

Will it be possible to boot a Pi 1 or Pi 2 using MSD?

Unfortunately not. The boot code is stored in the BCM2837 device only, so the Pi 1, Pi 2, and Pi Zero will all require SD cards.

(Source from post 1 above.)

thijstriemstra · 2017-02-16T20:23:47Z

ah thanks, hopeful.

FooDeas · 2017-02-16T20:29:35Z

What I wrote here has to work definitely! I'm confident that you'll get it working, too.

(Stupid "Close and comment" button...)

thijstriemstra · 2017-02-16T20:32:14Z

Alright this is better (on RPi 3):

$ vcgencmd otp_dump | grep 17:
17:3020000a

And booting now works! Feels like it's running a lot slower though..

thijstriemstra · 2017-02-16T20:51:02Z

So raspberrypi-ua-netinst basically needs to:

reject this feature on RPI <= 2
program USB boot mode once (wget the start.elf and bootcode.bin files and put them in place)
verify OTP has been programmed:

$ vcgencmd otp_dump | grep 17:
17:3020000a

The final step, preparing the USB device is more of a documentation thing I imagine.

Thoughts?

FooDeas · 2017-02-16T21:07:55Z

reject this feature on RPI <= 2 - Great idea!
program USB boot mode once - It's semi automatic at the moment because OTP changes are permanent and I was very careful while developing this. Have a look at the default config.txt of the latest release...
wget the start.elf and bootcode.bin files and put them in place - The problem is to keep them during a kernel (wheezy)/bootloader (jessy and above) update because they're not delivered by the raspberrypi.org packages yet. So they may be overwritten and your Pi stops booting until you replace them again...
verify OTP has been programmed

Everything else is already working! But I paused this topic because the main problem above - so at the moment the branch is a bit outdated (but should be mergable easily).

thijstriemstra · 2017-02-16T21:55:39Z

The problem is to keep them [start.elf and bootcode.bin] during a kernel (wheezy)/bootloader (jessy and above) update because they're not delivered by the raspberrypi.org packages yet. So they may be overwritten and your Pi stops booting until you replace them again...

I suppose this is something I can live with; I simply cannot update (the kernel using apt-get)? Until the next firmware branch is merged into master?

thijstriemstra · 2017-02-16T22:04:54Z

And what about these instructions: raspberrypi-ua-netinst does most of that right?

Create the boot and root file systems:

sudo mkfs.vfat -n BOOT -F 32 /dev/sda1
sudo mkfs.ext4 /dev/sda2

Mount the target file system and copy the running raspbian system to it:

sudo mkdir /mnt/target
sudo mount /dev/sda2 /mnt/target/
sudo mkdir /mnt/target/boot
sudo mount /dev/sda1 /mnt/target/boot/
sudo apt-get update; sudo apt-get install rsync
sudo rsync -ax --progress / /boot /mnt/target

FooDeas · 2017-02-16T22:44:18Z

Post 1: You can update, but you can't be sure that a reboot will work if the files got replaced.
Post 2: The system will be installed directly on the target USB device. Nothing special to copy/move/config.

thijstriemstra · 2017-05-20T01:37:29Z

Also see https://learn.adafruit.com/external-drive-as-raspberry-pi-root/hooking-up-the-drive-and-copying-slash

FooDeas · 2017-06-16T01:17:34Z

Starting with the 2017-04-10 release of Raspbian there finally is some progress here! (see raspberrypi/firmware@e44f66a)
I need some testing and will possibly release that in very few days.

FooDeas · 2017-06-18T22:41:10Z

OK - it's done! Checkout: ~~d00f3c9~~ 7ff846f

thijstriemstra · 2017-06-21T20:11:15Z

Awesome, will do!

FooDeas · 2017-07-08T14:04:51Z

Results?

thijstriemstra · 2017-09-11T22:40:40Z

Testing now with:

usbroot=1
usbboot=1

thijstriemstra · 2017-09-11T22:57:42Z

And it looks like it worked out of the box using a PiDrive. Nice work!

$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/root       233G  1,8G  220G   1% /
devtmpfs        236M     0  236M   0% /dev
tmpfs           240M     0  240M   0% /dev/shm
tmpfs           240M  4,7M  236M   2% /run
tmpfs           5,0M  4,0K  5,0M   1% /run/lock
tmpfs           240M     0  240M   0% /sys/fs/cgroup
tmpfs           240M     0  240M   0% /tmp
/dev/sda1       128M   22M  107M  17% /boot
tmpfs            48M     0   48M   0% /run/user/1000

thijstriemstra · 2017-09-11T23:50:32Z

==================================================
raspberrypi-ua-netinst
==================================================
Revision d01fd04
Built on ma 11 sep 2017 23:47:54 CEST
Running on Raspberry Pi version 3 Model B
==================================================
https://github.com/FooDeas/raspberrypi-ua-netinst/
==================================================
Starting HWRNG... OK
Mounting boot partition... OK
Copying boot files... OK
Executing installer-config.txt...
OK

This OK shouldn't be on newline.

Searching for deprecated variables...

Checking if config.txt needs to be modified before starting installation...
  Setting HDMI options... OK
  Checking USB boot flag... OK
OK

This OK doesn't have a description.


Unmounting boot partition... OK

Network configuration:
  ifname = eth0
  ip_addr = dhcp
  online_config = 

Waiting for eth0... OK
Configuring eth0 with DHCP... 192.168.2.54
Set time using ntpdate... OK

Installation started at Mon Sep 11 22:39:11 UTC 2017 (UTC).


=============================================================================================
                                  !!! IMPORTANT NOTICE !!!
Because you are installing from SD card and want to boot from USB,
the system will POWERED OFF after installation.
After finishing the installation, you must REMOVE the SD card and reboot the system MANUALLY.

The installation will continue in 15 seconds...
=============================================================================================

This notice should be in the documentation as well (if it's not already). And there's a typo:

'the system will be POWERED OFF after installation.'

And later:

Configuring user 'foo':
    Adding user `foo' ...
    Adding new group `foo' (1000) ...
    Adding new user `foo' (1000) with group `foo' ...
    Creating home directory `/home/foo' ...
    Copying files from `/etc/skel' ...

None of these lines have an OK.

FooDeas added the enhancement label Aug 5, 2016

FooDeas added this to the v1.2.0 milestone Aug 5, 2016

FooDeas modified the milestones: 2.0.0, v1.2.0, v2.0.0 Aug 12, 2016

FooDeas self-assigned this Aug 16, 2016

Repository owner locked and limited conversation to collaborators Aug 16, 2016

FooDeas removed this from the v2.0.0 milestone Jan 24, 2017

FooDeas closed this as completed Feb 16, 2017

FooDeas reopened this Feb 16, 2017

FooDeas added this to the v2.0.0 milestone Sep 30, 2017

FooDeas closed this as completed Sep 30, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement booting from USB storage #5

Implement booting from USB storage #5

FooDeas commented Aug 5, 2016

FooDeas commented Aug 8, 2016 •

edited

Loading

FooDeas commented Aug 12, 2016 •

edited

Loading

thijstriemstra commented Nov 29, 2016

FooDeas commented Nov 29, 2016

thijstriemstra commented Dec 5, 2016

FooDeas commented Dec 5, 2016

thijstriemstra commented Dec 5, 2016

FooDeas commented Dec 5, 2016

thijstriemstra commented Dec 29, 2016 •

edited

Loading

FooDeas commented Dec 29, 2016

thijstriemstra commented Dec 29, 2016

thijstriemstra commented Feb 16, 2017 •

edited

Loading

thijstriemstra commented Feb 16, 2017 •

edited

Loading

FooDeas commented Feb 16, 2017

thijstriemstra commented Feb 16, 2017

FooDeas commented Feb 16, 2017 •

edited

Loading

thijstriemstra commented Feb 16, 2017

FooDeas commented Feb 16, 2017 •

edited

Loading

thijstriemstra commented Feb 16, 2017

FooDeas commented Feb 16, 2017 •

edited

Loading

thijstriemstra commented Feb 16, 2017 •

edited

Loading

thijstriemstra commented Feb 16, 2017

FooDeas commented Feb 16, 2017 •

edited by thijstriemstra

Loading

thijstriemstra commented Feb 16, 2017 •

edited

Loading

thijstriemstra commented Feb 16, 2017

FooDeas commented Feb 16, 2017 •

edited

Loading

thijstriemstra commented May 20, 2017

FooDeas commented Jun 16, 2017 •

edited

Loading

FooDeas commented Jun 18, 2017 •

edited

Loading

thijstriemstra commented Jun 21, 2017

FooDeas commented Jul 8, 2017

thijstriemstra commented Sep 11, 2017

thijstriemstra commented Sep 11, 2017 •

edited

Loading

thijstriemstra commented Sep 11, 2017 •

edited

Loading

Implement booting from USB storage #5

Implement booting from USB storage #5

Comments

FooDeas commented Aug 5, 2016

FooDeas commented Aug 8, 2016 • edited Loading

FooDeas commented Aug 12, 2016 • edited Loading

thijstriemstra commented Nov 29, 2016

FooDeas commented Nov 29, 2016

thijstriemstra commented Dec 5, 2016

FooDeas commented Dec 5, 2016

thijstriemstra commented Dec 5, 2016

FooDeas commented Dec 5, 2016

thijstriemstra commented Dec 29, 2016 • edited Loading

FooDeas commented Dec 29, 2016

thijstriemstra commented Dec 29, 2016

thijstriemstra commented Feb 16, 2017 • edited Loading

thijstriemstra commented Feb 16, 2017 • edited Loading

FooDeas commented Feb 16, 2017

thijstriemstra commented Feb 16, 2017

FooDeas commented Feb 16, 2017 • edited Loading

thijstriemstra commented Feb 16, 2017

FooDeas commented Feb 16, 2017 • edited Loading

thijstriemstra commented Feb 16, 2017

FooDeas commented Feb 16, 2017 • edited Loading

thijstriemstra commented Feb 16, 2017 • edited Loading

thijstriemstra commented Feb 16, 2017

FooDeas commented Feb 16, 2017 • edited by thijstriemstra Loading

thijstriemstra commented Feb 16, 2017 • edited Loading

thijstriemstra commented Feb 16, 2017

FooDeas commented Feb 16, 2017 • edited Loading

thijstriemstra commented May 20, 2017

FooDeas commented Jun 16, 2017 • edited Loading

FooDeas commented Jun 18, 2017 • edited Loading

thijstriemstra commented Jun 21, 2017

FooDeas commented Jul 8, 2017

thijstriemstra commented Sep 11, 2017

thijstriemstra commented Sep 11, 2017 • edited Loading

thijstriemstra commented Sep 11, 2017 • edited Loading

FooDeas commented Aug 8, 2016 •

edited

Loading

FooDeas commented Aug 12, 2016 •

edited

Loading

thijstriemstra commented Dec 29, 2016 •

edited

Loading

thijstriemstra commented Feb 16, 2017 •

edited

Loading

thijstriemstra commented Feb 16, 2017 •

edited

Loading

FooDeas commented Feb 16, 2017 •

edited

Loading

FooDeas commented Feb 16, 2017 •

edited

Loading

FooDeas commented Feb 16, 2017 •

edited

Loading

thijstriemstra commented Feb 16, 2017 •

edited

Loading

FooDeas commented Feb 16, 2017 •

edited by thijstriemstra

Loading

thijstriemstra commented Feb 16, 2017 •

edited

Loading

FooDeas commented Feb 16, 2017 •

edited

Loading

FooDeas commented Jun 16, 2017 •

edited

Loading

FooDeas commented Jun 18, 2017 •

edited

Loading

thijstriemstra commented Sep 11, 2017 •

edited

Loading

thijstriemstra commented Sep 11, 2017 •

edited

Loading