Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Failed to write complete file to USB device" when using custom hardware #30

Open
FiesoDuck opened this issue Mar 28, 2018 · 28 comments
Open

Comments

@FiesoDuck
Copy link

@FiesoDuck FiesoDuck commented Mar 28, 2018

Hi

in an effort to streamline the process of programming multiple CMs we designed a custom pcb following the official IO-Boards usb circuit.
flash adapter

flasher3

When booting the CMs as a mass storage device it works as intended. Currently we are using 16x flash pcbs at once and some command line magic to make it all happen.

Now we tried to boot the CMs with the directory option
sudo ./rpiboot -d /outputdir

This fails with following output:

philip@Dell:~/usbboot$ sudo ./rpiboot -v -d /media/philip/ext4_Entwicklung/scriptexecutor/output
Waiting for BCM2835/6/7
Device located successfully
Initialised device correctly
Found serial number 0
Sending bootcode.bin
libusb_bulk_transfer returned 0
Writing 50216 bytes
libusb_bulk_transfer returned 0
Successful read 4 bytes
Waiting for BCM2835/6/7
Device located successfully
Initialised device correctly
Found serial number 1
Second stage boot server
Received message GetFileSize: autoboot.txt
Cannot open file autoboot.txt
Received message GetFileSize: config.txt
File size = 47885 bytes
Received message ReadFile: config.txt
File read: config.txt
libusb_bulk_transfer returned 0
Received message GetFileSize: recovery.elf
Cannot open file recovery.elf
Received message GetFileSize: start.elf
File size = 2855972 bytes
Received message ReadFile: start.elf
File read: start.elf
libusb_bulk_transfer returned 0
Received message GetFileSize: fixup.dat
File size = 6669 bytes
Received message ReadFile: fixup.dat
File read: fixup.dat
libusb_bulk_transfer returned 0
Received message GetFileSize: recovery.elf
Cannot open file recovery.elf
Received message GetFileSize: config.txt
File size = 47885 bytes
Received message ReadFile: config.txt
File read: config.txt
libusb_bulk_transfer returned -7
Failed to write complete file to USB device

If i switch the exact CM to the official IO-Board and execute the exact command it works as expected.

Tested on Ubuntu 17.10

@ghollingworth

This comment has been minimized.

Copy link
Contributor

@ghollingworth ghollingworth commented Mar 28, 2018

@FiesoDuck

This comment has been minimized.

Copy link
Author

@FiesoDuck FiesoDuck commented Apr 3, 2018

Is that 67R choke differential or common mode impedence?

Yes it is. Replacing with an 90R did not yield better results.

If you've just picked up a random cable then try a nice quality one, at least one with good twisted pairs and a continuous shield.

This helped but now i get this error:

Received message GetFileSize: dt-blob.bin
Cannot open file dt-blob.bin

While using the official IO-Board the dt-blob.bin is not needed to be present in the output folder. And even after compiling (sudo dtc -I dts -O dtb -o /boot/dt-blob.bin dt-blob.dts) the dt-blob.bin and copying it to the output folder the board still won't boot:

philip@Dell:~/usbboot$ sudo ./rpiboot -v -d /media/philip/ext4_Entwicklung/scriptexecutor/output
Waiting for BCM2835/6/7
Device located successfully
Initialised device correctly
Found serial number 0
Sending bootcode.bin
libusb_bulk_transfer returned 0
Writing 50216 bytes
libusb_bulk_transfer returned 0
Successful read 4 bytes
Waiting for BCM2835/6/7
Device located successfully
Initialised device correctly
Found serial number 1
Second stage boot server
Received message GetFileSize: autoboot.txt
Cannot open file autoboot.txt
Received message GetFileSize: config.txt
File size = 28 bytes
Received message ReadFile: config.txt
File read: config.txt
libusb_bulk_transfer returned 0
Received message GetFileSize: recovery.elf
Cannot open file recovery.elf
Received message GetFileSize: start.elf
File size = 2855972 bytes
Received message ReadFile: start.elf
File read: start.elf
libusb_bulk_transfer returned 0
Received message GetFileSize: fixup.dat
File size = 6669 bytes
Received message ReadFile: fixup.dat
File read: fixup.dat
libusb_bulk_transfer returned 0
Received message GetFileSize: recovery.elf
Cannot open file recovery.elf
Received message GetFileSize: config.txt
File size = 28 bytes
Received message ReadFile: config.txt
File read: config.txt
libusb_bulk_transfer returned 0
Received message GetFileSize: dt-blob.bin
File size = 48418 bytes
Received message ReadFile: dt-blob.bin
File read: dt-blob.bin
libusb_bulk_transfer returned 0

The command never finishes after that.

The dt-blob.bin is compiled into the .elf files iirc, why is it necessary to supply the dt-blob.bin when using our custom design and not when using the official IO-Board? And why is the freshly compiled dt-blob.bin not accepted when copied to the output folder?

@pelwell

This comment has been minimized.

Copy link
Contributor

@pelwell pelwell commented Apr 3, 2018

I think the absence of dt-blob.bin is a red herring. There is no way to check for the presence of a file other than trying open it. As you've seen, providing a dt-blob.bin doesn't solve the problem, so attempting to load the dt-blob is just the last thing that the loader manages to do.

@FiesoDuck

This comment has been minimized.

Copy link
Author

@FiesoDuck FiesoDuck commented Apr 3, 2018

Here is the output for the official IO-Board:

philip@Dell:~/usbboot$ sudo ./rpiboot -v -d /media/philip/ext4_Entwicklung/scriptexecutor/output
Waiting for BCM2835/6/7
Device located successfully
Initialised device correctly
Found serial number 0
Sending bootcode.bin
libusb_bulk_transfer returned 0
Writing 50216 bytes
libusb_bulk_transfer returned 0
Successful read 4 bytes
Waiting for BCM2835/6/7
Device located successfully
Initialised device correctly
Found serial number 1
Second stage boot server
Received message GetFileSize: autoboot.txt
Cannot open file autoboot.txt
Received message GetFileSize: config.txt
File size = 28 bytes
Received message ReadFile: config.txt
File read: config.txt
libusb_bulk_transfer returned 0
Received message GetFileSize: recovery.elf
Cannot open file recovery.elf
Received message GetFileSize: start.elf
File size = 2855972 bytes
Received message ReadFile: start.elf
File read: start.elf
libusb_bulk_transfer returned 0
Received message GetFileSize: fixup.dat
File size = 6669 bytes
Received message ReadFile: fixup.dat
File read: fixup.dat
libusb_bulk_transfer returned 0
Received message GetFileSize: recovery.elf
Cannot open file recovery.elf
Received message GetFileSize: config.txt
File size = 28 bytes
Received message ReadFile: config.txt
File read: config.txt
libusb_bulk_transfer returned 0
Received message GetFileSize: dt-blob.bin
Cannot open file dt-blob.bin
Received message GetFileSize: recovery.elf
Cannot open file recovery.elf
Received message GetFileSize: config.txt
File size = 28 bytes
Received message ReadFile: config.txt
File read: config.txt
libusb_bulk_transfer returned 0
Received message GetFileSize: bootcfg.txt
Cannot open file bootcfg.txt
Received message GetFileSize: cmdline.txt
File size = 111 bytes
Received message ReadFile: cmdline.txt
File read: cmdline.txt
libusb_bulk_transfer returned 0
Received message GetFileSize: recovery8.img
Cannot open file recovery8.img
Received message GetFileSize: recovery8-32.img
Cannot open file recovery8-32.img
Received message GetFileSize: recovery7.img
Cannot open file recovery7.img
Received message GetFileSize: recovery.img
Cannot open file recovery.img
Received message GetFileSize: kernel8.img
Cannot open file kernel8.img
Received message GetFileSize: kernel8-32.img
Cannot open file kernel8-32.img
Received message GetFileSize: kernel7.img
Cannot open file kernel7.img
Received message GetFileSize: kernel.img
File size = 2900824 bytes
Received message GetFileSize: armstub8.bin
Cannot open file armstub8.bin
Received message GetFileSize: armstub8-32.bin
Cannot open file armstub8-32.bin
Received message GetFileSize: armstub7.bin
Cannot open file armstub7.bin
Received message GetFileSize: armstub.bin
Cannot open file armstub.bin
Received message GetFileSize: kernel.img
File size = 2900824 bytes
Received message ReadFile: kernel.img
File read: kernel.img
libusb_bulk_transfer returned 0
Received message GetFileSize: scriptexecute.img
File size = 669729 bytes
Received message ReadFile: scriptexecute.img
File read: scriptexecute.img
libusb_bulk_transfer returned 0
Received message GetFileSize: bcm2710-rpi-cm3.dtb
File size = 15346 bytes
Received message ReadFile: bcm2710-rpi-cm3.dtb
File read: bcm2710-rpi-cm3.dtb
libusb_bulk_transfer returned 0
Received message GetFileSize: config.txt
File size = 28 bytes
Received message ReadFile: config.txt
File read: config.txt
libusb_bulk_transfer returned 0
Received message Done: config.txt
Second stage boot server done

Especially this part:

Received message GetFileSize: dt-blob.bin
Cannot open file dt-blob.bin

As you can see the error is the same but the official board just continues to boot while our custom board hangs.

@pelwell

This comment has been minimized.

Copy link
Contributor

@pelwell pelwell commented Apr 3, 2018

If you are you able to get a shell prompt to the CM3 while it's in the CMIO board (either via serial port or ssh over ethernet), use sudo vcdbg log msg to view the activity log. You should be able to correlate the file accesses from rpi-boot with positions in the log:

$ sudo vcdbg log msg |& grep File

You may find it easier to send the output to a file and use an editor to read it:

$ sudo vcdbg log msg >& vcdbg.txt

On a 3B+ I see two reads of config.txt, followed by a read of cmdline.txt, etc. Linking that to your rpiboot log suggests that your booting is failing between those two config.txt reads. All I see in that gap are HDMI messages, but you might see something different.

@FiesoDuck

This comment has been minimized.

Copy link
Author

@FiesoDuck FiesoDuck commented Apr 3, 2018

If you are you able to get a shell prompt to the CM3 while it's in the CMIO board (either via serial port or ssh over ethernet), use sudo vcdbg log msg to view the activity log.

I tried that and got
-sh: vcdbg: not found

I am following this btw.: https://github.com/raspberrypi/scriptexecutor/wiki/Building-and-customizing

@pelwell

This comment has been minimized.

Copy link
Contributor

@pelwell pelwell commented Apr 3, 2018

You can copy the binary (the source isn't published for licensing reasons) from here: https://github.com/raspberrypi/firmware/blob/master/opt/vc/bin/vcdbg

@FiesoDuck

This comment has been minimized.

Copy link
Author

@FiesoDuck FiesoDuck commented Apr 3, 2018

I downloaded the file, put it on a usb stick and mounted the drive. When i try to ./vcdbg i still get -sh: vcdbg: not found. I also tried chmod +x vcdbg with no success. I also tried to copy the file to the root home dir first and then execute it but to no avail.

@pelwell

This comment has been minimized.

Copy link
Contributor

@pelwell pelwell commented Apr 3, 2018

You must also be missing some of the dependencies:

pi@raspberrypi:~ $ ldd /opt/vc/bin/vcdbg 
	linux-vdso.so.1 (0x7ee95000)
	/usr/lib/arm-linux-gnueabihf/libarmmem.so (0x76f3f000)
	libdebug_sym.so => /opt/vc/lib/libdebug_sym.so (0x76f17000)
	libelftoolchain.so => /opt/vc/lib/libelftoolchain.so (0x76ec2000)
	libvcos.so => /opt/vc/lib/libvcos.so (0x76ea8000)
	libpthread.so.0 => /lib/arm-linux-gnueabihf/libpthread.so.0 (0x76e7f000)
	libdl.so.2 => /lib/arm-linux-gnueabihf/libdl.so.2 (0x76e6c000)
	librt.so.1 => /lib/arm-linux-gnueabihf/librt.so.1 (0x76e55000)
	libstdc++.so.6 => /usr/lib/arm-linux-gnueabihf/libstdc++.so.6 (0x76d0d000)
	libm.so.6 => /lib/arm-linux-gnueabihf/libm.so.6 (0x76c8e000)
	libgcc_s.so.1 => /lib/arm-linux-gnueabihf/libgcc_s.so.1 (0x76c61000)
	libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0x76b22000)
	/lib/ld-linux-armhf.so.3 (0x76f55000)

I would start with the three from /opt/vc/lib.

@pelwell

This comment has been minimized.

@FiesoDuck

This comment has been minimized.

Copy link
Author

@FiesoDuck FiesoDuck commented Apr 3, 2018

I replicated the directory structure and moved the corresponding files. (libdebug_sym.so, libelftoolchain.so, libvcos.so)
Same result as before: -sh: vcdbg: not found

Is it worth to mention, that /opt was empty before copying the files?

@pelwell

This comment has been minimized.

Copy link
Contributor

@pelwell pelwell commented Apr 3, 2018

Your installation isn't putting /opt/vc/lib in the LD path. Put the libraries in /lib/arm-gnueabihf or /usr/local/lib instead.

@FiesoDuck

This comment has been minimized.

Copy link
Author

@FiesoDuck FiesoDuck commented Apr 3, 2018

Same result as before.

If libraries were missing, shouldn't the error be more like
vcdbg: error while loading shared libraries: libelftoolchain.so: cannot open shared object file: No such file or directory

@pelwell

This comment has been minimized.

Copy link
Contributor

@pelwell pelwell commented Apr 3, 2018

sudo ./vcdbg log msg?

@FiesoDuck

This comment has been minimized.

Copy link
Author

@FiesoDuck FiesoDuck commented Apr 3, 2018

-sh: sudo: not found

I am logged in as root

@pelwell

This comment has been minimized.

Copy link
Contributor

@pelwell pelwell commented Apr 3, 2018

./vcdbg log msg then.

@FiesoDuck

This comment has been minimized.

Copy link
Author

@FiesoDuck FiesoDuck commented Apr 3, 2018

As i stated above, same result as before.
-sh: vcdbg: not found

@pelwell

This comment has been minimized.

Copy link
Contributor

@pelwell pelwell commented Apr 3, 2018

ls -l vcdbg
file vcdbg
@FiesoDuck

This comment has been minimized.

Copy link
Author

@FiesoDuck FiesoDuck commented Apr 3, 2018

ls -l vcdbg
-rwxr-xr-x 1 root root 253273 Jan 1 01:05 vcdbg

file vcdbg
-sh: file: not found
@pelwell

This comment has been minimized.

Copy link
Contributor

@pelwell pelwell commented Apr 3, 2018

How about debugging your hardware issue with a known Raspbian Lite image?

@FiesoDuck

This comment has been minimized.

Copy link
Author

@FiesoDuck FiesoDuck commented Apr 3, 2018

I am trying this as we speak.

@FiesoDuck

This comment has been minimized.

Copy link
Author

@FiesoDuck FiesoDuck commented Apr 3, 2018

I copied the contents of the FAT partition from 2017-07-05-raspbian-jessie-lite.img to my output directory.

When using the official IO-Board i get:

philip@Dell:~/usbboot$ sudo ./rpiboot -v -d /media/philip/ext4_Entwicklung/t
Waiting for BCM2835/6/7
Device located successfully
Initialised device correctly
Found serial number 0
Sending bootcode.bin
libusb_bulk_transfer returned 0
Writing 50248 bytes
libusb_bulk_transfer returned 0
Successful read 4 bytes
Waiting for BCM2835/6/7
Device located successfully
Initialised device correctly
Found serial number 1
Second stage boot server
Received message GetFileSize: autoboot.txt
Cannot open file autoboot.txt
Received message GetFileSize: config.txt
File size = 1590 bytes
Received message ReadFile: config.txt
File read: config.txt
libusb_bulk_transfer returned 0
Received message GetFileSize: recovery.elf
Cannot open file recovery.elf
Received message GetFileSize: start.elf
File size = 2868132 bytes
Received message ReadFile: start.elf
File read: start.elf
libusb_bulk_transfer returned 0
Received message GetFileSize: fixup.dat
File size = 6688 bytes
Received message ReadFile: fixup.dat
File read: fixup.dat
libusb_bulk_transfer returned 0
Received message GetFileSize: recovery.elf
Cannot open file recovery.elf
Received message GetFileSize: config.txt
File size = 1590 bytes
Received message ReadFile: config.txt
File read: config.txt
libusb_bulk_transfer returned 0
Received message GetFileSize: dt-blob.bin
Cannot open file dt-blob.bin
Received message GetFileSize: recovery.elf
Cannot open file recovery.elf
Received message GetFileSize: config.txt
File size = 1590 bytes
Received message ReadFile: config.txt
File read: config.txt
libusb_bulk_transfer returned 0
Received message GetFileSize: bootcfg.txt
Cannot open file bootcfg.txt
Received message GetFileSize: cmdline.txt
File size = 190 bytes
Received message ReadFile: cmdline.txt
File read: cmdline.txt
libusb_bulk_transfer returned 0
Received message GetFileSize: recovery8.img
Cannot open file recovery8.img
Received message GetFileSize: recovery8-32.img
Cannot open file recovery8-32.img
Received message GetFileSize: recovery7.img
Cannot open file recovery7.img
Received message GetFileSize: recovery.img
Cannot open file recovery.img
Received message GetFileSize: kernel8.img
Cannot open file kernel8.img
Received message GetFileSize: kernel8-32.img
Cannot open file kernel8-32.img
Received message GetFileSize: kernel7.img
File size = 4581064 bytes
Received message GetFileSize: armstub8.bin
Cannot open file armstub8.bin
Received message GetFileSize: armstub8-32.bin
Cannot open file armstub8-32.bin
Received message GetFileSize: armstub7.bin
Cannot open file armstub7.bin
Received message GetFileSize: armstub.bin
Cannot open file armstub.bin
Received message GetFileSize: kernel7.img
File size = 4581064 bytes
Received message ReadFile: kernel7.img
File read: kernel7.img
libusb_bulk_transfer returned 0
Received message GetFileSize: bcm2710-rpi-cm3.dtb
File size = 16380 bytes
Received message ReadFile: bcm2710-rpi-cm3.dtb
File read: bcm2710-rpi-cm3.dtb
libusb_bulk_transfer returned 0
Received message GetFileSize: config.txt
File size = 1590 bytes
Received message ReadFile: config.txt
File read: config.txt
libusb_bulk_transfer returned 0
Received message Done: config.txt
Second stage boot server done

and then tries to boot. It gets stuck but that besides the point.

When using my custom board i get:

philip@Dell:~/usbboot$ sudo ./rpiboot -v -d /media/philip/ext4_Entwicklung/t
Waiting for BCM2835/6/7
Device located successfully
Initialised device correctly
Found serial number 0
Sending bootcode.bin
libusb_bulk_transfer returned 0
Writing 50248 bytes
libusb_bulk_transfer returned 0
Successful read 4 bytes
Waiting for BCM2835/6/7
Device located successfully
Initialised device correctly
Found serial number 1
Second stage boot server
Received message GetFileSize: autoboot.txt
Cannot open file autoboot.txt
Received message GetFileSize: config.txt
File size = 1590 bytes
Received message ReadFile: config.txt
File read: config.txt
libusb_bulk_transfer returned 0
Received message GetFileSize: recovery.elf
Cannot open file recovery.elf
Received message GetFileSize: start.elf
File size = 2868132 bytes
Received message ReadFile: start.elf
File read: start.elf
libusb_bulk_transfer returned 0
Received message GetFileSize: fixup.dat
File size = 6688 bytes
Received message ReadFile: fixup.dat
File read: fixup.dat
libusb_bulk_transfer returned 0
Received message GetFileSize: recovery.elf
Cannot open file recovery.elf
Received message GetFileSize: config.txt
File size = 1590 bytes
Received message ReadFile: config.txt
File read: config.txt
libusb_bulk_transfer returned 0
Received message GetFileSize: dt-blob.bin
Cannot open file dt-blob.bin

And i just halts from here. Same thing as before with the custom image build with https://github.com/raspberrypi/scriptexecutor

@FiesoDuck

This comment has been minimized.

Copy link
Author

@FiesoDuck FiesoDuck commented Apr 3, 2018

I am also curious as to why using sudo ./rpiboot works as intended but sudo ./rpiboot -d [directory] fails the way it does.

I can "flash" the CMs and use them just fine in our application. We followed the original schematics closely when designing our board. If the hardware is faulty, how does the device boot in mass storage at all? How can this issue be hardware related?

@pelwell

This comment has been minimized.

Copy link
Contributor

@pelwell pelwell commented Apr 3, 2018

Very little is done between those two reads of config.txt (at least in terms of functionality - there's a fair amount of code involved). The initial startup is complicated due to some chicken vs egg sequencing problems when processing the config settings. As a result, an initial pass is performed to establish some facts about the platform. After this, just enough is done to determine which of the config,txt conditional sections to interpret and which to skip before it is parsed again, this time for real.

The additional steps are:

  1. reading the EDID from any attached HDMI device,
  2. reading the serial number from OTP, and
  3. reading and acting upon dt-blob.bin (external or embedded).

Apart from reinitialising some internal state before the second pass, the next thing it does is attempt to find recovery.elf. To me this says that the problem is all in the setting of the GPIO states and their effects on the board.

Specifying a directory affects the way that rpiboot locates files to send. The only way that should make a difference to the target device is if the files being downloaded are different. Compare the contents of your user directory with /usr/share/rpiboot/msd/.

You can also get extra diagnostic output from rpiboot with -vv.

@ghollingworth

This comment has been minimized.

Copy link
Contributor

@ghollingworth ghollingworth commented Apr 3, 2018

@pelwell

This comment has been minimized.

Copy link
Contributor

@pelwell pelwell commented Apr 3, 2018

The fact that it goes on to look for dt-blob.bin suggests that can't be the case.

@FiesoDuck

This comment has been minimized.

Copy link
Author

@FiesoDuck FiesoDuck commented Apr 9, 2018

Specifying a directory affects the way that rpiboot locates files to send. The only way that should make a difference to the target device is if the files being downloaded are different. Compare the contents of your user directory with /usr/share/rpiboot/msd/.

ls /usr/share/rpiboot/msd/
bootcode.bin
start.elf

When i diff these files against the user files they are obviously different.
I proceeded with copying /usr/share/rpiboot/msd/bootcode.bin and start.elf to the user directory and
did sudo ./rpiboot -d [directory] again.

Both boards (custom and official) then boot into MSD. Sorry but i am kinda lost here, what does that
imply? Is this issue better suited for https://github.com/raspberrypi/scriptexecutor ?

@FiesoDuck

This comment has been minimized.

Copy link
Author

@FiesoDuck FiesoDuck commented Apr 9, 2018

To me this says that the problem is all in the setting of the GPIO states and their effects on the board.

As shown in the schematics above we haven't customized the GPIOs or the usage of them in our board. What problems with the GPIO states are you referring to?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.