writing data to storage while powered over USB messes stuff up

[turbinenreiter]

While working on a datalogger, I have the pyboard connected to the PC for power. Upon reset, the PC tries to mount the storage, and thereby messes up the file I'm trying to write.
When I write to the storage quicker (less data), it finishes before the PC messes everything up. Writing more data takes longer and the issue comes back up.
I have also tried to make my PC not automatically mount the drive, but that doesn't work, the file still ends up empty.
Is there a way to tell the board to only take power from USB and only connect after it's finished writing it's data?

[lurch]

See the discussion in #287

[dpgeorge]

The pyboard presents either 0:/ or 1:/ to the PC, depending on whether the SD card is inserted or not (see micropython.org/help/). So, the way around this would be to have the SD inserted, but write to 0:/.

But, that's not really a great solution.

Other solution is to edit boot.py to configure the pyboard to present CDC + HID (not CDC + MSC) to the PC. See the factory boot.py on 0:/.

If you put your custom boot.py on the SD card (1:/) with the pyb.usb_mode('CDC+HID') option, then whenever you have the SD card plugged in it will go into this mode. With the SD card out, it'll go into CDC+MSC mode with 0:/ presented to the PC.

Other solution would be for me to add an option in boot.py to force it to present either 0:/ or 1:/ to the PC (instead of choosing 1:/ if SD is inserted, else 0:/).

[turbinenreiter]

Ok, if it's mounted as HID I can not access the files on the SD-card via the pyboard?
So, I would:
*change boot to connect as HID
*put the main.py on the flash
*write the data to the sd-card
*take out the sd-card and put it in the PC to read my data-file

Is it possible to change the pyb.usb_mode() from the main.py or only during boot?
like:

pyb.usb_mode('CDC+HID')
file = open('file')
file.write(data)
file.close()
pyb.usb_mode('CDC+MSC')

[dpgeorge]

Try the following:

1. Use SD card for boot.py and main.py.
2. Configure boot.py to use CDC+HID
3. Write data to SD card.
4. Put SD card in PC to read data file.

No, you can't change the usb_mode once the board has booted.

[dpgeorge]

Alternatively, power it via a battery to log the data. Use VIN and GND for the battery and make sure it's connected the right way around! 3.6v to 10v is allowed for battery voltage.

[turbinenreiter]

In the end, that is what I want to do. It's just for the sake of easier development that I'd wish the board could do three things at once.
Thanks.

[dpgeorge]

I'd wish the board could do three things at once.

That would be nice :)

One final thing you can do: you can boot into safe mode by holding the usr switch while it (hard) boots. See micropython.org/help/. Safe mode does not execute boot.py, so you could have the SD card inserted, safe boot, and then you could read the SD card (it would also not execute your main.py, which I think is just what you want).

[turbinenreiter]

That always mounts the flash instead of the sd-card tough.

[dpgeorge]

That's true. I will need to add an option to boot.py to select which filesystem to present to the PC.

[chrismas9]

I think there will be a lot of data logger style applications where pyboard will regularly or continuously write files, and occasionally be plugged in to download those files. This conflict between MCU access and host MSC access will be a problem while debugging.

Would it be possible to have an option to control MSC access with the usr switch? This would allow you to debug your script using the CDC console, then enable the MSC to download a new version, or read the log files.

1. Enable the option in boot.py (not uPy rebuild). There are already USB options there.
2. When this option is enabled uPy would boot with the MSC disabled. I don't know how to do this, either don't start the USB driver, or make the drive read only to the host or not ready (like a USB DVD drive with no disk in it).
3. When the USR button is pressed enable host access to the flash and SD. Maybe indicate the mode with a LED.
4. Maybe another USR press to toggle back or if this is not possible a reset would be ok to get back to run / debug mode.

[lurch]

One way to do it (that works right now with the current uPy setup) would be to place this into boot.py :

# boot.py -- run on boot-up
# can run arbitrary Python, but best to keep it minimal

import pyb
#pyb.main('main.py') # main script to run after this one
#pyb.usb_mode('CDC+MSC') # act as a serial and a storage device
#pyb.usb_mode('CDC+HID') # act as a serial device and a mouse
pyb.LED(3).on()
pyb.delay(2000)
pyb.LED(4).on()
pyb.LED(3).off()
sw = pyb.Switch()
if sw():
    pyb.usb_mode('CDC+MSC')
    pyb.main('debug_app.py')
else:
    pyb.usb_mode('CDC+HID')
    pyb.main('normal_app.py')
pyb.LED(4).off()

So if you boot up / reset the pyboard without doing anything, after 2 seconds it will start running normal_app.py without any MSC, and the MCU will have exclusive access to the flash. But if you power up / reset the pyboard and press the USR button before the end of 2 seconds (and hold it down until the orange LED goes off), then it will start running debug_app.py (which of course could just be an empty file) and the flash will get mounted to the host in MSC mode :-)

You need to be careful to make sure you let go of the reset button before holding down the USR button, because otwerwise you might accidentally trigger the flash-reformat code. Damien - I wonder if it's worth making this harder to trigger so there's less likelihood of it being triggered accidentally? (e.g. hold down for three seconds, release for 2 seconds, hold down for another 3 seconds; with different LED colours giving you timing prompts)

[turbinenreiter]

That's beautiful, lurch.
Tuesday evening you will see an acceleration data plot from a water rocket posted to the forums. Case I get the adxl377 (which should be easy to read with the ADCs) and the bmp180 (harder, SPI) to work, you will get that too.

I think that an datalogger that logs the acceleration data to the SD-card and let's you switch modes the way lurch proposed would be a good thing to have in the examples. I will put that together and commit it.

[lurch]

Last time I looked at accelerometers (a couple of years ago?) they had a number of different sensitivity scales they could work at (+/- 2G, +/-8G, etc.) so I guess you might need to experiment to see which range works best with your water rocket. Looking forward to seeing the plots :) (matplotlib - another python project - is good for that)

[lurch]

Oh, and you'll also want to make sure that the micro sd card is secured and can't fall out / become disconnected ;-)

[turbinenreiter]

Well, the adxl377 has a range of +-200g, which is why I use it. With that big range we will hopefully see the start acceleration, which, according to calculations, can be up to 40g. The built in accelerometer with, as far as I know, +- 1.5g, will see the microgravity during freefall.

Already did part of that using Arduinos, but this board is way easier.

Yeah, talk about losing SD-cards. There is still one lying somewhere in the field with data measured by an MPU6050^^.

[KeithJRome]

Just curious - why wouldn't you just keep the storage private and provide a means to stream data out over the usb port in serial format when needed? It won't be as fast as direct file system access, but the root problem here is ultimately a shared-resource issue.

Rebooting the device into another mode isn't a good solution for systems that are actively operating - and the imposed startup delay might also be a problem in other cases.

It might work for some cases though. If you are OK with taking the device offline before extracting the data log then it would be fine.

[KeithJRome]

By the way, that's not meant as a criticism (I re-read what I wrote and realize it might be interpreted that way). I'm just trying to understand the problem that was faced as I don't have any hardware here to try anything myself and my first approach would have been to allow reading the data off the comm port without needing direct access to it.

[turbinenreiter]

Because I not only want to access the data, but also the scripts. This problem is about development workflow mainly.

[KeithJRome]

I see, thanks. Yeah, in that scenario the startup mode toggle seems like a good solution.

[EmmBee]

I've see something like what you are trying to do with a small action camera.
If the camera is powered over the usb port, but the data wires are open circuit then the board would enter its video recording function, keeping an eye on one or both USB data lines,
As this thing had its own small battery, it could then see you unplugs the power only connector and connect a real USB port - it now enters is USB FILE mode and allows you to ship out the stored data files (yes more than one, so any nasty only wrecks a relativity small lump of data.
because it has a small battery (or superCap) it can fail gracefully when the power goes.
It also had a function whereby earthing pin 4 (some times called pin X) to pin 5, on the USB socket turned the USB off, and used the data wires (2&3) for audio & video out).
Mike

[lurch]

My A-level Physics project actually involved water-rockets and data-loggers, many many years ago...

I wanted to investigate what proportion of water/air would give the most thrust; so I had the water-rocket clamped to a stand, with the 'exhaust' forced through an impeller, with the impeller connected to a dynamo, and the dynamo connected to a datalogger. This was so long ago the school's only data-logger was a big standalone device with an 8-segment-LEDs readout, so after each test I had to manually step through each reading one at a time, writing down all the values in my notebook, and then when I got home I had to copy the values from my notebook into a spreadsheet program on my Amiga. How times have changed!! IIRC, the most "thrust" I got with this setup was starting with a 50:50 mix, but there was probably huge margins of error.

However the best part was one time when the wrong pipe popped off under pressure, and I accidentally ended up soaking the head of department who was walking past at the time :-D