DMA transfers not working on the STM32F7

[ruffy91]

The sd card on a STM32F7-DISCO board can no longer be mounted since DMA has been added to the sdcard.c driver (7319e96b9e2b5d9d3cc4897118e6f62d47524712) .
The shown error is:

PYB: sync filesystems
PYB: soft reboot
[SD] could not mount SD card
MicroPython v1.5.1-71-g9cc1831 on 2015-12-03; F7DISC with STM32F746
Type "help()" for more information.
>>> 

sd.read() gives back a bytearray which contains corrupt data.
example (double and triple , %, q, o, h, y, _ etc.):

>>> buffer=pyb.SD.read(512)
>>> print(buffer)
bytearray(b'\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x01\x00\x00%\x01\x00\x00\x00\x01\x00\x00\\\x02\x00\x00\x00\x02\x00\x00]\x00\x00\x00\x00\x01\x00\x00b\x01\x00\x00\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x01\x00\x00n\x02\x00\x00\x00\x02\x00\x00o\x00\x00\x00\x00\x02\x00\x00q\x00\x00\x00\x00\x02\x00\x00s\x00\x00\x00\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x01\x00\x00e\x02\x00\x00\x00\x01\x00\x00h\x00\x00\x00\x00\x01\x00\x00g\x00\x00\x00\x00\x01\x00\x00m\x01\x00\x00\x00\x01\x00\x00n\x02\x00\x00\x00\x01\x00\x00o\x00\x00\x00\x00\x01\x00\x00q\x00\x00\x00\x00\x01\x00\x00s\x00\x00\x00\x00\x01\x00\x00y\x00\x00\x00\x00\x01\x00\x00\x89\x00\x00\x00\x00\x01\x00\x00\x88\x00\x00\x00\x00_\xff\xd0\xdc\xc6\xd5\x9e\xbc\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff')

[dpgeorge]

I don't have an F7 board so I can't work on this.

[dhylands]

I'll put this on my to-do list

[ruffy91]

The same problem applies to I2C reads/writes which use DMA. The result contains corrupted data, no matter how many bytes I read with I2C.mem_read(). Also doesn't matter if I use 100kBaud or 400kBaud.

[dpgeorge]

We probably need a compile time option to use/not use the DMA. Eg MICROPY_HW_USE_DMA.

[ruffy91]

I think DMA should be fixed on the F7. The problem can't be very big, as the F7 DMA peripheral is very similar to the F1 and F4 series. I'm trying to do it myself but until now I couldn't find the cause for the corrupted transfers.
i2c.c checks if IRQs are disabled and then uses the non-DMA method, but I don't know how to force the IRQs to disabled while doing I2C transfers.
I will change the code to always use the non-DMA method and report back tomorrow.
I already got the SD-Card to work again with the older sdcard.c which doesn't use DMA.

[dpgeorge]

but I don't know how to force the IRQs to disabled while doing I2C transfers.

In Python:

irq_state = pyb.disable_irq()
i2c.send(...)
pyb.enable_irq(irq_state)

In C, it's similar, but use the functions disable_irq and enable_irq.

[ruffy91]

Disabling I and D caches (in HAL_MspInit()) gets the DMA to work!
There seems to be additional work to be done so that the caches can be enabled.
And changing the clock configuration for the STM32F7DISC from (50MHz USB Clock)
#define MICROPY_HW_CLK_PLLN (400)
#define MICROPY_HW_CLK_PLLQ (8)
to (48MHz USB Clock)
#define MICROPY_HW_CLK_PLLN (336)
#define MICROPY_HW_CLK_PLLQ (7)
got USB MSC to work without problems with the sdcard

[dhylands]

Interesting. I know that with the I & D caches disabled, the non-DMA path for te sdcard doesn't work reliably (the code slows down enough that it allows the FIFO to empty which the sdcard doesn't like).

That's a really useful clue (about the caches). I guess we need to do cache flushing/invalidation just like on the A-series or something (I'm familiar with caching issues and DMA on the larger processors - and the M4 has DMA going through the cache - so there aren't any problems. Obviously the M7 is different).

Do you want to do a PR for the clock configuration? If not, I can do it.

[ruffy91]

I'll do a pull request for the clock configuration
For now i'll try to reenable the I-Cache and test again. I'll then do a pull request for the cache configuration (next week).
Second step is enabling the D-cache in write-through mode which should work as with disabled D-cache (I'm not sure if this is only true for TX or also for RX), but with a chance that the processor halts while waiting for the underlying write operation to finish. This should none the less be faster than a disabled D-cache.
In the future the DMA driver should be extended so that it supports the F7, which requires different things:

• RX and TX DMA buffers must be aligned to the D-cache lines
• Cache lines must be flushed + invalidated to the right times
• DMA buffers must not be in the DTCM memory

I've found some very good resources on this here: http://www.nuttx.org/doku.php?id=wiki:howtos:port-drivers_stm32f7

[dhylands]

Second step is enabling the D-cache in write-through mode which should work as with disabled D-cache (I'm not sure if this is only true for TX or also for RX), but with a chance that the processor halts while waiting for the underlying write operation to finish. This should none the less be faster than a disabled D-cache.

Yeah - that's just a fact of life when dealing with DMA. There are occasions where using uncached buffers can offer better performance/less cache thrashing, but I think that type of operation is only available when you have an MMU.

The data sheet for the F7 series specifically mentions that the DMA can use DTCM.

Generally the operations for performing cache coherency goes something like this:

For Memory to Peripheral direction, you need to flush the caches to memory and wait for the flush to complete before initiating the DMA.

For Peripheral to Memory direction, you need to invalidate the caches for the destination memory addresses and wait for the invalidate to complete before starting the DMA.

For bi-dierctional, you have to do both. Flush first, wait for flush to complete, invalidate second, wait for invalidate to finish, and then start DMA.

The buffers don't necessarily need to cache line aligned, but it definitely helps. The places that will cause problems if if you invalidate a cache line, and while you're performing your DMA, an interrupt handler comes along and touches something in the cache line but outside the buffer.

[Anton-2]

I'll need to dig further on the documentation, but I think that you can tag regions as cachable or not, using the mpu.
As configured before disabling I/D cache, with micropython on a STM32F7 DISCO, internal sram goes thru the D/I cache, but external SDRAM does not : I've got display artifacts on the LCD when the frame buffer is on internal SRAM, and not in external SDRAM.

[Anton-2]

I've got I2C with DMA working reliably on the F7 by doing SCB_CleanDCache_by_Addr before sending data and SCB_CleanInvalidateDCache_by_Addr before receiving it.

Clean is CMSIS way of saying flush, so it's basically what @dhylands said above. And I still don't understand why you need to Flush and Invalidate before starting the DMA when receiving data, but doing it after does not work...

I'm calculating all the cache lines spanned by the buffer, so the buffer does not need to be cache line aligned.

If nobody's working on this, I'm ready to prepare a PR for it, but need help on :

• where to put those functions (or macros ?) so that it's available for all device drivers ?
• what are the existing drivers using DMA (beside I2C and sd card) that need to be patched ?
• testing it (i've got only very basic I2C devices)...