-
Notifications
You must be signed in to change notification settings - Fork 192
Implement I2S #232
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement I2S #232
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for writing this driver ! Nice that you got it working ❤️
I have a couple of questions about the heavy use of generics for the pins. Did you had a problem somewhere i didn't see why this generic approach was needed? I would think if the Configs just take Options for things the user don't need a lot of this can be reduced.
I left a couple of comments but keep in mind a maybe totally wrong about thinks.
to give an concrete example how i would have made it without generics on the gpiostruct ->
with the .pin() method you always have access to the underlying gpio number |
I wouldn't hurry up with this. Two problems:
I'll follow up with alternative suggestions tmr. |
yeah should have put Peripheral there .. so more something like --->
|
Thanks for the comments @Vollbrecht and @ivmarkov. I'll wait for @ivmarkov's suggestions before proceeding, though will fix Config.toml to revert my accidental local commit. |
in the other drivers like spi we just consume the Peripheral and dont store it after config is finishd. So this is something he might be playing into but we will see . |
The fact that we do not (need to) store the pins does not mean we consume them. |
Thanks for all the comments. I'm looking into incorporating them now. |
Ugh. Ok, I realized why it's problematic initialize the channels in the driver The alternative is to add an interrupt handler argument in there, too, but... the parameters are getting messy. I don't like having the pins taken by position, either, since it's easy to mix them up (taking them in the config by name made this nice at the expense of generic hell). There are other things that people might want to do to the channel while it's in the REGISTERED state before it moves to the READY state, too. |
Ok, I refactored this based on all the comments. This sample code has also been updated to work again. The CI workflows won't pass until there's a new release of esp-idf-sys with esp-rs/esp-idf-sys@ce83a37 in there. |
Whew. Ok, I think this is ready for re-review, @ivmarkov and @Vollbrecht. After a lot of testing with code in https://github.com/dacut/test-esp-sound, I realized that trying to do lifetimes with the callbacks was next to impossible. You quickly end up with self-referential lifetimes between the driver and the callback, and that's just a mess. It's far better to pass the callback into the driver and let it own it. It does mean the callback needs to be 'static, however. Alas, my testing uncovered fundamental issues with the callback mechanism in ESP-IDF 5.0.x. It looks like this is broken because there's a queue that needs to be messaged to mark DMA buffers as available for reuse, but this queue is private. I've marked this as a bug in the commentary. Without writing to the DMA buffers directly, however, I'm experiencing pops, likely due to the delay between a DMA transfer completes and the copy into the next DMA buffer is ready. You can hear an example in this video, which comes from this sample code for a SparkFun Thing Plus. |
Can you broadly speaking elaborate if this only affecting a part that is only available in idf v.5.x or - is there an equivalent present in this wrapper api for this functionality using idf v4.4 ? I will have a look in the next days, if i can get it running on my c3 hardware and play with it a bit. |
Looks like the equivalent callback functionality isn't even available in 4.4. The generic write code is almost identical, however, so something must have been working; I can't imagine Espressif have not had working audio-out with ESP-IDF all these years. I do see a note in the 4.4 docs about using the APLL clock source for "high frequency clock applications." I've been using the default to the 160 MHz clock. I didn't imagine that "high frequency" would mean 48 kHz audio rates (1.5 MHz bit clock), but I wonder if this applies to the I2S MClk (which I'm not using, but the I2S peripheral might be)? That's running at 256x sample rate by default (12.288 MHz). I'll give this a shot when I get back to my test setup. |
Switching to APLL didn't help, but switching to a triangle wave output fixed this issue. The problem was actually in my sine wave computation; I had code that addressed the discontinuity at the edge of the DMA buffer length, but forgot to add that back in when I rewrote this. You're hearing the discontinuity in the sine wave, not anything to do with the I2S code. Please go ahead and review. I will add 4.4 compatibility in a separate PR. |
I do have v4.4 and PDM/TDM support now in a separate branch, but it's a lot more to review. (https://github.com/dacut/esp-idf-hal/tree/i2s-modes) I'll hold off on that PR until you get a chance to look at this updated PR. |
Let me know if you need help reviewing this and would like to have a virtual meeting to go over it. |
Very good thank you! I definitely want |
@dacut Fork https://github.com/dacut/esp-idf-hal/tree/i2s-modes looks quite good I must say! I might have some minor comments/suggestions only. Let me take a deeper look today/tmr... |
@ivmarkov Would you like me to merge it into this PR? I was holding off because I thought it might be too much to review at once. |
Merging? I thought your other branch is in fact superseding this PR? |
It builds on top of this PR (it's based off the i2s branch). I was keeping it separate just for sanity, but if you're already looking, I'll just merge i2s-modes into i2s. I think it requires one more change to esp-idf-sys as well. |
Please do. I was quickly comparing against the esp-idf-hal |
Alright, |
This change includes the I2S headers for ESP-IDF v4.4, allowing the Rust I2S driver being considered in this pull request to build on that version of ESP-IDF. esp-rs/esp-idf-hal#232
This change in esp-idf-sys is required to build on v4.4: The current HEAD is sufficient for building on v5.x. |
This change includes the I2S headers for ESP-IDF v4.4, allowing the Rust I2S driver being considered in this pull request to build on that version of ESP-IDF. esp-rs/esp-idf-hal#232
This isn't strictly necessary in alloc mode, but it makes the code easier to use.
Channels aren't used, so pools aren't needed (and the types aren't valid).
@ivmarkov, when you get a chance, can you fire off another CI check? I believe I've fixed all of the fixable issues (that is, everything except the compiler error): https://github.com/dacut/esp-idf-hal/actions/runs/5264225899/jobs/9515249060 Making it Also, if you have a link for the compiler issue, let me know. I didn't see it in esp-rs/rust. Thanks! |
LGTM, final look over the weekend, and then we likely merge! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Forget about the minor Debug
nits. The main two questions that needs addressing are:
- We are leaking memory when
alloc
is enabled, right? Is this how things are supposed to be? - Given all the equilibristics for implementing a
no-alloc
callback-safe version, why do we have the alloc variants? Can't we just remove them, which would take care of removing the memory leaks. This can also make the code safer, as you don't have to pass around and keep*mut
raw pointers. You can reduce the unsafe and pass around&'static mut
references instead.
src/i2s.rs
Outdated
callback: None, | ||
}; | ||
|
||
Box::leak(Box::new(channel)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This memory seems to be never reclaimed? Also, why do we need this impl, igven that you have a no-alloc impl anyway, which can be used regardless of whether the alloc
module is used or not?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Argh, you're correct. I was reclaiming it from a Box::from_raw
elsewhere, but that code went away.
To sum up: I wasn't aware that no_std
was even an option until the CI tooling failed with the Box
with this enabled. I put the no-alloc option in at the last minute to make it happy.
My main concern is this eats up size_of::<I2sChannel<dyn I2sRxCallback/I2sTxCallback>>
* 2 (32) bytes of RAM (*4 == 64 for ESP32), even for folks who don't use I2S, and there's not a lot of it (160 kB).
Should I add in the Box::from_raw
code, or eat the 32/64 bytes of RAM for everyone?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What I usually do in code elsewhere is that the "callback" code is only available when the alloc
feature is enabled. Otherwise, the user just does not have the option to use callbacks.
In practice this is not a problem, because using no_std
+ allocator with esp-idf-*
is more of a "hygiene" thing (as in don't overuse the Rust STD API if you don't need it) rather than a solution to an actual problem. Further, using no_std
without alloc
is even less of an actual use case, as ESP-IDF itself is heavily allocator-based, so the fact that the Rust wrappers on top won't allocate is not solving anything other than putting some discipline and forcing the Rust driver implementor to think where allocations are really needed.
So maybe you could leave the alloc
code only then.
Now, I realized your code has an even bigger problem - the "callback" stuff - the way you implemented it - is viral, because you demand that the channel itself (I2s(Rx|Tx)Channel
) has a stable memory location. This - in turn - means that the channels - if you leave only the Box-ing code and remove the static mut
MaybeUninit
tricks - will be boxed. But then, if the users compile without alloc
enabled, she would lose not just the option to set callbacks, but in fact the whole i2s code!
Putting the whole driver under an alloc
requirement then is also an option. BUT: digging even further, I think the crux of the problem is that your I2s(Rx|Tx)Callback
trait has a stronger API than necessary - it provides to the user
a &mut
reference to the I2s(Rx|Tx)Channel
instance - which - I think - is not really necessary? The trait (or closure) should just get an indication of what that channel it is (port number) and the event itself. The user_ctx
should be
the closure itself, so that the user can close over whatever state she wants. Not to mention that providing a &mut
reference to the channel is unsound in the first place, as you get mutable aliasing in that way (use might be already inside read/write or other functions of the channel that take a &mut
ref to itself when the callback ISR triggers).
Moreover, setting the callback should be marked with unsafe
, because the callback is executed in an ISR context and as such is extremely dangerous - for one, users cannot use the Rust STD API inside.
My suggestion:
- Leave only the
alloc
based code, remove thestatic mut
tricks - Demote the callback from an
I2s(RX|TX)Callback
instance to justF: FnMut(...)
- Parameterize the
set_callback
methods byF
and not the whole channel - Pass everything needed - channel port number, what type of event this is and the event itself as parameters to the closure
- User does not get access to the channel instance from inside the closure
- Store the
F
inside a double box (using two boxes of indirection is important with closures, see the callback code everywhere else) - Demote the tx and rx channels to regular members of the driver
- Last but not least - mark the
set_callback
methods as unsafe, because the closure will be executed from an ISR context
Finally, and if you are wondering, for what the callbacks might be useful - the one useful feature is implementing async
APIs on top of that driver. The callbacks will just awake the executor, while the read/write APIs will then be called with a timeout of 0.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@dacut Let me know what you think. If these changes feel as too much work for you, I can also try to implement them.
It is just that I'll be able to test it earliest in July.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, I think I see how the callbacks are supposed to work now. (Ignore my deleted comment.) Instead of handling the DMA event in the ISR, you're just supposed to wake up a task to handle the DMA event.
I'm used to platforms where the DMA has to be handled in the ISR (acknowledged at the very least) or the entire subsystem goes awry.
Will look at this tonight or tomorrow. (Work got busy...)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ivmarkov Can you clarify why double-boxing is used? (Note that it's missing in pcnt.rs, but present in gpio.rs and timer.rs). I would expect a Pin<Box<...>>
instead of Box<Box<...>>
.
src/i2s.rs
Outdated
callback: None, | ||
}; | ||
|
||
Box::leak(Box::new(channel)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ditto?
src/i2s.rs
Outdated
callback: None, | ||
}; | ||
|
||
Box::leak(Box::new(channel)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What I usually do in code elsewhere is that the "callback" code is only available when the alloc
feature is enabled. Otherwise, the user just does not have the option to use callbacks.
In practice this is not a problem, because using no_std
+ allocator with esp-idf-*
is more of a "hygiene" thing (as in don't overuse the Rust STD API if you don't need it) rather than a solution to an actual problem. Further, using no_std
without alloc
is even less of an actual use case, as ESP-IDF itself is heavily allocator-based, so the fact that the Rust wrappers on top won't allocate is not solving anything other than putting some discipline and forcing the Rust driver implementor to think where allocations are really needed.
So maybe you could leave the alloc
code only then.
Now, I realized your code has an even bigger problem - the "callback" stuff - the way you implemented it - is viral, because you demand that the channel itself (I2s(Rx|Tx)Channel
) has a stable memory location. This - in turn - means that the channels - if you leave only the Box-ing code and remove the static mut
MaybeUninit
tricks - will be boxed. But then, if the users compile without alloc
enabled, she would lose not just the option to set callbacks, but in fact the whole i2s code!
Putting the whole driver under an alloc
requirement then is also an option. BUT: digging even further, I think the crux of the problem is that your I2s(Rx|Tx)Callback
trait has a stronger API than necessary - it provides to the user
a &mut
reference to the I2s(Rx|Tx)Channel
instance - which - I think - is not really necessary? The trait (or closure) should just get an indication of what that channel it is (port number) and the event itself. The user_ctx
should be
the closure itself, so that the user can close over whatever state she wants. Not to mention that providing a &mut
reference to the channel is unsound in the first place, as you get mutable aliasing in that way (use might be already inside read/write or other functions of the channel that take a &mut
ref to itself when the callback ISR triggers).
Moreover, setting the callback should be marked with unsafe
, because the callback is executed in an ISR context and as such is extremely dangerous - for one, users cannot use the Rust STD API inside.
My suggestion:
- Leave only the
alloc
based code, remove thestatic mut
tricks - Demote the callback from an
I2s(RX|TX)Callback
instance to justF: FnMut(...)
- Parameterize the
set_callback
methods byF
and not the whole channel - Pass everything needed - channel port number, what type of event this is and the event itself as parameters to the closure
- User does not get access to the channel instance from inside the closure
- Store the
F
inside a double box (using two boxes of indirection is important with closures, see the callback code everywhere else) - Demote the tx and rx channels to regular members of the driver
- Last but not least - mark the
set_callback
methods as unsafe, because the closure will be executed from an ISR context
Finally, and if you are wondering, for what the callbacks might be useful - the one useful feature is implementing async
APIs on top of that driver. The callbacks will just awake the executor, while the read/write APIs will then be called with a timeout of 0.
Add Debug derivations where missing. Add Default derivations to enums now that MSRV supports them.
Oof... As I implement some of the suggestions, I'm recalling why some of the complexity had to be mirrored into Rust. The C SDK doesn't pass the port into the callback; only the SDK channel handle. Removing the Rust channel from the callback loses the I2S port as a result. You can't get the I2S port from an We can either store the port number alongside the callback, or just deduce it from the pointer value. |
This makes the rx/tx callbacks into functions instead of traits per @ivmarkov's recommendations. This helps with a number of ergonomic issues with the code (particularly around channels and lifetimes).
This mode was only supported in the v4 driver series, and not all that well at that. This doesn't appear to be a standard I2S operating mode anyway.
I've done a quick first-round implementation of these changes, but not tested them yet. I need to see if I can change the dyn FnMut() into a proper trait at some level (or, rather why it wasn't compiling when I had that syntax in there). |
Looks much better. A few nits:
Finally w.r.t. double-boxing - you are doing it already, kind of. Or in a way, the user - with the current implementation - would do the first |
Hey @dacut the changes above ^^^ have become quite big. How about me merging what we have already, and then you provide the final cleanups/fixes as a separate PR? |
@ivmarkov Will do. I was running into issues with removing the |
It should be always possible to do |
This makes a few changes suggested in the last bit of esp-rs#232: * Trait functions take impl FnMut instead of Box<dyn FnMut>. * Callback bookkeeping is no longer static (like the GPIO callbacks); instead, it's stored with the channel. * Added rx/tx_unsubscribe() methods to the channels. Also, fixed the naming of the I2sRxEvent/I2sTxEvent enums to be more inline with the brevity of Rust in general: * DataReceived -> RxDone * ReceiveOverflow -> RxOverflow * DataTransmitted -> TxDone * TransmitOverflow -> TxOverflow
Yep, have a new PR coming your way once my CI runs and verifies I've not done something else stupid. |
This makes a few changes suggested in the last bit of esp-rs#232: * Trait functions take impl FnMut instead of Box<dyn FnMut>. * Callback bookkeeping is no longer static (like the GPIO callbacks); instead, it's stored with the channel. * Added rx/tx_unsubscribe() methods to the channels. Also, fixed the naming of the I2sRxEvent/I2sTxEvent enums to be more inline with the brevity of Rust in general: * DataReceived -> RxDone * ReceiveOverflow -> RxOverflow * DataTransmitted -> TxDone * TransmitOverflow -> TxOverflow
* Ergonomic changes for I2S callbacks. This makes a few changes suggested in the last bit of #232: * Trait functions take impl FnMut instead of Box<dyn FnMut>. * Callback bookkeeping is no longer static (like the GPIO callbacks); instead, it's stored with the channel. * Added rx/tx_unsubscribe() methods to the channels. Also, fixed the naming of the I2sRxEvent/I2sTxEvent enums to be more inline with the brevity of Rust in general: * DataReceived -> RxDone * ReceiveOverflow -> RxOverflow * DataTransmitted -> TxDone * TransmitOverflow -> TxOverflow * Fix no-std, no-alloc builds. Fix leak introduced by changing the UnsafeCallback Box<Box<...>> to a *mut Box<...> (which isn't dropped automatically).
* Ergonomic changes for I2S callbacks. This makes a few changes suggested in the last bit of esp-rs/esp-idf-hal#232: * Trait functions take impl FnMut instead of Box<dyn FnMut>. * Callback bookkeeping is no longer static (like the GPIO callbacks); instead, it's stored with the channel. * Added rx/tx_unsubscribe() methods to the channels. Also, fixed the naming of the I2sRxEvent/I2sTxEvent enums to be more inline with the brevity of Rust in general: * DataReceived -> RxDone * ReceiveOverflow -> RxOverflow * DataTransmitted -> TxDone * TransmitOverflow -> TxOverflow * Fix no-std, no-alloc builds. Fix leak introduced by changing the UnsafeCallback Box<Box<...>> to a *mut Box<...> (which isn't dropped automatically).
This implements I2S standard mode, as a first run fix for #205. It does not yet implement PDM, TDM, or PCM modes.
There is an issue here that I'd like comments on. Currently, once a GPIO pin is passed into
esp_idf_hal::i2s::config::StdGpioConfigBuilder
, it cannot be reused, even after the driver is dropped. This seems to fit into the intended design ofPeripheral
, as documented here:But this is a surprise to @ivmarkov in this comment and doesn't feel natural to me, either.