Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ESP32 RMT Implementation #5184

Closed
wants to merge 20 commits into from
Closed

Conversation

@mattytrentini
Copy link
Contributor

mattytrentini commented Oct 5, 2019

This is the start of a MicroPython implementation for RMT on the ESP32.

RMT allows accurate (down to 12.5ns) sending and receiving of pulses. This implementation currently only provides transmit features and there are some limitations:

  • Blocking only (send_pulses is now non-blocking)
    • Non-blocking is possible but increases complexity
    • May need some help looking at how to implement the ISR and pass that back as an optional callback...
  • Looping is not exposed (it's possible to loop a pattern indefinitely)
    -Requires non-blocking
  • Carrier features - selecting a base frequency that can be modulated - is not yet provided
  • Idle level is not configurable
  • Only one memory block per channel

(Update: non-blocking and looping have been implemented)

I'd be curious which of these features is most important to people.

The short-term intent is to allow this to be used to provide a solid NeoPixel implementation. On the ESP32 it's difficult to make toggle pins accurately enough since the operating system periodically interrupts. The RMT provides accurate timing independent of the OS.

Note that FastLED, a popular C library for controlling Neopixels, successfully uses RMT for the ESP32 implementation.

Basic usage:

>>> import esp32
>>> from machine import Pin
>>> r = esp32.RMT(1, pin=Pin(18), clock_div=80)
>>> r
RMT(channel=1, pin=18, clock_div=80)
>>> r.send_pulses((100, 2000, 100, 4000), start=0)  # Send 0 for 100*1e-06s, 1 for 2000*1e-06s etc

The length of the buffer is restricted only by available memory.

Managing resources is not-quite-right yet (once an RMT channel has been allocated it isn't deintialised on soft boot) but I expect to address that soon.

From PyCom's RMT documentation (take care not to inspect their implementation since it's an incompatible license) they appear to offer three different ways to configure a tx buffer. The second model is similar to the one implemented here. Which, if any, of the others would be of most use?

For reference there are a few related issues:
#5117
#5157
#3453

Although this RMT implementation isn't complete yet I think it is in a useful state and I thought it worthwhile to submit and gather feedback for the future direction.

docs/esp32/quickref.rst Outdated
The RMT is a module, specific to the ESP32, that allows pulses of very accurate interval
and duration to be sent and received::
Comment on lines 371 to 372

This comment has been minimized.

Copy link
@mcauser

mcauser Oct 5, 2019

Contributor

I'd expand this paragraph to show RMT is short for Remote Control and that it was originally built for IR but can be repurposed.

The RMT (Remote Control) module, specific to the ESP32, was designed to send and receive infrared remote control signals. Due to flexibility of module, the driver can also be used to generate or receive many other types of digital signals. The module allows pulses of very accurate interval and duration to be sent and received::

This comment has been minimized.

Copy link
@mattytrentini

mattytrentini Oct 7, 2019

Author Contributor

Good suggestion! I've updated the paragraph accordingly.

@mcauser

This comment has been minimized.

Copy link
Contributor

mcauser commented Oct 5, 2019

Using this driver, I am able to generate a signal for a FS1000A ASK/OOK 433MHz transmitter to ring an Aldi Delta doorbell.

# convert binary data to pulses
def build_pulses(data, long, short, reset, preamble):
    pulses = [preamble, short, reset]
    one = [long, short]
    zero = [short, long]
    for c in reversed(data):
        pulses[1:1] = one if c == '1' else zero
    return pulses

# convert int data to zero padded binary string
def data_to_str(data, len):
    s = ('{:0%db}' % len).format(data)
    return s[-len:]

# transmit
def tx(data, len, repeats):
    if not isinstance(data, str):
        data = data_to_str(data, len)
    pulses = build_pulses(data, 1070, 358, 11070, 100)
    for _ in range(repeats):
        r.send_pulses(pulses)

# main
import esp32
from machine import Pin
r = esp32.RMT(1, Pin(18), 80)

# data = "100111000001110000000001"
data = 10230785
data_len = 24 # bits
repeats = 20

tx(data, data_len, repeats)
# doorbell rings!!
TinyPICO FS1000A
18 Data
3V3 VCC
GND GND

fs1000a

urh

@MrSurly

This comment has been minimized.

Copy link
Contributor

MrSurly commented Oct 7, 2019

Although this RMT implementation isn't complete yet I think it is in a useful state and I thought it worthwhile to submit and gather feedback for the future direction.

I'm wondering if this could be used to parse RC PPM (multiple servo channels on one wire), and then drive servos?

I have an old robot that I built that receives all the inputs from the RC TX via PPM, decodes them, then has to drive the wheels using normal RC PWM signals. There's little direct coupling between them because there's a decent amount of math that converts the incoming signals to the proper motor drive. The original runs on a teensy, and I've since lost the code. Would be neat to re-implement in MP.

@UnexpectedMaker

This comment has been minimized.

Copy link

UnexpectedMaker commented Oct 9, 2019

I got this working driving panels and strips of GRBW and GRB ws2812's in my live stream this morning and it's working great! We had some teething problems yesterday, but once @mattytrentini worked out how to force the idle state to be low, everything fell into place.

We definitely need a way of having deinit be auto called on an initialised channel when code fails, or at least have it clear itself on a soft reboot. Right now if the code fails before deinit is called, a hard reset is required to clear it again.

Or even just a:
deinit( channel )
So deinit can be called to clear a channel before an init attempt.

Also, the default error that appears when it's not cleared and you try to re-init is vague:
OSError: ESP_ERR_INVALID_STATE

Otherwise, I'm excited to keep using this and start building a new modern RGB LED driver library for the ESP32!

@mattytrentini

This comment has been minimized.

Copy link
Contributor Author

mattytrentini commented Oct 9, 2019

I got this working driving panels and strips of GRBW and GRB ws2812's in my live stream this morning and it's working great! We had some teething problems yesterday, but once @mattytrentini worked out how to force the idle state to be low, everything fell into place.

I have pushed that (one character!) fix. Thanks for finding it! A better fix will be to expose the idle level to allow it to be configured for the channel. Will get on to that.

(BTW it's really not clear to me what the purpose is of setting config.tx_config.idle_output_en to 0 is...)

We definitely need a way of having deinit be auto called on an initialised channel when code fails, or at least have it clear itself on a soft reboot. Right now if the code fails before deinit is called, a hard reset is required to clear it again.

I hear you, this is the next issue on my list. :)

Or even just a:
deinit( channel )
So deinit can be called to clear a channel before an init attempt.

Oh, there is a deinit: r.deinit(). Should probably document that 😛

Otherwise, I'm excited to keep using this and start building a new modern RGB LED driver library for the ESP32!

Thanks for sticking with it when there were issues!

@nevercast

This comment has been minimized.

Copy link
Contributor

nevercast commented Oct 9, 2019

BTW it's really not clear to me what the purpose is of setting config.tx_config.idle_output_en to 0 is...

That manipulates register RMT_IDLE_OUT_EN_CHn, which I gather disables the output enable driver on the GPIO so that the pin is not driven. This is specultation.

You could test the idle_en feature by disabling it and see if you can sink or source any current through that pin.

I noticed the build is failing with a code size issue, @mcauser was having a similar issue with 4 bytes being the problem, the solution was to rebase on to master.

#5118 (comment)

@mattytrentini

This comment has been minimized.

Copy link
Contributor Author

mattytrentini commented Oct 9, 2019

I'm wondering if this could be used to parse RC PPM (multiple servo channels on one wire), and then drive servos?

Obviously we'd need to add receive capabilities but yes, from that link you sent, I believe RMT could decode PPM.

I guess that means receive functionality would be high on your list of requests? ;)

@UnexpectedMaker

This comment has been minimized.

Copy link

UnexpectedMaker commented Oct 9, 2019

Oh, there is a deinit: r.deinit(). Should probably document that 😛

I am already using that - but if it never get's hit ( code crash before ) then it never gets called. And it requires the rmt channel object, so I can't use that if I can't re-init the channel ;)

I'm after a deinit ( channel number ) - to clear a channel, without having a reference to the channel object.

Is that even a Pythonic thing to do? ;)

@nevercast

This comment has been minimized.

Copy link
Contributor

nevercast commented Oct 9, 2019

Oh, there is a deinit: r.deinit(). Should probably document that

I am already using that - but if it never get's hit ( code crash before ) then it never gets called. And it requires the rmt channel object, so I can't use that if I can't re-init the channel ;)

Maybe its worth not throwing an error when we init, just reinit. I can call Pin.init as many times as I like. Can we think of anywhere else in MicroPython where calling .init or constructing a machine object throws instead of gracefully reinit ?

@UnexpectedMaker

This comment has been minimized.

Copy link

UnexpectedMaker commented Oct 9, 2019

Oh, there is a deinit: r.deinit(). Should probably document that

I am already using that - but if it never get's hit ( code crash before ) then it never gets called. And it requires the rmt channel object, so I can't use that if I can't re-init the channel ;)

Maybe its worth not throwing an error when we init, just reinit. I can call Pin.init as many times as I like. Can we think of anywhere else in MicroPython where calling .init or constructing a machine object throws instead of gracefully reinit ?

That sounds even better! Can haz plz?

@dpgeorge dpgeorge added the port-esp32 label Oct 9, 2019
@nevercast

This comment has been minimized.

Copy link
Contributor

nevercast commented Oct 9, 2019

Umm. Did you merge instead or rebase? That's a lotta commits

@mattytrentini

This comment has been minimized.

Copy link
Contributor Author

mattytrentini commented Oct 9, 2019

Apologies, in my rush to rebase yesterday I’ve messed something up. Will try to sort it out asap.

@mattytrentini mattytrentini force-pushed the mattytrentini:esp32_rmt branch 2 times, most recently Oct 11, 2019
@nevercast

This comment has been minimized.

Copy link
Contributor

nevercast commented Oct 11, 2019

image

@mattytrentini

This comment has been minimized.

Copy link
Contributor Author

mattytrentini commented Oct 11, 2019

Apologies, in my rush to rebase yesterday I’ve messed something up. Will try to sort it out asap.

Should be sorted out now, huge thanks to @mcauser for help fixing the screwed-up rebase!

@nevercast

This comment has been minimized.

Copy link
Contributor

nevercast commented Oct 19, 2019

Hi @mattytrentini

Do you want to develop this incrementally and get the TX functionality landed in master earlier, or do you want to continue developing your future features first and then bring it all in at once?

@nevercast

This comment has been minimized.

Copy link
Contributor

nevercast commented Oct 19, 2019

Have tried this RMT code with my own OOK transmitter that just arrived from China, and its a bundle of joy!
image

@mattytrentini

This comment has been minimized.

Copy link
Contributor Author

mattytrentini commented Oct 24, 2019

@nevercast I'd prefer to get it in early. Correctly managing resources is critical (and, until fixed, must prevent this being merged) but other transmit methods - different ways to specify the pulse pattern as well as specifying using the carrier methods - are nice-to-have.

Receive can come later; I'm guessing that API will be more difficult to get right.

@mattytrentini

This comment has been minimized.

Copy link
Contributor Author

mattytrentini commented Dec 16, 2019

The last couple of commits have slightly changed the API.

pin and clock_div are now keyword arguments rather than positional, the former is required, the latter is not (defaults to 8, providing a 100ns resolution). This is for some measure of consistency with other machine API's - it was also pretty obscure what the third parameter was.

Documentation has been updated as well so there's now a library reference as well as the quick ref entry.

resolution and max_pulse_length have been removed. source_freq and clock_div now have accessors. The idea is that resolution can be determined from these two parameters.

@nevercast I have given it a little thought (but only a little!); the carrier details are in addition to other configuration options. The carrier settings allows the high/low output of the RMT to be modulated to a frequency/duty - but the bit stream is defined exactly the same way. So I'm expecting additional parameters to be specified when constructing the RMT channel.

@robert-hh I'm still trying to get the additional transmit options in but I'm fast running out of time...

The RMT (Remote Control) module, specific to the ESP32, was originally designed
to send and receive infrared remote control signals. However, due to a flexible
design and very accurate (as low as 12.5ns) pulse generation, it can also be
used to transmit or receive many other types of digital signals::

This comment has been minimized.

Copy link
@dpgeorge

dpgeorge Dec 16, 2019

Member

This kind of historical language is better suited to the reference docs of esp32.RMT rather than a quick ref. For here I'd suggest something short and to the point like "The RMT is esp32-specific and allows generation of accurate digital pulse trains with 12.5ns resolution. Usage is:".

So, in the example above, the 80MHz clock is divided by 8. Thus the
resolution is (1/(80Mhz/8)) 100ns. Since the ``start`` level is 0 and toggles
with each number, the bitstream is ``0101`` with durations of [100ns, 2000ns,
100ns, 4000ns].

This comment has been minimized.

Copy link
@dpgeorge

dpgeorge Dec 16, 2019

Member

Again, these 3 paragraphs are better suited to the reference docs. Here should be just enough info to use it, and a link to the main reference.

@@ -101,6 +101,50 @@ The Ultra-Low-Power co-processor

Start the ULP running at the given *entry_point*.

RMT
---

This comment has been minimized.

Copy link
@dpgeorge

dpgeorge Dec 16, 2019

Member

Here could go some background info about it, moved from the quickref.

required and identifies which RMT channel (0-7) will be configured. *pin*,
also required, configures which Pin is bound to the RMT channel. *clock_div*
is an 8-bit clock divider that divides the source clock (80MHz) to the RMT
channel allowing the *resolution* to be specified.

This comment has been minimized.

Copy link
@dpgeorge

dpgeorge Dec 16, 2019

Member

resolution shouldn't be in *'s because it's not a parameter.


.. method:: RMT.clock_div()

Return the clock divider. Note that *resolution* is

This comment has been minimized.

Copy link
@dpgeorge

dpgeorge Dec 16, 2019

Member

resolution shouldn't be in *'s

mp_uint_t num_items = (pulses_length / 2) + (pulses_length % 2);
if (num_items > self->num_items)
{
//printf("realloc required, old=%u, new=%u\n", self->num_items, num_items);

This comment has been minimized.

Copy link
@dpgeorge

dpgeorge Dec 16, 2019

Member

Please remove these debugging printfs.

@dpgeorge

This comment has been minimized.

Copy link
Member

dpgeorge commented Dec 16, 2019

Re additional ways to specify the pulse train: would they all go through the same send_pulses() method, or use new methods? Would be good to have an idea now how this will work, so that send_pulses() doesn't need to be changed/renamed later on.

Also, considering what other peripheral API use (eg SPI.write_readinto, I2C.writeto, DAC.write_timed, ADC.read_u16), how about renaming send_pulses() to writepulses() or write_pulses()?

@mattytrentini

This comment has been minimized.

Copy link
Contributor Author

mattytrentini commented Dec 16, 2019

(I agree with your feedback on the documentation; I'll update it)

Re additional ways to specify the pulse train: would they all go through the same send_pulses() method, or use new methods? Would be good to have an idea now how this will work, so that send_pulses() doesn't need to be changed/renamed later on.

I was expecting to follow PyCom's RMT interface here and use the same method but use type detection to determine which method to use. There's a combination of duration/data and int/list that will uniquely identify which method to use.

Also, considering what other peripheral API use (eg SPI.write_readinto, I2C.writeto, DAC.write_timed, ADC.read_u16), how about renaming send_pulses() to writepulses() or write_pulses()?

Sure. I prefer following PEP8 unless there's a good reason not to, so I'll change it to write_pulses().

@dpgeorge

This comment has been minimized.

Copy link
Member

dpgeorge commented Dec 16, 2019

I was expecting to follow PyCom's RMT interface here and use the same method but use type detection to determine which method to use. There's a combination of duration/data and int/list that will uniquely identify which method to use.

Ok, as long as it can be done without keyword args (apart from start) it sounds good to me.

@robert-hh

This comment has been minimized.

Copy link
Contributor

robert-hh commented Dec 16, 2019

Using Pycom's API would be fine. SO code behaves the same in both families. Also, I'm still worried about the freq/divider discussion. At the end, the resolution is important. So why not specifying that? The implementation might be a little bit more complicated, because it has to derive the best matching freq/divider pair. But that should not be too complicated.

@dpgeorge

This comment has been minimized.

Copy link
Member

dpgeorge commented Dec 17, 2019

Also, I'm still worried about the freq/divider discussion. At the end, the resolution is important. So why not specifying that?

So there are 3 options that I can think of:

  1. Stay close to what the IDF exposes / the hardware, and use source_freq and clock_div.
  2. Provide a minor level of abstraction via resolution_ps (picoseconds!).
  3. Follow how machine.Timer works and use tick_hz and period, such that the resolution is period/tick_hz. Could default tick_hz=1_000_000_000 then period would default to nanoseconds.

Options 2 and 3 require using very large integers. The arguments for and against 1 vs 2 are discussed in comments above.

Would be good to agree on what to do here...

@mattytrentini

This comment has been minimized.

Copy link
Contributor Author

mattytrentini commented Dec 17, 2019

I'm biased ;) but IMO option 1 is the only one without problematic implementation issues.

If we want to provide higher-level abstractions we can do it above the RMT interface; for example, provide a function to return a clock divider for a given desired resolution. Perhaps return a tuple of (source_freq, clock_div, actual_resolution) for a given desired_resolution?

@dpgeorge

This comment has been minimized.

Copy link
Member

dpgeorge commented Dec 18, 2019

I'm biased ;) but IMO option 1 is the only one without problematic implementation issues.

If we want to provide higher-level abstractions we can do it above the RMT interface

Ok, let's just go with clock_div for now. The API of this module is anyway preliminary and we can improve it based on feedback from users.

@dpgeorge

This comment has been minimized.

Copy link
Member

dpgeorge commented Dec 19, 2019

@mattytrentini if this is ready please remove the "WIP" from the title.

@mattytrentini mattytrentini changed the title WIP: ESP32 RMT Implementation ESP32 RMT Implementation Dec 19, 2019
@dpgeorge

This comment has been minimized.

Copy link
Member

dpgeorge commented Dec 20, 2019

Squashed in to 2 commits (for implementation and docs) and merged in 0e0e613 and 7f235cb

Thanks to all involved!

Note: I renamed send_pulses to write_pulses as agreed above, and also used mp_obj_get_array() instead of the mp_obj_tuple_get()/mp_obj_list_get() pair.

@dpgeorge dpgeorge closed this Dec 20, 2019
@mattytrentini

This comment has been minimized.

Copy link
Contributor Author

mattytrentini commented Dec 20, 2019

Note: I renamed send_pulses to write_pulses as agreed above, and also used mp_obj_get_array() instead of the mp_obj_tuple_get()/mp_obj_list_get() pair.

Makes good sense - thanks!

@robert-hh

This comment has been minimized.

Copy link
Contributor

robert-hh commented Dec 20, 2019

Thank you for providing this module.
I just made an initial trial of that RMT class. What I am missing or misunderstanding is the option to define the 'quiet' level of the channel. As I see it now, it is always 0. But it should be the opposite of the value given in start=xx for the level of the first pulse.
And still, I would have preferred the API, where you specify both level and duration of each pulse, as the basic interface, which then can easily be used to derive the other specialized variants, like the one you have implemented.

@nevercast

This comment has been minimized.

Copy link
Contributor

nevercast commented Dec 20, 2019

@robert-hh

This comment has been minimized.

Copy link
Contributor

robert-hh commented Dec 20, 2019

No. According to my test, the quite level is 0. If it was the last value in the data, it should change, whether you have an odd or even number of data points. But it does not.

@mattytrentini

This comment has been minimized.

Copy link
Contributor Author

mattytrentini commented Dec 20, 2019

I just made an initial trial of that RMT class.

Thanks for trying it out!

What I am missing or misunderstanding is the option to define the 'quiet' level of the channel. As I see it now, it is always 0. But it should be the opposite of the value given in start=xx for the level of the first pulse.

For this release I wasn't able to add "idle level" which is the name Espressif use for the feature you're referring to. So, for now, the idle level can only be zero. Incidentally Espressif don't infer it from the values you specify, it's explicitly set.

And still, I would have preferred the API, where you specify both level and duration of each pulse, as the basic interface, which then can easily be used to derive the other specialized variants, like the one you have implemented.

I'm afraid I ran out of time before I could implement it. I'm most of the way through the implementation (for all three of the tx methods) but haven't added any documentation yet and just couldn't squeeze out another couple of hours to complete it all. I'll raise it as a PR as soon as I can!

@robert-hh

This comment has been minimized.

Copy link
Contributor

robert-hh commented Dec 20, 2019

Incidentally Espressif don't infer it from the values you specify, it's explicitly set.

There must be a way to set the idle level, because the Pycom version supports it.

@robert-hh

This comment has been minimized.

Copy link
Contributor

robert-hh commented Dec 20, 2019

Setting in your code, line 101

config.tx_config.idle_level = 1;
config.tx_config.idle_output_en = true;

works as expected to set an idle level of 1. According to my tests, the second line seems not to be required, but it sounds related.
P.S.: Adding a keyword parameter idle_level=x also works. The changes to the code:
After line 74, add:

        { MP_QSTR_idle_level,                  MP_ARG_KW_ONLY | MP_ARG_INT, {.u_int = 0} }, // 0 as default

Change then line 102 int0:

    config.tx_config.idle_level = !! args[3].u_int;
@mattytrentini

This comment has been minimized.

Copy link
Contributor Author

mattytrentini commented Dec 22, 2019

Incidentally Espressif don't infer it from the values you specify, it's explicitly set.

There must be a way to set the idle level, because the Pycom version supports it.

Sorry, I chose my wording poorly; as you've already figured out, yes it's possible to set the idle level. My minor point was that, at the IDF API, the idle level isn't inferred from the data specified, instead it must be configured explicitly.
It would be possible however to infer it (same level as last value) inside the MicroPython RMT module. Though I think I prefer requiring it to be explicitly set (with the kw argument as you've suggested)...

@mcauser

This comment has been minimized.

Copy link
Contributor

mcauser commented Dec 22, 2019

To make it output a 38KHz carrier signal for an IR TX LED:

config.tx_config.carrier_en = 1; // was 0
config.tx_config.carrier_duty_percent = 30; // was 0
config.tx_config.carrier_freq_hz = 38000; // was 0

Should be pretty easy to parameterise these options.

Adding the carrier lets me emulate a physical remote and is detected by both IR RX 1838 LED via Saleae Logic and YS-IRTM uart module. Without the carrier, the IR TX LED is generating a signal the receivers aren't detecting.

Works on my TinyPICO running v1.12 with above carrier enable:

import esp32
from machine import Pin
r = esp32.RMT(0, pin=Pin(18), clock_div=80)
def nec(add, cmd, pulse=562):
	data = [16*pulse, 8*pulse] # 9ms leading, 4.5ms space
	zero = [pulse, pulse] # 562.5us
	one = [pulse, 3*pulse] # 562.5us, 1687.5us
	for i in range(8):
		data.extend(one if (add >> i) & 1 else zero) # address
	for i in range(8):
		data.extend(zero if (add >> i) & 1 else one) # inverse address
	for i in range(8):
		data.extend(one if (cmd >> i) & 1 else zero) # command
	for i in range(8):
		data.extend(zero if (cmd >> i) & 1 else one) # inverse command
	data.extend(zero) # not sure if this is needed
	return data
data = nec(0x00, 0x45, 562)
r.write_pulses(data, start=1)

I had to use clock_div=80 instead of clock_div=8 as 9ms is 90,000 and exceeds the 15bits for the period. 1/2 a microsecond in lost precision doesn't seem to throw the IR receivers.

@IveJ

This comment has been minimized.

Copy link

IveJ commented Dec 22, 2019

@mattytrentini

This comment has been minimized.

Copy link
Contributor Author

mattytrentini commented Dec 29, 2019

@robert-hh (and anyone else interested) I have started on the next revision of RMT. I've added idle_level, tx_carrier settings and the other methods (same as PyCom) to specify the transmit pulses - including the double tuple variant you requested.

Sorry I couldn't get any of this done before v1.12!

I'll raise a WIP PR in the coming days.

@MrSurly

This comment has been minimized.

Copy link
Contributor

MrSurly commented Jan 15, 2020

@mattytrentini I'm interested -- thanks for your hard work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
9 participants
You can’t perform that action at this time.