How to read UART bytes in queue after break? (IDFGH-2424)

[luksal]

I'm currently working on a DMX512 library for the ESP32 https://github.com/luksal/ESP32-DMX-RX .
DMX512 is a protocol based on RS485 where after a break signal 512 bytes are transmitted.

The main concept is running an infinite loop and call xQueueReceive periodically and then checking the event.type.
When a UART_DATA is received, I'm copying the received data to an array and waiting for the next event. That way I'm getting 120 bytes each event, in total 480 until the break occurs.

How can I receive the last 32 bytes? Because when event.type is UART_BREAK, then calling uart_get_buffered_data_len equals 0 and trying to use uart_read_bytes fails at this point.

[xiongyumail]

Can you provide the timing diagram of your UART port through a logic analyzer or oscilloscope?Especially before and after the UART_BREAK event. We want to know if there is a UART_DATA event from the UART_RXFIFO_TOUT after the UART_BREAK event

[macdroid53]

I have attached a csv file of the data/timing on the DMX input port. (I had to change the file name, just rename without '.txt'.

I have also provided the Saleae Logic capture file as well. You can download their software to look at it. (Same with the name, just rename without '.txt'.

DMX_capture.logicdata.txt

I have no idea how to capture UART_DATA event. Isn't that internal to the UART and reported by interrupt?
DMX_capture.csv.txt

[xiongyumail]

@luksal @macdroid53
Thank you for the wave file provided, I have a general understanding of what kind of protocol DMX-512 is. I think we can use the UART register directly, which can solve your problem faster and provide it as a solution. Before that, if you have implemented ESP32 as the source to send DMX-512, I can use another ESP32 for debugging.

[macdroid53]

DMX protocol is very simple. It starts with a break (the break is a low signal for approximately 88us). It then sends 513, 8bit values (each with 1 start bit, two stop bits, for a total of 11 bits per byte). This is sent at 250k baud.

I have no transmit code to offer, I'm working on a receive only application. (though I believe any serial source that could send 513 bytes @250K baud should be sufficient. See my reasoning below...)

The following describes what I think I'm seeing:
The receive code sees the break event (event.type == UART_BREAK). It then sees the data event (event.type == UART_DATA) 4 times. Each time uart_get_buffered_data_len() returns 120. Thus it reads 480 bytes. It never receives another UART_DATA event (because the threshold is never reached again during a transmission). The break event is the next event received. When this is received, uart_get_buffered_data_len() still returns 120 (this probably means uart_get_buffered_data_len() always returns 120 and is not set to the current number of bytes received, or the buffer has been cleared when the break is received), and any attempt to read the remaining 32 bytes (assuming they were received but the threshold never reached 120, so never triggers the UART_DATA event). Attempting to read the 32 bytes at this point causes a memory access fault.

Note the value 120. It is the same as #define UART_FULL_THRESH_DEFAULT (120) defined in uart.c.

If this description is correct and the driver does not takes steps to return bytes remaining in the buffer when less than the threshold and the break is detected, then any attempt to send an amount of data not evenly divisible by 120 would loose the remaining data. So, any serial source, say another esp32, could send 513 bytes. (I say 513, because the DMX "slots" are numbered 0 through 512, so the protocol actually sends 513 bytes.))

I think the driver knows it has data. It has two variables that imply this:
int rx_buffered_len; /*!< UART cached data length */
or possibly:
uint8_t rx_stash_len; /*!< stashed data length.(When using flow control, after reading out FIFO data, if we fail to push to buffer, we can just stash them.) */

If I read uart.c correctly, this condition would occur when:
break detect == true && rx_buffered_len > 0

[macdroid53]

I have been studying uart.c.

In the function:
uart_rx_intr_handler_default()

else if(uart_intr_status & UART_BRK_DET_INT_ST_M) { uart_reg->int_clr.brk_det = 1; uart_event.type = UART_BREAK;

it appears the actual break detect interrupt does nothing but return event.type set to UART_BREAK.
Once this has happened, the FIFO never gets read to the buffer because none of the things that trigger the FIFO read has happened. The actual move FIFO to buffer happens earlier in the code when this else statement is true:
else if ((uart_intr_status & UART_RXFIFO_TOUT_INT_ST_M) || (uart_intr_status & UART_RXFIFO_FULL_INT_ST_M) || (uart_intr_status & UART_AT_CMD_CHAR_DET_INT_ST_M) ) {

So...maybe the break detect needs to check the FIFO > 0 and act accordingly?

Or, am I way off base and confused (which is very possible. ;) )

[macdroid53]

Just wanted to check if there is anything else I can contribute to this?

[negativekelvin]

So...maybe the break detect needs to check the FIFO > 0 and act accordingly?

That sounds right. Now, the fifo is only emptied on threshold or idle timeout but you need it to be emptied on break also.

[koobest]

@luksal,
When the break event comes, you wait for the next event and then read the remaining 32 bytes of the previous packet. Whether this solution can solve your issue?

thanks !!

[negativekelvin]

@koobest that would add unnecessary latency because the next event could be threshold which would be an extra 1000 bit periods

[luksal]

@koobest that's not a good solution, because as @negativekelvin says this adds latency.

Perfect solution would be the possibility to check whether there are bytes in the fifo left and read them, when the break event occurs.

[macdroid53]

My experiments found the remaining bytes aren't there after the new break is detected.

As @luksal notes, being able to determine that there are remaining bytes when a break occurs would be the correct solution.

[negativekelvin]

What if you change this

esp-idf/components/driver/uart.c

Lines 799 to 802 in 647cb62

	else if ((uart_intr_status & UART_INTR_RXFIFO_TOUT)
	|| (uart_intr_status & UART_INTR_RXFIFO_FULL)
	|| (uart_intr_status & UART_INTR_CMD_CHAR_DET)
	) {

To

else if ( (uart_intr_status & (UART_INTR_RXFIFO_TOUT | UART_INTR_RXFIFO_FULL | 
                             UART_INTR_CMD_CHAR_DET ) )
              || ( (uart_intr_status & UART_INTR_BRK_DET) 
                      && (uart_ll_get_rxfifo_len(uart_context[uart_num].hal.dev) > 0) ) ){

You should then get data event and break event

[macdroid53]

Well, the uart.c I have is apparently older than the one you quote...so, I need to figure out how to update my local "/home/mac/.platformio/packages/framework-espidf/" I guess...

That said, I have questions about your proposed change.

I think I agree this will get a data and break events. But, what triggers the move of the FIFO contents to the ring buffer (which I think is where uart_get_buffered_data_len() and uart_read_bytes() get the size and data from...)

[negativekelvin]

what triggers the move of the FIFO contents to the ring buffer

That if block evaluating to true

[macdroid53]

Ok, so, not being able to figure out how to update my platformio 3.x ESP32-IDF, I show horned parts of the 4.x until I could compile. Then I modified uart.c with the suggestion above.

It appears to trigger the fifo read. But, I never see an event.size less than 120.

(I did try to read only 32 bytes at the break & first data event after, but that causes a panic.)

[negativekelvin]

I don't know if your changes caused any problems but what happens if you use the stock uart events example code?

[macdroid53]

Hmm...didn't try it with any stock examples. I don't think I'm using anything but stock uart events...but, I may not understand.

But, I did tinker some more with my code. And, the mod does appear to work... BUT I never see an event.size less than 120.

I think, there is something going on that causes the reading of data to run past the 512 bytes. I see the input count go over 900, then back to 0 when a break is detected again. (So, I get to 480 when the break is detected and the next data event has event.size at 120. A read of that gets the last 32 bytes, plus 88. Haven't figured out to check if the 88 are junk or the beginning of the next 512...) But, with the current code I get correct values for byte 1 thru 512. And, get correct values as they change (that is, as the source changes the byte values of various channels.)

I need to look at what I have currently and experiment some more.

[macdroid53]

After more experimentation, I always get event.size = 120.
When the break/data event is received, event.size is 120.
I read 120 and the first 32 bytes are the remainder of the 512 burst.

[negativekelvin]

For 1 packet you should get
DATA: 120
DATA: 120
DATA: 120
DATA: 120
DATA: 32
BREAK

You should not read anything at the break event

[macdroid53]

I added code to watch the even.size that is received when the event is triggered. It always returns event.size = 120.

So here's how I think the low level driver works (Note I am now using pre release code. The new code has added a hal level and the mod mentioned above is not being used.):

1. Break is detected.
2. uart begins to wait for 120 characters (120 is set by the driver code)
3. uart receives 120 characters, transfers them to the buffer, and triggers an event and event.size is set to 120. Low level FIFO proceeds to receive more bytes.
4. upstream code receives the event and reads 120 bytes, buffer is cleared.
5. go to step 2 three more times to receive 120 x 4 bytes.
6. uart receives 32 bytes, does nothing because 120 threshold has not been reached.
7. uart detects break
8. a break detect and data event is triggered (event.size = 120)
9. return to step 1

[negativekelvin]

Yes the mod is required because of 6

[macdroid53]

But, with the mod or without I get event.size = 120, never 32.

And the first 32 bytes of that 120 are the expected last 32 of the DMX burst.

[negativekelvin]

Then the mod is not working. Show your code.

[macdroid53]

As for the mod, the newer uart.c has changed so much I can't figure out where to put the mod now...

uart.c at line 850 now is:
else if ((uart_intr_status & UART_RXFIFO_TOUT_INT_ST_M) || (uart_intr_status & UART_RXFIFO_FULL_INT_ST_M) || (uart_intr_status & UART_AT_CMD_CHAR_DET_INT_ST_M) ) {

My code is: https://github.com/macdroid53/NoArduino

uart.c that I have is in src/uart.c.fyi

I think the pertinent code is src/dmx.c

The print statement at line 113 shows four 120 byte data events, then one break event with 120 bytes, then repeats.

[negativekelvin]

Yes it goes at 850. Break event doesn't have a size and shouldn't trigger a read bytes.

[macdroid53]

I'm confused. Of course the break event doesn't have a size...The original thought was that 32 bytes were being left in the buffer when the break triggered and never read because the threshold was not reached and the FIFO was not transferred to the buffer. And the mod was to have a read event triggered when a break was detected, so that any orphan bytes in the FIFO, in this case 32 bytes, would be transferred from the FIFO to the buffer so as to loose them because of clearing that happens when a break is detected.

[negativekelvin]

What delays the interrupt?

Other interrupts
Flash operations if UART_ISR_IN_IRAM is no

When parsing a frame do not read more than 512 bytes even if they are available

[koobest]

Hi, @luksal
According to DMX512 protoco

`1 pack = 1 break + 1 MAB(mark after break) + 1 SC (start code) + 512 slots + 1 MTBP (mark time between packets)

and the MTBP is within 0 ～ 1S,

so if MTBP > rx_idle, the hardware will generate the rx_idle interrupt, and the application can get DATA event.

The software can receive data packets according to the following process:

1. Wait for the break event and prepare to receive a new packet.

2. Wait for the data event and read the data frame according to event.size.

3. Wait for the break event. This break indicates that the previous data packet transmission is completed and the software can process the data packets (because the time interval between each data packet is MTBP> rx_idle, so at this time a complete data packet has been received and then started Ready to receive new packets.

thanks !!

[macdroid53]

Hi, @luksal
According to DMX512 protoco

`1 pack = 1 break + 1 MAB(mark after break) + 1 SC (start code) + 512 slots + 1 MTBP (mark time between packets)

and the MTBP is within 0 ～ 1S,

so if MTBP > rx_idle, the hardware will generate the rx_idle interrupt, and the application can get DATA event.

The software can receive data packets according to the following process:

1. Wait for the break event and prepare to receive a new packet.

2. Wait for the data event and read the data frame according to event.size.

3. Wait for the break event. This break indicates that the previous data packet transmission is completed and the software can process the data packets (because the time interval between each data packet is MTBP> rx_idle, so at this time a complete data packet has been received and then started Ready to receive new packets.

thanks !!

In practice the MTBP is 2 stop bits (~8uS). Thus never triggering rx_idle (unless I don't understand...always possible)
Your description is what my code does and it does as I've described above.
And, does not work with the un-modified uart.c , as discussed above.

Also, I find no reference to rx_idle in uart.c and no event definition for it.
Also, the start code is never returned as part of the first event.

[macdroid53]

I re-checked a few things.

My logic analyzer capture shows that the SC byte is in the packet.
The following code reads the uart buffer.
uart_read_bytes(DMX_UART_NUM, dtmp, event.size, portMAX_DELAY); printf("%d,%d:%d,%d,%d,%d\n",event.size,current_rx_addr,dtmp[0],dtmp[1], dtmp[2],dtmp[3]);
dtmp[0] after the break detect never shows the SC, but always shows the value of the first DMX slot. (Except in what ever condition causes the index shift mentioned above. In which case dtmp[0] is the value of DMX slot 2.)

So where is the SC going?
The data event after the break happens four times with event.size = 120 and once with event.size=33
That's 513 bytes. So it appears that the correct number of bytes IS being read.
Upon more tinkering around, it appears the SC byte is at index 512 (i.e. the 513th byte)
So why isn't it at index 0?
This I don't get...confused.

[negativekelvin]

Because you read too many bytes of the frame before. You should be keeping track independently of event.size.

[macdroid53]

Ok, so I finally think I get what you've been saying. My apologies for having old, slow synapses.

If we call the 513 bytes a packet and packets are received over time, we can number the packets Pn. Where "n" is the number of the latest packet.

Since there can be data received (because of any latency source) after the break is detected and will be in the buffer. So the data event triggered by the break detect (with code modification as discussed) will include the remainder of packet Pn and the start code (and possibly more) of packet Pn+1.

I've modified my test code up line of the modified uart.c and it does appear to be working.

So, what needs to happen to get the modified uart.c code integrated as a fix?

[negativekelvin]

Either emptying the fifo on break has to become default behavior or there has to be a new config option to enable it.

[acf-bwl]

I would also agree with merging this change into esp-idf. As it stands, the break is essentially delivered asynchronously to the serial data, which doesn't make sense in many applications where detecting a break is required. This patch just makes sure the break is delivered in the correct place in the serial stream. Personally I don't see much reason why it shouldn't be the default behaviour.

Perhaps someone could prepare a pull request, if those are accepted here? I can do it if required.

-Alan

[acf-bwl]

@xiongyumail Would Espressif be willing to merge such a patch if it were provided?

[Alvin1Zhang]

@acf-bwl Thanks for contribution, a Pull Request is welcome and appreciated. Thanks.

[mark-hahn]

I have been lurking and I have a question. I am about to code an app the requires receiving arbitrary length packets. There is a break sent immediately after the last byte in order to tell the receiver the packet has ended. Will the bug being described here make it impossible to implement this?

Another question: If the break is received asynchronously and then the last bytes cannot be read, wouldn't that mean that data can be lost if a break happens shortly after receiving? This seems like a serious breach of generic uart standards.

EDIT: Never mind. I'm changing the break to an idle in my protocol. I don't want to take any chances.

[acf-bwl]

Sorry for the delay. I've made a pull request #5959 with @negativekelvin 's above patch.

Thanks,
Alan

[alisitsyn]

There is the other way of handling DMX512 packet using UART. I propose to use MARK before BREAK - MBB time (2Sec > MBB > 0) to detect end of packet without additional changes in UART driver. In most of DMX512 controllers this time is not zero and it allows to use existing hardware UART TOUT feature to detect the end of DMX512 packet. The start of DMX condition is still BREAK.
The simplified receiver code would be like :

static void uart_event_task(void *pvParameters)
{
    uart_event_t event;
    size_t buffered_size;
    uint8_t* dtmp = (uint8_t*) malloc(RD_BUF_SIZE);
    for(;;) {
        //Waiting for UART event.
        if(xQueueReceive(uart0_queue, (void * )&event, (portTickType)portMAX_DELAY)) {
            bzero(dtmp, RD_BUF_SIZE);
            ESP_LOGI(TAG, "uart[%d] event:", EX_UART_NUM);
            switch(event.type) {
                //Event of UART receving data
                /*We'd better handler data event fast, there would be much more data events than
                other types of events. If we take too much time on data event, the queue might
                be full.*/
                case UART_DATA:
                    // The approach uses the UART TOUT feature which triggers an event and gets the received buffer 
                    // if no any transmission during configured period of time (1 symbol as example).
                    // This time corresponds to the MARK before BREAK - MBB time (2Sec > MBB > 0) of DMX protocol.
                    // The approach assumes that MBB not less than 1 symbol transmission time on current baud rate.
                    // Usually most of DMX512 controllers satisfy to this requirement.
                    if (event.timeout_flag && break_detected) {
                        ESP_LOGI(TAG, "[DATA TOUT EVENT]: Add to ring buffer = %d bytes.", event.size);
                        uart_get_buffered_data_len(EX_UART_NUM, &event.size);
                        uart_read_bytes(EX_UART_NUM, dtmp, event.size, portMAX_DELAY);
                        memmove(&dmx_channels[0], dtmp, event.size);
                        print_dmx_data(dtmp, event.size);
                        break_detected = false;
                        // Send data back to receiver to check correctness (not part of DMX512 proto)
                        uart_write_bytes(EX_UART_NUM, (const char*) dtmp, event.size);
                    } else {
                        ESP_LOGI(TAG, "[UART DATA]: Add to ring buffer = %d bytes.", event.size);
                    }
                    break;
                //Event of HW FIFO overflow detected
                case UART_FIFO_OVF:
                    ESP_LOGI(TAG, "hw fifo overflow");
                    // If fifo overflow happened, you should consider adding flow control for your application.
                    // The ISR has already reset the rx FIFO,
                    // As an example, we directly flush the rx buffer here in order to read more data.
                    uart_flush_input(EX_UART_NUM);
                    xQueueReset(uart0_queue);
                    break;
                //Event of UART ring buffer full
                case UART_BUFFER_FULL:
                    ESP_LOGI(TAG, "ring buffer full");
                    // If buffer full happened, you should consider encreasing your buffer size
                    // As an example, we directly flush the rx buffer here in order to read more data.
                    uart_flush_input(EX_UART_NUM);
                    xQueueReset(uart0_queue);
                    break;
                //Event of UART RX break detected
                case UART_BREAK:
                    if (!break_detected) {
                        break_detected = true;
                        //uart_flush_input(EX_UART_NUM); // allows to remove first zero byte in the received packet
                    }
                    ESP_LOGI(TAG, "uart rx break");
                    break;
                //Event of UART parity check error
                case UART_PARITY_ERR:
                    ESP_LOGI(TAG, "uart parity error");
                    break;
                //Event of UART frame error
                case UART_FRAME_ERR:
                    ESP_LOGI(TAG, "uart frame error");
                    break;
                //UART_PATTERN_DET
                case UART_PATTERN_DET:
                    break;
                //Others
                default:
                    ESP_LOGI(TAG, "uart event type: %d", event.type);
                    break;
            }
        }
    }
    free(dtmp);
    dtmp = NULL;
    vTaskDelete(NULL);
}

void app_main(void)
{
    esp_log_level_set(TAG, ESP_LOG_INFO);

    /* Configure parameters of an UART driver,
     * communication pins and install the driver */
    uart_config_t uart_config = {
        .baud_rate = 250000,
        .data_bits = UART_DATA_8_BITS,
        .parity = UART_PARITY_DISABLE,
        .stop_bits = UART_STOP_BITS_1,
        .flow_ctrl = UART_HW_FLOWCTRL_DISABLE,
        .source_clk = UART_SCLK_APB,
    };
    //Install UART driver, and get the queue.
    uart_driver_install(EX_UART_NUM, BUF_SIZE * 2, BUF_SIZE * 2, 20, &uart0_queue, 0);
    uart_param_config(EX_UART_NUM, &uart_config);

    //Set UART log level
    esp_log_level_set(TAG, ESP_LOG_INFO);
    //Set UART pins (using UART0 default pins ie no changes.)
    uart_set_pin(EX_UART_NUM, DMX_TEST_TXD, DMX_TEST_RXD, UART_PIN_NO_CHANGE, UART_PIN_NO_CHANGE);
    // We are using the UART TIMEOUT feature to detect and of DMX message
    uart_set_rx_timeout(EX_UART_NUM, DMX_NO_SYM_TIME); //  set MBB = 1 symbol time on current baudrate
    uart_set_always_rx_timeout(EX_UART_NUM, true); 
    break_detected = false;
    
    //Create a task to handler UART event from ISR
    xTaskCreate(uart_event_task, "uart_event_task", 2048, NULL, 12, NULL);
}

The log:

I tested my dmx512 receiver and transmitter over RS485 and this approach works just fine.
Please let me know if you have any comments.

[negativekelvin]

@alisitsyn that is definitely better when the timeout can be used to detect end of frame and in that case you would not want the break to empty the buffer so if there is still a need to have that it should probably be configurable at runtime

[alisitsyn]

@negativekelvin,
It can be configured and works just fine and may be enough but your approach to catch buffer on break event also can be implemented. In this case the break event may be propagated to user application through event.flags. So, the user app can recognize that the recieve of buffer was terminated by break event. What do you think?

[wmoors]

Hi all,

I ran into the same problem and after a bit of searching I stumbled onto this thread.

So with all this info I managed to fix the issue by evaluating the break condition first, then copy over the rx data from the fifo and pass the remainder length in the UART_BREAK event. ( at the lines specified by @negativekelvin) I'm not sure though if this could potentially impact the correct working of other serial protocols though, but for DMX512 it seems to work fine.

         ...
       }
        else if(uart_intr_status & UART_INTR_BRK_DET) {
            uart_hal_clr_intsts_mask(&(uart_context[uart_num].hal), UART_INTR_BRK_DET);
            rx_fifo_len = uart_hal_get_rxfifo_len(&(uart_context[uart_num].hal));
            uart_hal_read_rxfifo(&(uart_context[uart_num].hal), p_uart->rx_data_buf, &rx_fifo_len);
            uart_event.type = UART_BREAK;
            uart_event.size = rx_fifo_len;
        }
        else if ((uart_intr_status & UART_INTR_RXFIFO_TOUT)
                || (uart_intr_status & UART_INTR_RXFIFO_FULL)
                || (uart_intr_status & UART_INTR_CMD_CHAR_DET)
                ) {
          ...

This way, I can read the out when handling the UART_BREAK event.

        case UART_BREAK:
          if(udev->state == DMX_RX_DATA)
          {
            //read the remainder
            uart_read_bytes(udev->uart.nr,
                            tmp, // do something interesting with these bytes
                            event.size, 
                            portMAX_DELAY);
            ESP_LOGI(TAG, "DATA (%d bytes)\n", event.size);
          }
          else
          {
            uart_flush_input(udev->uart.nr);
          }
          xQueueReset(udev->rx_queue);
          udev->state = DMX_RX_BREAK;
          break;

...
I (2342) DMX_RX: DATA (120 bytes)
I (2342) DMX_RX: DATA (120 bytes)
I (2342) DMX_RX: DATA (120 bytes)
I (2352) DMX_RX: DATA (120 bytes)
I (2352) DMX_RX: DATA (33 bytes)
I (2362) DMX_RX: BREAK
...

The 33 bytes left, is because I am planning to work on an RDM controller so I need the startcode to distinguish so I don't discard the first byte in the buffer.

[wmoors]

after a quick further read of the uart driver, I guess the select notification callback should also be called if in fact there was data in the rx fifo... So you'd end up with:

...
       }
        else if(uart_intr_status & UART_INTR_BRK_DET) {
            uart_hal_clr_intsts_mask(&(uart_context[uart_num].hal), UART_INTR_BRK_DET);
            rx_fifo_len = uart_hal_get_rxfifo_len(&(uart_context[uart_num].hal));
            uart_hal_read_rxfifo(&(uart_context[uart_num].hal), p_uart->rx_data_buf, &rx_fifo_len);
            uart_event.type = UART_BREAK;
            uart_event.size = rx_fifo_len;
            UART_ENTER_CRITICAL_ISR(&uart_selectlock);
            if (rx_fifo_len && p_uart->uart_select_notif_callback) {
              p_uart->uart_select_notif_callback(uart_num, UART_SELECT_READ_NOTIF, &HPTaskAwoken);
            }
            UART_EXIT_CRITICAL_ISR(&uart_selectlock);
        }
        else if ((uart_intr_status & UART_INTR_RXFIFO_TOUT)
                || (uart_intr_status & UART_INTR_RXFIFO_FULL)
                || (uart_intr_status & UART_INTR_CMD_CHAR_DET)
                ) {
          ...

Anyway, I am curious as to what you guys think...

[zjykymt]

Hi,

I also found this situation in the ESP-IDF 4.1 version that triggered the event only after receiving 120 bytes or idle timeout, and then I used uart_set_rx_full_threshold and passed in the value of threshold 2 to solve this problem(If set to 1, there will be an error byte data of 0).

[AxelLin]

@acf-bwl Thanks for contribution, a Pull Request is welcome and appreciated. Thanks.

But there is no response to the pull request for 9 months? #5959

[SuGlider]

A possible solution for this issue is very simple: read byte by byte from UART instead of reading 120 bytes when its FIFO is full.

BREAK can happen any time.... just make sure that IDF driver always gets every single byte from UART and copy it to its internal RingBuffer. Doing so, when BREAK happens, the RingBuffer would be already complete and it will be possible to catch the BREAK event and then process the data already copied.

In order to read byte by byte, just initialize the UART IDF Driver and then execute uart_set_rx_full_threshold(uart_port_t uart_num, int threshold), with correct uart_num and threshold = 1.

[SuGlider]

After some testing, I found out that ERROR event takes precedence over DATA UART event.
So, for instance, when sending 120 bytes with a BREAK at the end, the receiver may behave differently depending on how UART is configured, regarding mainly to FIFO Full parameter.

Example:
Some device sends 100 bytes with a BREAK at the end.

The IDF UART driver will raise these events in the Event Queue:

1- IDF setting FIFO Full to 20 bytes:

• It will create 4 DATA events for the first 80 bytes, and in the last 20 bytes, it will first raise BREAK error event and after that raise DATA event for the remaining 20 bytes.

2- IDF setting FIFO Full to 120 bytes (IDF default setting):

• It will first raise BREAK error event and after that it will raise a DATA event for the "remaining" 100 bytes (it is lower than the 120 FIFO full event and a Timeout RX event will be raised instead).

Therefore, it will behave differently depending on how UART IDF RX IRQ is set.

[phatpaul]

This issue is directly relevant to LIN bus. In LIN, each frame starts with a break. So I need to clear my rx buffer whenever a break is detected and then it fills-up until a number of bytes is received. Then I will know that the first byte is the Sync, then PID, etc...

But the ESP-IDF sdk functions abstract this too much and I have to hack around these functions that are not ideal for this use case.

I was planning to implement my own ISR to do this, but then I see that that custom ISR will be unsupported in the future?!: 473974c#diff-e69821beb8fd51e70825b947e7b520b90aae6cfc98241204cb6f30bcc79c4730

How should I synchronize my rx buffer to the break? i.e. every break should reset the buffer.

FYI, here's my stupid code that works 90% of the time but sometimes gets out of sync:
(it did work with IDF 3.x but in IDF 4.x it is getting out-of-sync frequently)
(this function is called periodically from a freertos task, but perhaps it would work better inside an event handler? You can see that I tried to workaround an unreliable contents of the buffer.)

//// this is a terrible hack, please don't actually use this!

// returns length read
int LIN_driver_rx_header(uint8_t *pid, TickType_t ticksToWait)
{
    uart_event_t event;
    while (xQueueReceive(lin_uart_event_queue, (void *)&event, ticksToWait))
    {
        if (event.type == UART_BREAK)
        {
            //Event of UART RX break detected
            // run slave task once per break detected
            break;
        }
    }

    if (event.type == UART_BREAK)
    {
        // Is there a LIN frame header received?
        uint8_t slave_hdr_buf[4]; // we might recv some junk before the break (0) and sync (55)
        int rx_got_len = 0;
        int tries = 0; // prevent lockup
        while (tries++ < 4 && rx_got_len < sizeof(slave_hdr_buf))
        {
            rx_got_len += uart_read_bytes(lin_uart_num, (uint8_t *)&slave_hdr_buf[rx_got_len], 1, ticksToWait);
            if (rx_got_len >= 2) // need at least 2 bytes
            {
                ESP_LOG_BUFFER_HEX_LEVEL("LIN HDR", slave_hdr_buf, rx_got_len, ESP_LOG_VERBOSE);
                if (rx_got_len == 2 && slave_hdr_buf[0] == 0x55) //check break (0) and sync (55) are in the correct spot
                {
                    *pid = slave_hdr_buf[1]; // PID is last byte of header
                    return 1;                // non-zero means we rec a header with PID
                }
                if (rx_got_len == 3 && slave_hdr_buf[1] == 0x55) //check break (0) and sync (55) are in the correct spot
                {
                    *pid = slave_hdr_buf[2]; // PID is last byte of header
                    return 1;                // non-zero means we rec a header with PID
                }
                if (rx_got_len == 4 && slave_hdr_buf[2] == 0x55) //check break (0) and sync (55) are in the correct spot
                {
                    *pid = slave_hdr_buf[3]; // PID is last byte of header
                    return 1;                // non-zero means we rec a header with PID
                }
            }
        }
    }

    return 0; // 0 means we didn't get a valid header (i.e. timeout)
}

the event queue was setup like this:
uart_driver_install(lin_uart_num, UART_FIFO_LEN * 2, 0, 10, &lin_uart_event_queue, 0);