Task Watchdog & AccelStepper

[joeltbennett]

Hi - I have what I believe to be a unique scenario. I'm sure someone smarter will say I'm an idiot, but I will describe it to you anyway:

I have an ESP32 WROOM with core1 running functions for a Nextion screen (controlled with serial communications back and forth) and stepper motor functions on core0.

I ran into the classic TWD idle reboots described in this thread here. I tried the vTaskDelay(10) method and it worked great, right up until we got to a loop containing my stepper motor run functions (stepper1.run() / stepper2.run()). With no vTaskDelay in that loop, I get a reboot. With a vTaskDelay of any size (right down to 1), the program functions but the motors run pitifully slow.

Accel stepper requires these run functions to run in the loop unmolested, when you start adding things in it skips steps or just slows everything down.

I then decided to try to disable TWD entirely for core0 (functions outlined here) - this worked great, no delays necessary. But when I got to that loop with my .run functions in it, I get a reboot with no TWD message. Actually, there's just no message at all when TWD is disabled - just reboots. Can't backtrace it, but I'm guessing it's the AccelStepper .run() function causing a panic.

So now I kind of have a "damned if you do, damned if you don't" type scenario.

Here's the backtrace when I don't have a vTaskDelay in the stepper.run loop:

Decoding stack results
0x40088dd8: invoke_abort at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/esp32/panic.c line 155
0x40089005: abort at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/esp32/panic.c line 170
0x400dd5df: task_wdt_isr at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/esp32/task_wdt.c line 174
0x40081137: delayMicroseconds at /Users/JTB/Library/Arduino15/packages/esp32/hardware/esp32/1.0.2/cores/esp32/esp32-hal-misc.c line 157
0x400d4476: AccelStepper::step1(long) at /Users/JTB/Documents/Arduino/libraries/AccelStepper/AccelStepper.cpp line 391
0x400d4341: AccelStepper::step(long) at /Users/JTB/Documents/Arduino/libraries/AccelStepper/AccelStepper.cpp line 348
0x400d45c2: AccelStepper::runSpeed() at /Users/JTB/Documents/Arduino/libraries/AccelStepper/AccelStepper.cpp line 60
0x400d47c9: AccelStepper::run() at /Users/JTB/Documents/Arduino/libraries/AccelStepper/AccelStepper.cpp line 185
0x400d3110: MotorsTc(void*) at /Users/JTB/Downloads/AC-1 :: Work/Sterling-AC-1/Sterling-AC-1.ino line 1801
0x40087925: vPortTaskWrapper at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/freertos/port.c line 143

This is the .cpp file referenced in the backtrace.

I can't see it saying much else besides AccelStepper's .run function (and the others it subsequently calls on) aren't going to work with watchdog...would it help to modify that function in AccelStepper to feed the watchdog without any delay?

---

edit to put in some sketch info:

this is the loop that's causing the panic:

void MotorsTc( void * pvParameters ){
Serial.print("Motors running on core ");
Serial.println(xPortGetCoreID());
for(;;){
while (windStatus == true) {
stepper1.enableOutputs();
stepper2.enableOutputs();
s2_zero();
s2_bobComp();
for (int i = 0; i < 20; i ++) {
stepper1.setCurrentPosition(0);
stepper1.setMaxSpeed(MaxSpeed1);
stepper1.setAcceleration(Accel1);
stepper2.setMaxSpeed(MAX_SPD);
stepper2.setAcceleration(MAX_XCL);
stepper2.moveTo(Move2_A[i]);
stepper2.runToPosition();
stepper2.setMaxSpeed(MaxSpeed2[i]);
stepper2.moveTo(Move2_B[i]);
if (dirCW != true) {Move1[i] = -Move1[i];}
stepper1.moveTo(Move1[i]);
if (dirCW != true) {
VVV THIS ONE HERE INSIDE THE WHILE STATEMENT(S) VVV
while (stepper1.distanceToGo() < 0) {
stepper1.run();
if (stepper2.currentPosition() == Move2_A[i]) {
stepper2.moveTo(Move2_B[i]);}
if (stepper2.currentPosition() == Move2_B[i]) {
stepper2.moveTo(Move2_A[i]);}
stepper2.run();
}
} else {
while (stepper1.distanceToGo() > 0) {
stepper1.run();
if (stepper2.currentPosition() == Move2_A[i]) {
stepper2.moveTo(Move2_B[i]);}
if (stepper2.currentPosition() == Move2_B[i]) {
stepper2.moveTo(Move2_A[i]);}
stepper2.run();
};
}
};
stepper2.moveTo(0);
stepper2.runToPosition();
windStatus = false;
stepper1.disableOutputs();
stepper2.disableOutputs();
vTaskDelay(10);
};
vTaskDelay(10);
}
}

[lbernstone]

Looks like step1 is using delayMicroseconds instead of vTaskDelay. delayMicroseconds does not yield, so it is blocking, and the WDT catches it.

[joeltbennett]

Looks like step1 is using delayMicroseconds instead of vTaskDelay. delayMicroseconds does not yield, so it is blocking, and the WDT catches it.

Ohh that would explain a lot. I'm not sure why it reboots when TWD is disconnected too - it runs perfectly fine in a single core configuration.

I'm really new to all this, so I'm just wondering if there's a way to address it - maybe tell TWD to ignore that? I guess if I did that, there would still be the need to feed TWD from within the run loop, which is still going to cause a break in the stepper motor function that slows everything down...

What do you think might be the solution here? I'd like to disable TWD completely but I have no idea how to get it to stop rebooting on that loop when referenced by a task assignment.

[lbernstone]

The WDT is there to show you that you have a problem. Taking off the bandaid doesn't heal the wound. As I said, replace delayMicroseconds with vTaskDelay.

[joeltbennett]

The WDT is there to show you that you have a problem. Taking off the bandaid doesn't heal the wound. As I said, replace delayMicroseconds with vTaskDelay.

OK, I'm gonna give that a try tomorrow. I imagine this question will come up again in the future too, so I'll report back if it works.

[lbernstone]

It doesn't seem like a very robust piece of code on arduino's part either. I'll look at the delayMicroseconds function and see if there is some legit reason to block.

[joeltbennett]

It doesn't seem like a very robust piece of code on arduino's part either. I'll look at the delayMicroseconds function and see if there is some legit reason to block.

I'm totally unfamiliar with ESP32, this is my first time using it, so don't laugh, but I tried replacing this (line 391 in AccelStepper.cpp):

delayMicroseconds(_minPulseWidth);

with this:

vTaskDelay(_minPulseWidth/portTICK_PERIOD_MS);

The motors ran, but it was still delaying the steps too long - it may have been a millisecond rather than 20 microseconds. The minimum pulse width in this case (and the standard for most stepper motor applications) is 20.

I didn't study computer science and I was a poor math student, so I'll really need some help here getting milliseconds into microseconds if that's ok with you. This thread here was useful in explaining some of the ESP timing stuff, but the only solution posted was the one above.

I'm going to do some more research, but if you have any better ideas, I'd be happy to try them out.

[atanisoft]

The motors ran, but it was still delaying the steps too long - it may have been a millisecond rather than 20 microseconds. The minimum pulse width in this case (and the standard for most stepper motor applications) is 20.

This would be _minPulseWidth milliseconds and not microseconds. You would need to divide by 1000 to get to microseconds...

Also the reason this is using NOP ASM instructions today is there is no sub-millisecond delay function that would prevent task switching during this short delay.

[lbernstone]

Yup^^. Just tested. As @atansoft says, vTaskDelay is approx in milliseconds.
A delay of 20 microseconds should not be triggering a watchdog even if blocking. This would imply that your code is looping through this block many times without giving up focus. Motors don't respond in microseconds, so it seems like your time frames are a bit out of whack here. Rather than bitbanging each pulse, perhaps you can use one of the dedicated peripherals that provide PWM.

[joeltbennett]

Also the reason this is using NOP ASM instructions today is there is no sub-millisecond delay function that would prevent task switching during this short delay.

Am I just out of luck then when it comes to using AccelStepper in a task assignment then? vTaskDelay(_minPulseWidth/1000); gave me the same crash (though it did run the correct speed) - I'm way out of my depth with this now, but I can't understand why it functions just fine on a single core configuration but still crashes when assigned with watchdog disabled.

[atanisoft]

Am I just out of luck then when it comes to using AccelStepper in a task assignment then?

With the ESP32 yes, there are better options available likely. The MCPWM hardware driver would be better suited for the task of driving a stepper motor. An example app which with a few adjustments for usage in Arduino IDE should work.

[joeltbennett]

Rather than bitbanging each pulse, perhaps you can use one of the dedicated peripherals that provide PWM.

I don't believe this stuff is within my capabilities, but I'm googling some things anyway.

[dpharris]

Sorry if this post is late to the party. Surely we can get the AccelStepper library to function, its a well established library. This seems to be the pertinent routine:

// 1 pin step function (ie for stepper drivers) // This is passed the current step number (0 to 7) // Subclasses can override void AccelStepper::step1(long step) { (void)(step); // Unused // _pin[0] is step, _pin[1] is direction setOutputPins(_direction ? 0b10 : 0b00); // Set direction first else get rogue pulses setOutputPins(_direction ? 0b11 : 0b01); // step HIGH // Caution 200ns setup time // Delay the minimum allowed pulse width delayMicroseconds(_minPulseWidth); setOutputPins(_direction ? 0b10 : 0b00); // step LOW }

The delayMicroseconds(_minPulseWidth) needs to be substituted, or delayMicroseconds() fixed. This thread seems useful: https://www.esp32.com/viewtopic.php?t=880 Especially this snippet:

#include "freertos/FreeRTOS.h" #define NOP() asm volatile ("nop") unsigned long IRAM_ATTR micros() { return (unsigned long) (esp_timer_get_time()); } void IRAM_ATTR delayMicroseconds(uint32_t us) { uint32_t m = micros(); if(us){ uint32_t e = (m + us); if(m > e){ //overflow while(micros() > e){ NOP(); } } while(micros() < e){ NOP(); } } }

David

…

On Fri, Jun 14, 2019 at 10:31 AM joeltbennett ***@***.***> wrote: Rather than bitbanging each pulse, perhaps you can use one of the dedicated peripherals that provide PWM. I don't believe this stuff is within my capabilities, but I'm googling some things anyway. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#2892?email_source=notifications&email_token=AAEDQSQC6C633IBBHFGK4G3P2PIWZA5CNFSM4HYCQ3XKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXXO33I#issuecomment-502197741>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAEDQSXNWVC7WJWST2OAMHDP2PIWZANCNFSM4HYCQ3XA> .

[joeltbennett]

The delayMicroseconds(_minPulseWidth) needs to be substituted, or delayMicroseconds() fixed. This thread seems useful: https://www.esp32.com/viewtopic.php?t=880 Especially this snippet:

Hi - I tried vTaskDelay/1000 to get down to microseconds and feed TWD, but only the first part worked - still had a panic reboot.

I have to stress that I am just an idiot who can follow instructions sometimes, so forgive me if this question is really stupid: are you suggesting that by adding that snippet to my sketch, delayMicroseconds will be able to work with TWD enabled?

This one:

#include "freertos/FreeRTOS.h"

#define NOP() asm volatile ("nop")

unsigned long IRAM_ATTR micros()
{
return (unsigned long) (esp_timer_get_time());
}
void IRAM_ATTR delayMicroseconds(uint32_t us)
{
uint32_t m = micros();
if(us){
uint32_t e = (m + us);
if(m > e){ //overflow
while(micros() > e){
NOP();
}
}
while(micros() < e){
NOP();
}
}
}

e: this guy who did the Grbl port to esp32 seems to know the way around (the "interrupts" paragraph).

[atanisoft]

The delayMicroseconds(_minPulseWidth) needs to be substituted, or delayMicroseconds() fixed. This thread seems useful: https://www.esp32.com/viewtopic.php?t=880 Especially this snippet:
#include "freertos/FreeRTOS.h" #define NOP() asm volatile ("nop") unsigned long IRAM_ATTR micros() { return (unsigned long) (esp_timer_get_time()); } void IRAM_ATTR delayMicroseconds(uint32_t us) { uint32_t m = micros(); if(us){ uint32_t e = (m + us); if(m > e){ //overflow while(micros() > e){ NOP(); } } while(micros() < e){ NOP(); } } }

This is actually what the current delayMicrosecond() function uses.

are you suggesting that by adding that snippet to my sketch, delayMicroseconds will be able to work with TWD enabled?

It wouldn't be any different than the current delayMicroseconds() code that is being used now. The problem is the code that is calling this does not feed the watchdog within the required time.

[dpharris]

I, too, am a newby at ESP32. That snippet would appear to implement delayMicrosecond() in an ESP32 friendly way. You can turn the watch-dog on and off on each core separately. David

…

On Fri, Jun 14, 2019 at 11:11 AM joeltbennett ***@***.***> wrote: The delayMicroseconds(_minPulseWidth) needs to be substituted, or delayMicroseconds() fixed. This thread seems useful: https://www.esp32.com/viewtopic.php?t=880 Especially this snippet: Hi - I tried vTaskDelay/1000 to get down to microseconds and feed TWD, but only the first part worked - still had a panic reboot. I have to stress that I am just an idiot who can follow instructions sometimes, so forgive me if this question is really stupid: are you suggesting that by adding that snippet to my sketch, delayMicroseconds will be able to work with TWD enabled? This one: #include "freertos/FreeRTOS.h" #define NOP() asm volatile ("nop") unsigned long IRAM_ATTR micros() { return (unsigned long) (esp_timer_get_time()); } void IRAM_ATTR delayMicroseconds(uint32_t us) { uint32_t m = micros(); if(us){ uint32_t e = (m + us); if(m > e){ //overflow while(micros() > e){ NOP(); } } while(micros() < e){ NOP(); } } } — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2892?email_source=notifications&email_token=AAEDQSUM4DSHTM6QIQUAP4LP2PNNNA5CNFSM4HYCQ3XKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXXSA2A#issuecomment-502210664>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAEDQSW2MGL5SZDSRWU3MCTP2PNNNANCNFSM4HYCQ3XA> .