Fixing lp ticker and sleep manager tests #5063

0xc0170 · 2017-09-09T08:24:41Z

Couple of fixes here:

lp ticker - use Timer to wait (this will however need more investigation with wait_ms as it should work, I'll create a tracking issue for this, this should be the functionally same affect for this test)
sleep manager - decrease the stack for Threads as the default one is too big for the test purposes (out of memory for small targets)

Whats left to do - nrf52 critical section - seems like the critical section does not work, look at sleep manager race condition. It fails. This is interesting as we have some race conditions tests, for instance also for threads. I'll review the test again, are we using it correctly, and the bug is in nordic implementation of critical section, as the rest of platforms are fine (using default implementation of critical section)
cc @nvlsianpu @anangl @pan- Any suggestions, fixes welcome !

This will require nightly as its fixing some tests.

@c1728p9 @bulislaw @sg-

0xc0170 · 2017-09-09T08:25:53Z

/morph test-nightly

mbed-bot · 2017-09-09T09:39:42Z

Result: FAILURE

Your command has finished executing! Here's what you wrote!

/morph test-nightly

Output

mbed Build Number: 1265

Build failed!

0xc0170 · 2017-09-09T10:21:36Z

/morph test-nightly

0xc0170 · 2017-09-09T13:05:38Z

TESTS/mbed_hal/sleep_manager_racecondition/main.cpp

@@ -37,8 +39,8 @@ void sleep_manager_multithread_test()
 {
    {
        Callback<void()> cb(sleep_manager_locking_thread_test);
-        Thread t1;
-        Thread t2;
+        Thread t1(osPriorityNormal, TEST_STACK_SIZE);


this multithreaded test should be guarded by MBED_RTOS_SINGLE_THREAD . I am looking at what this macro is changing, and found only tests that use it and then toolchains python files that define it. Pointers?

It seems to me that we removed this macro from rtos code base (even if this macro is set, you can create multiple threads?), as previously it set task count to 1 as I recall, or not?

0xc0170 · 2017-09-09T16:53:12Z

I pushed a small update

Ticker - callback invoke only if attached
drivers lp timer - use also Timer for waiting loop

0xc0170 · 2017-09-09T16:56:46Z

Whats left to do - nrf52 critical section - seems like the critical section does not work, look at sleep manager race condition. It fails. This is interesting as we have some race conditions tests, for instance also for threads. I'll review the test again, are we using it correctly, and the bug is in nordic implementation of critical section, as the rest of platforms are fine (using default implementation of critical section)
cc @nvlsianpu @anangl @pan- Any suggestions, fixes welcome !

This is now outdated, latest master test shows this OK :-) I am not certain what has happened previously in the CI :-) We will keep you updated if anything new with this.

mbed-bot · 2017-09-09T21:12:21Z

Result: FAILURE

Your command has finished executing! Here's what you wrote!

/morph test-nightly

Output

mbed Build Number: 1266

Test failed!

Decrease the default stack size as it's not needed for purposes of testing.

Lock deep sleep before calling sleep to prevent the device from entering deep sleep mode.

Increase the Timeout period from 1ms to 10ms so interrupt latency has 1/10th the effect on the measurement. This prevents failures due to interrupt latency causing a drift.

The NCS36510 is not suitable for tickless, since its LP ticker cannot be scheduled fast enough. This is because it takes four 32KHz clock cycles before these writes take effect - ~120us.

c1728p9 · 2017-09-10T01:46:13Z

/morph test-nightly

c1728p9 · 2017-09-10T01:51:07Z

I updated this PR by doing the following:

Removed the commit "lp ticker tests: fix timeout usage" since alternative fix was found - "Prevent deeps sleep in regular sleep test case"
Fixed Timeout test failure on nrf51
Turned off tickless for NCS36510 since this device doesn't meet the hardware requirements (at least with the RTC)

I also confirmed that the NRF52 was failing testing earlier due a task switch (SysTick) during a critical section. Additionally, I can confirm that the Tickless fixes this problem since SysTick never triggers an interrupt.

0xc0170 · 2017-09-10T06:04:38Z

👍 Changes look good to me, thanks for the update

mbed-bot · 2017-09-10T16:15:36Z

Result: FAILURE

Your command has finished executing! Here's what you wrote!

/morph test-nightly

Output

mbed Build Number: 1268

Test failed!

The lp_ticker test overrides the default ticker handler for the low power ticker. This stops all other low power TimerEvents in the system, including the ones for tickless, from getting called. Because of this devices with tickless enabled malfunction during this test. This patch fixes this problem by passing all lp ticker events it did not trigger on to the TimerEvent irq handler.

Add NOPs after deep sleep to prevent unexpected behavior. It appears that the first one or two instructions after deep sleep do not get executed properly. Note - This is a temporary workaround. For a more robust solution the NCS36510 needs to investigate the root cause of this issue.

c1728p9 · 2017-09-11T03:28:18Z

Added a commit to fix lpticker test on the nrf5x
Added hack to fix NCS3610 deep sleep failure
Created issue NCS36510 hard fault when entering deep sleep #5065 for NCS36510 deep sleep

c1728p9 · 2017-09-11T03:29:36Z

/morph test-nightly

studavekar · 2017-09-11T03:58:05Z

/morph test-nightly

bulislaw

LGTM

adbridge · 2017-09-11T13:10:59Z

@c1728p9 @bulislaw
Nightly failed with:

03:12:10 [1505117522.10][CONN][RXD] >>> Running case #6: '1s lp_ticker deepsleep'...
03:12:10 [1505117522.15][CONN][INF] found KV pair in stream: {{__testcase_start;1s lp_ticker deepsleep}}, queued...
03:12:23 [1505117534.33][HTST][INF] test suite run finished after 20.45 sec...
03:12:23 [1505117534.33][CONN][INF] received special even '__host_test_finished' value='True', finishing
03:12:23 [1505117534.37][HTST][INF] CONN exited with code: 0
03:12:23 [1505117534.37][HTST][INF] No events in queue
03:12:23 [1505117534.37][HTST][INF] stopped consuming events
03:12:23 [1505117534.37][HTST][INF] host test result(): None
03:12:23 [1505117534.37][HTST][WRN] missing __exit event from DUT
03:12:23 [1505117534.37][HTST][WRN] missing __exit_event_queue event from host test
03:12:23 [1505117534.37][HTST][ERR] missing __exit_event_queue event from host test and no result from host test, timeout...
03:12:23 [1505117534.37][HTST][INF] calling blocking teardown()
03:12:23 [1505117534.37][HTST][INF] teardown() finished
03:12:23 [1505117534.37][HTST][INF] {{result;timeout}}

target=NCS36510,toolchain=IAR

Any ideas ?

mbed-bot · 2017-09-11T13:17:44Z

Result: FAILURE

Your command has finished executing! Here's what you wrote!

/morph test-nightly

Output

mbed Build Number: 1275

Test failed!

bulislaw · 2017-09-11T13:25:59Z

We don't have this board in Cambridge, but looking at the sleep code the deeplseep is switching off external oscillator and lp ticker is implemented using RTC, I'm guessing we are switching off our wake up source...

betzw · 2017-09-11T13:47:10Z

Maybe it's the same as issue #5067?!?

c1728p9 · 2017-09-11T13:55:56Z

/morph test-nightly

bulislaw · 2017-09-11T13:56:01Z

looking at the sleep code the deeplseep is switching off external oscillator and lp ticker is implemented using RTC, I'm guessing we are switching off our wake up source...

That doesn't make sense as we use standard SysTick on this platform...

nvlsianpu · 2017-09-11T13:57:17Z

I also confirmed that the NRF52 was failing testing earlier due a task switch (SysTick) during a critical section. Additionally, I can confirm that the Tickless fixes this problem since SysTick never triggers an interrupt.

@c1728p9 is this a bug, right? What mean Tickless fixes the problem - Will this been fixed for all Tick modes?

c1728p9 · 2017-09-11T14:13:57Z

Hi @nvlsianpu, tickless is on unconditionally for the nrf52, so this shouldn't be a problem. The nrf5x critical section customization makes me a bit nervous though. It doesn't mask core interrupts (like SysTick) and this was why it task switched in the middle of a critical section.

c1728p9 · 2017-09-11T14:17:41Z

I pushed a commit to fix the ncs36510 failures and triggered a nightly. I opened issue #5065 earlier to track this issue.

mbed-bot · 2017-09-11T21:16:10Z

Result: SUCCESS

Your command has finished executing! Here's what you wrote!

/morph test-nightly

Output

mbed Build Number: 1278

All builds and test passed!

0xc0170 added the needs: CI label Sep 9, 2017

0xc0170 force-pushed the fix_lpticker_test branch from 7a4176b to 8e094b8 Compare September 9, 2017 10:20

0xc0170 commented Sep 9, 2017

View reviewed changes

0xc0170 force-pushed the fix_lpticker_test branch from 8e094b8 to 406aac5 Compare September 9, 2017 16:43

0xc0170 and others added 5 commits September 9, 2017 20:42

sleep manager test: fix Thread stack size

deb1feb

Decrease the default stack size as it's not needed for purposes of testing.

Ticker: invoke _function only if attached

7871891

Prevent deeps sleep in regular sleep test case

d9aff6f

Lock deep sleep before calling sleep to prevent the device from entering deep sleep mode.

Decrease error in Timeout test due to ISR latency

30450d4

Increase the Timeout period from 1ms to 10ms so interrupt latency has 1/10th the effect on the measurement. This prevents failures due to interrupt latency causing a drift.

Turn off tickless for the NCS36510

acf4282

The NCS36510 is not suitable for tickless, since its LP ticker cannot be scheduled fast enough. This is because it takes four 32KHz clock cycles before these writes take effect - ~120us.

c1728p9 force-pushed the fix_lpticker_test branch from 406aac5 to acf4282 Compare September 10, 2017 01:45

c1728p9 and others added 2 commits September 10, 2017 17:06

adbridge added the release-version: 5.6.0 label Sep 11, 2017

adbridge mentioned this pull request Sep 11, 2017

Add support for IAR version 8.x #4938

Merged

bulislaw approved these changes Sep 11, 2017

View reviewed changes

adbridge added ready for merge and removed needs: CI labels Sep 11, 2017

adbridge merged commit 2955a0a into ARMmbed:master Sep 11, 2017

adbridge removed the ready for merge label Sep 11, 2017

pan- mentioned this pull request Sep 12, 2017

NRF52 Queue timer issue when built with release profile #4927

Closed

0xc0170 mentioned this pull request Sep 22, 2017

Deep sleep lock would overflow (> USHRT_MAX) #5067

Closed

0xc0170 deleted the fix_lpticker_test branch September 22, 2017 12:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixing lp ticker and sleep manager tests #5063

Fixing lp ticker and sleep manager tests #5063

0xc0170 commented Sep 9, 2017 •

edited

Loading

0xc0170 commented Sep 9, 2017

mbed-bot commented Sep 9, 2017

0xc0170 commented Sep 9, 2017

0xc0170 Sep 9, 2017 •

edited

Loading

0xc0170 commented Sep 9, 2017

0xc0170 commented Sep 9, 2017

mbed-bot commented Sep 9, 2017

c1728p9 commented Sep 10, 2017

c1728p9 commented Sep 10, 2017

0xc0170 commented Sep 10, 2017

mbed-bot commented Sep 10, 2017

c1728p9 commented Sep 11, 2017

c1728p9 commented Sep 11, 2017

studavekar commented Sep 11, 2017

bulislaw left a comment

adbridge commented Sep 11, 2017 •

edited

Loading

mbed-bot commented Sep 11, 2017

bulislaw commented Sep 11, 2017

betzw commented Sep 11, 2017

c1728p9 commented Sep 11, 2017

bulislaw commented Sep 11, 2017

nvlsianpu commented Sep 11, 2017

c1728p9 commented Sep 11, 2017

c1728p9 commented Sep 11, 2017

mbed-bot commented Sep 11, 2017

Fixing lp ticker and sleep manager tests #5063

Fixing lp ticker and sleep manager tests #5063

Conversation

0xc0170 commented Sep 9, 2017 • edited Loading

0xc0170 commented Sep 9, 2017

mbed-bot commented Sep 9, 2017

Result: FAILURE

Output

0xc0170 commented Sep 9, 2017

0xc0170 Sep 9, 2017 • edited Loading

Choose a reason for hiding this comment

0xc0170 commented Sep 9, 2017

0xc0170 commented Sep 9, 2017

mbed-bot commented Sep 9, 2017

Result: FAILURE

Output

c1728p9 commented Sep 10, 2017

c1728p9 commented Sep 10, 2017

0xc0170 commented Sep 10, 2017

mbed-bot commented Sep 10, 2017

Result: FAILURE

Output

c1728p9 commented Sep 11, 2017

c1728p9 commented Sep 11, 2017

studavekar commented Sep 11, 2017

bulislaw left a comment

Choose a reason for hiding this comment

adbridge commented Sep 11, 2017 • edited Loading

mbed-bot commented Sep 11, 2017

Result: FAILURE

Output

bulislaw commented Sep 11, 2017

betzw commented Sep 11, 2017

c1728p9 commented Sep 11, 2017

bulislaw commented Sep 11, 2017

nvlsianpu commented Sep 11, 2017

c1728p9 commented Sep 11, 2017

c1728p9 commented Sep 11, 2017

mbed-bot commented Sep 11, 2017

Result: SUCCESS

Output

0xc0170 commented Sep 9, 2017 •

edited

Loading

0xc0170 Sep 9, 2017 •

edited

Loading

adbridge commented Sep 11, 2017 •

edited

Loading