Reliably running host_tests on a u-blox C027 board #992

RobMeades · 2015-03-23T11:50:12Z

Przemek Wirkus's push request from here:

...makes it possible to run host_tests on a u-blox C027 board but, even with this change, they are not completely reliable. For instance, running test MBED_A1 five times gives me:

TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [TIMEOUT] in 20.48 of 10 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [TIMEOUT] in 20.49 of 10 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.36 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.34 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [TIMEOUT] in 20.47 of 10 sec

In general I get a timeout about 1/3 of the time. Running with -v and looking at a fail case, it says:

Executing 'python host_test.py -d D: -f "C:\projects\mbed\build\test\LPC1768\GCC_ARM\MBED_A1\basic.bin" -p COM31 -t 10 -C 1.5 -m LPC1768'
Test::Output::Start
MBED: Instrumentation: "COM31" and disk: "D:"
HOST: Copy image onto target...
HOST: Initialize serial port...
....port ready!
HOST: Reset target...
HOST: Unknown property:
HOST: Unknown property:
HOST: Unknown property:
HTest::Output::Finish

The first Unknown property: is sometimes followed by { or {{timeout;2. I've tried adding a 5 second reset_tout delay but that doesn't seem to help. Is there a more detailed level of debug I could look at to find out what's going wrong? Watching the C027 board as things run, it's difficult to correlate when the problem occurs with the behaviour of the LEDs, but it is something to do with the end of the copy and the reset I guess. What else could I do to make this reliable?

The text was updated successfully, but these errors were encountered:

RobMeades · 2015-03-23T15:02:50Z

The references to the Vector Table issue and pull request are a mistake, please ignore them.

PrzemekWirkus · 2015-03-24T11:12:14Z

Hi,
I will have a look at that when I'm back in the office next Monday :)
Unfortunately I'm traveling now and I do not have access to LPC1768 hardware.

PrzemekWirkus · 2015-03-24T11:18:03Z

Just one comment from me, maybe two:

You are testing LPC1768,
Please make sure that LPC1768's disk is not filled with .bin files because if it is you may have flashing problems from time to time due to (semihosted) disk full.

RobMeades · 2015-03-24T11:53:15Z

Hi, and thanks for responding. Yes, I am testing LPC1768, though only those tests which happen to work on the C027 platform. The .bin files don't appear to be retained; at least, when I ask Windows about the mapped drive it says that 8 kbytes are used and 512 kbytes are free. Since you're travelling I'll go do something else for the remainder of the week.

PrzemekWirkus · 2015-03-30T08:59:35Z

Rob,
Would you be so kind and share your ARM GCC version:

$> arm-none-eabi-gcc-4.8.3.exe --version

arm-none-eabi-gcc-4.8.3.exe (GNU Tools for ARM Embedded Processors) 4.8.3 20131129 (release) [ARM/embedded-4_8-branch revision 205641]
Copyright (C) 2013 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

PrzemekWirkus · 2015-03-30T10:54:17Z

Rob,
I want to check this behavior with the same GCC compiler so please be so kind and let me know your GCC version so I can reproduce it :)

Ran few tests to check your defect and got this.

$  singletest.py --auto -n MBED_A1 --global-loops 20

MBEDLS: Detecting connected mbed-enabled devices...
MBEDLS: Detected LPC1768, port: COM77, mounted: E:
Building library CMSIS (LPC1768, ARM)
Building library MBED (LPC1768, ARM)
Building project BASIC (LPC1768, ARM)
TargetTest::LPC1768::ARM::MBED_A1::Basic [OK] in 2.22 of 20 sec
TargetTest::LPC1768::ARM::MBED_A1::Basic [OK] in 2.20 of 20 sec
TargetTest::LPC1768::ARM::MBED_A1::Basic [OK] in 2.15 of 20 sec
TargetTest::LPC1768::ARM::MBED_A1::Basic [OK] in 2.20 of 20 sec
TargetTest::LPC1768::ARM::MBED_A1::Basic [OK] in 2.18 of 20 sec
TargetTest::LPC1768::ARM::MBED_A1::Basic [OK] in 2.24 of 20 sec
TargetTest::LPC1768::ARM::MBED_A1::Basic [OK] in 2.17 of 20 sec
TargetTest::LPC1768::ARM::MBED_A1::Basic [OK] in 2.16 of 20 sec
TargetTest::LPC1768::ARM::MBED_A1::Basic [OK] in 2.23 of 20 sec
TargetTest::LPC1768::ARM::MBED_A1::Basic [OK] in 2.26 of 20 sec
TargetTest::LPC1768::ARM::MBED_A1::Basic [OK] in 2.17 of 20 sec
TargetTest::LPC1768::ARM::MBED_A1::Basic [OK] in 2.16 of 20 sec
TargetTest::LPC1768::ARM::MBED_A1::Basic [OK] in 2.15 of 20 sec
TargetTest::LPC1768::ARM::MBED_A1::Basic [OK] in 2.25 of 20 sec
TargetTest::LPC1768::ARM::MBED_A1::Basic [OK] in 2.16 of 20 sec
TargetTest::LPC1768::ARM::MBED_A1::Basic [OK] in 2.17 of 20 sec
TargetTest::LPC1768::ARM::MBED_A1::Basic [OK] in 2.18 of 20 sec
TargetTest::LPC1768::ARM::MBED_A1::Basic [OK] in 2.18 of 20 sec
TargetTest::LPC1768::ARM::MBED_A1::Basic [OK] in 2.22 of 20 sec
TargetTest::LPC1768::ARM::MBED_A1::Basic [OK] in 2.16 of 20 sec
Test summary:
+--------+---------+-----------+---------+------------------+--------------------+---------------+-------+
| Result | Target  | Toolchain | Test ID | Test Description | Elapsed Time (sec) | Timeout (sec) | Loops |
+--------+---------+-----------+---------+------------------+--------------------+---------------+-------+
| OK     | LPC1768 | ARM       | MBED_A1 | Basic            |        2.16        |       20      | 20/20 |
+--------+---------+-----------+---------+------------------+--------------------+---------------+-------+
Result: 1 OK

Completed in 111.80 sec

RobMeades · 2015-03-30T15:26:16Z

Interesting. Is that run on a C027? I'm using this ARM-GCC version:

4.9.3 20141119 (release) [ARM/embedded-4_9-branch revision 218278]

RobMeades · 2015-03-30T15:31:48Z

Just to be clear (having re-read the thread it might not be clear): I'm running LPC1768 tests but I am running them on the C027 HW platform, not the LPC1768 HW platform, hence I'm only running those tests which I know should work on the C027 HW platform (which contains an LPC1768 processor). FYI I get the same kind of unstable results if I run any of the C027 tests, it was just that I started out with these LPC1768 tests so thought I'd stick with them as nice, vanilla, tests.

The problem, I think, is specific to running tests on the C027 HW platform.

PrzemekWirkus · 2015-03-31T08:58:54Z

Yes, it is misunderstanding.

So for me on U-BLOX test MBED_A1 works well:

So I got my U-BLOX connected:

$ mbedls

+---------------------+-------------------+-------------------+--------------------------------+
|platform_name        |mount_point        |serial_port        |target_id                       |
+---------------------+-------------------+-------------------+--------------------------------+
|UBLOX_C027           |E:                 |COM101             |12340200E38AD607992A0941        |
+---------------------+-------------------+-------------------+--------------------------------+

And I configure test suite to assume my U-BLOX is LPC1768:

$ singletest.py -i test_spec.json -M muts_all.json -n MBED_A1 -V --global-loops 5 --config

MUTs configuration in muts_all.json:
+-------+-------------+---------+------+--------+-------------+
| index | peripherals | mcu     | disk | port   | copy_method |
+-------+-------------+---------+------+--------+-------------+
| 1     |             | LPC1768 | E:\  | COM101 | None        |
+-------+-------------+---------+------+--------+-------------+

Test specification in test_spec.json:
+---------+---------+
| mcu     | GCC_ARM |
+---------+---------+
| LPC1768 | Yes     |
+---------+---------+

And now five MBED_A1 tests in a row:

$ singletest.py -i test_spec.json -M muts_all.json -n MBED_A1 -V --global-loops 5
.
.
.
Executing 'python host_test.py -d E:\ -f "C:\Work\mbed\build\test\LPC1768\GCC_ARM\MBED_A1\basic.bin" -p COM101 -t 10 -C 1.5 -m LPC1768'
Test::Output::Start
MBED: Instrumentation: "COM101" and disk: "E:\"
HOST: Copy image onto target...
HOST: Initialize serial port...
....port ready!
HOST: Reset target...
HOST: Property 'timeout' = '20'
HOST: Property 'host_test_name' = 'default_auto'
HOST: Property 'description' = 'Basic'
HOST: Property 'test_id' = 'MBED_A1'
HOST: Start test...
{{success}}
{{end}}
Test::Output::Finish
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.35 of 20 sec
Test summary:
+--------+---------+-----------+---------+------------------+--------------------+---------------+-------+
| Result | Target  | Toolchain | Test ID | Test Description | Elapsed Time (sec) | Timeout (sec) | Loops |
+--------+---------+-----------+---------+------------------+--------------------+---------------+-------+
| OK     | LPC1768 | GCC_ARM   | MBED_A1 | Basic            |        1.35        |       20      |  5/5  |
+--------+---------+-----------+---------+------------------+--------------------+---------------+-------+
Result: 1 OK

Completed in 26.35 sec

PrzemekWirkus · 2015-03-31T09:01:20Z

And the same test 20 times LPC1768 tests on U-BLOX C027:

$ singletest.py -i test_spec.json -M muts_all.json -n MBED_A1 --global-loops 20

Building library CMSIS (LPC1768, GCC_ARM)
Building library MBED (LPC1768, GCC_ARM)
Building project BASIC (LPC1768, GCC_ARM)
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.35 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.35 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.37 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.35 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.35 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.36 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.35 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 0.59 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.34 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.32 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.34 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.34 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.32 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.35 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.35 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.38 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.34 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.35 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.35 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.35 of 20 sec
Test summary:
+--------+---------+-----------+---------+------------------+--------------------+---------------+-------+
| Result | Target  | Toolchain | Test ID | Test Description | Elapsed Time (sec) | Timeout (sec) | Loops |
+--------+---------+-----------+---------+------------------+--------------------+---------------+-------+
| OK     | LPC1768 | GCC_ARM   | MBED_A1 | Basic            |        1.35        |       20      | 20/20 |
+--------+---------+-----------+---------+------------------+--------------------+---------------+-------+
Result: 1 OK

Completed in 104.18 sec

PrzemekWirkus · 2015-03-31T09:15:48Z

And one more time LPC1768 tests on U-BLOX C027:

with ARM, uARM compilers and
GCC ARM 4.9.3 20150303 (release) [ARM/embedded-4_9-branch revision 221220].

$ singletest.py -i test_spec.json -M muts_all.json -n MBED_A1 --global-loops 10

Building library CMSIS (LPC1768, GCC_ARM)
Building library MBED (LPC1768, GCC_ARM)
Building project BASIC (LPC1768, GCC_ARM)
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.38 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.34 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.39 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.35 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.33 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.34 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.34 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.33 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.34 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.35 of 20 sec
Building library CMSIS (LPC1768, ARM)
Building library MBED (LPC1768, ARM)
Building project BASIC (LPC1768, ARM)
TargetTest::LPC1768::ARM::MBED_A1::Basic [OK] in 1.33 of 20 sec
TargetTest::LPC1768::ARM::MBED_A1::Basic [OK] in 1.33 of 20 sec
TargetTest::LPC1768::ARM::MBED_A1::Basic [OK] in 1.35 of 20 sec
TargetTest::LPC1768::ARM::MBED_A1::Basic [OK] in 1.35 of 20 sec
TargetTest::LPC1768::ARM::MBED_A1::Basic [OK] in 0.54 of 20 sec
TargetTest::LPC1768::ARM::MBED_A1::Basic [OK] in 0.26 of 10 sec
TargetTest::LPC1768::ARM::MBED_A1::Basic [OK] in 1.37 of 20 sec
TargetTest::LPC1768::ARM::MBED_A1::Basic [OK] in 0.54 of 20 sec
TargetTest::LPC1768::ARM::MBED_A1::Basic [OK] in 0.56 of 20 sec
TargetTest::LPC1768::ARM::MBED_A1::Basic [OK] in 0.26 of 10 sec
Building library CMSIS (LPC1768, uARM)
Building library MBED (LPC1768, uARM)
Building project BASIC (LPC1768, uARM)
TargetTest::LPC1768::uARM::MBED_A1::Basic [OK] in 1.34 of 20 sec
TargetTest::LPC1768::uARM::MBED_A1::Basic [OK] in 1.38 of 20 sec
TargetTest::LPC1768::uARM::MBED_A1::Basic [OK] in 1.35 of 20 sec
TargetTest::LPC1768::uARM::MBED_A1::Basic [OK] in 1.39 of 20 sec
TargetTest::LPC1768::uARM::MBED_A1::Basic [OK] in 1.36 of 20 sec
TargetTest::LPC1768::uARM::MBED_A1::Basic [OK] in 1.35 of 20 sec
TargetTest::LPC1768::uARM::MBED_A1::Basic [OK] in 1.38 of 20 sec
TargetTest::LPC1768::uARM::MBED_A1::Basic [OK] in 1.37 of 20 sec
TargetTest::LPC1768::uARM::MBED_A1::Basic [OK] in 1.33 of 20 sec
TargetTest::LPC1768::uARM::MBED_A1::Basic [OK] in 1.36 of 20 sec
Test summary:
+--------+---------+-----------+---------+------------------+--------------------+---------------+-------+
| Result | Target  | Toolchain | Test ID | Test Description | Elapsed Time (sec) | Timeout (sec) | Loops |
+--------+---------+-----------+---------+------------------+--------------------+---------------+-------+
| OK     | LPC1768 | GCC_ARM   | MBED_A1 | Basic            |        1.35        |       20      | 10/10 |
| OK     | LPC1768 | ARM       | MBED_A1 | Basic            |        0.26        |       10      | 10/10 |
| OK     | LPC1768 | uARM      | MBED_A1 | Basic            |        1.36        |       20      | 10/10 |
+--------+---------+-----------+---------+------------------+--------------------+---------------+-------+
Result: 3 OK

Completed in 143.39 sec

RobMeades · 2015-03-31T09:52:16Z

Very odd. Out of interest, are you driving the tests from Linux or Windows? I've just run the tests again with GCC_ARM and with target set up as LPC1768, driven from Windows, and in a loop of 20 I never get a clean run. In my first run of 20, two of them hung, in the next run of 20 six of them hung. With -v output the hang is always immediately after the line "HOST: Reset target..."

C:\projects\mbed\workspace_tools>python singletest.py -i test_spec.json -M muts_all.json -n MBED_A1 --global-loops 20
Building library CMSIS (LPC1768, GCC_ARM)
Building library MBED (LPC1768, GCC_ARM)
Building project BASIC (LPC1768, GCC_ARM)
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.34 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.34 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [TIMEOUT] in 20.02 of 10 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.37 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.34 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.38 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.36 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.37 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.36 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.35 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.38 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [TIMEOUT] in 40.27 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.36 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [TIMEOUT] in 20.03 of 10 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.35 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [TIMEOUT] in 20.48 of 10 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.36 of 20 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [TIMEOUT] in 20.51 of 10 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [TIMEOUT] in 20.48 of 10 sec
TargetTest::LPC1768::GCC_ARM::MBED_A1::Basic [OK] in 1.33 of 20 sec
Test summary:
+--------+---------+-----------+---------+------------------+--------------------+---------------+-------+
| Result | Target  | Toolchain | Test ID | Test Description | Elapsed Time (sec) | Timeout (sec) | Loops |
+--------+---------+-----------+---------+------------------+--------------------+---------------+-------+
| FAIL   | LPC1768 | GCC_ARM   | MBED_A1 | Basic            |        1.33        |       20      | 14/20 |
+--------+---------+-----------+---------+------------------+--------------------+---------------+-------+
Result: 1 FAIL

Is there any additional debug information I can retrieve for you?

Rob

PrzemekWirkus · 2015-03-31T10:44:05Z

Hi Rob,

Yes I use Windows 7 machine.
You can use option -V (capital 'V', or maybe you already used that) to get more output (serial port data exchange between C027 board and host PC).

Have you tried to manually flash test and run it multiple times (reset board using 'send bread' command from your terminal) ?

RobMeades · 2015-03-31T12:04:05Z

Aaaaaaaaa! On a hunch, I dug another board out of a desk drawer and that works 100%. There must be something very subtly wrong with the other one. Phew. Sorry for bothering you and thanks for your persistence.

RobMeades changed the title ~~Reliably running host_tests on C027 board~~ Reliably running host_tests on a u-blox C027 board Mar 23, 2015

RobMeades mentioned this issue Mar 23, 2015

Vector table must be 256 aligned #994

Merged

RobMeades closed this as completed Mar 31, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Reliably running host_tests on a u-blox C027 board #992

Reliably running host_tests on a u-blox C027 board #992

RobMeades commented Mar 23, 2015

RobMeades commented Mar 23, 2015

PrzemekWirkus commented Mar 24, 2015

PrzemekWirkus commented Mar 24, 2015

RobMeades commented Mar 24, 2015

PrzemekWirkus commented Mar 30, 2015

PrzemekWirkus commented Mar 30, 2015

RobMeades commented Mar 30, 2015

RobMeades commented Mar 30, 2015

PrzemekWirkus commented Mar 31, 2015

PrzemekWirkus commented Mar 31, 2015

PrzemekWirkus commented Mar 31, 2015

RobMeades commented Mar 31, 2015

PrzemekWirkus commented Mar 31, 2015

RobMeades commented Mar 31, 2015

Reliably running host_tests on a u-blox C027 board #992

Reliably running host_tests on a u-blox C027 board #992

Comments

RobMeades commented Mar 23, 2015

RobMeades commented Mar 23, 2015

PrzemekWirkus commented Mar 24, 2015

PrzemekWirkus commented Mar 24, 2015

RobMeades commented Mar 24, 2015

PrzemekWirkus commented Mar 30, 2015

PrzemekWirkus commented Mar 30, 2015

RobMeades commented Mar 30, 2015

RobMeades commented Mar 30, 2015

PrzemekWirkus commented Mar 31, 2015

PrzemekWirkus commented Mar 31, 2015

PrzemekWirkus commented Mar 31, 2015

RobMeades commented Mar 31, 2015

PrzemekWirkus commented Mar 31, 2015

RobMeades commented Mar 31, 2015