New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Skip a few tests if not enough memory can be allocated for them #7465

Merged
merged 1 commit into from Aug 1, 2018

Conversation

Projects
None yet
9 participants
@davidsaada
Contributor

davidsaada commented Jul 10, 2018

Description

Some tests need large amounts of memory in order to work. These tests will fail on low end boards, having low memory. This is also shown in issue #7115 where these tests fail on the TMPM066 board. This PR fixes the problem by simply skipping the test if there's not enough memory. All allocations use the std::nothrow directive.
PR depends on #7448.

Pull request type

[x] Fix
[ ] Refactor
[ ] New target
[ ] Feature
[ ] Breaking change

@cmonr cmonr requested review from ARMmbed/mbed-os-core Jul 10, 2018

@cmonr cmonr added the needs: review label Jul 10, 2018

@davidsaada

This comment has been minimized.

Contributor

davidsaada commented Jul 10, 2018

@soramame21 Can you test if this fixes #7115?

@adbridge

This comment has been minimized.

Contributor

adbridge commented Jul 10, 2018

@davidsaada I don't think you should have commit e0ca511 in this PR ? Isn't that what #7448 is ?

@cmonr

This comment has been minimized.

Contributor

cmonr commented Jul 11, 2018

@adbridge It'll go away once a rebase is done to this PR.
Plus, even if a rebase isn't done, because the hashes are the same, git will not duplicate the commits.

(I've done something similar and it worked out)

@soramame21

This comment has been minimized.

Contributor

soramame21 commented Jul 13, 2018

I had passed the test cases related to this fix. please see the log.

mbed test -m TMPM066 -t GCC_ARM -vv -n features-tests-filesystem-util_block_device

[1531465438.32][CONN][RXD] >>> Running 3 test cases...
[1531465438.37][CONN][INF] found KV pair in stream: {{__testcase_name;Testing slicing of a block device}}, queued...
[1531465438.43][CONN][INF] found KV pair in stream: {{__testcase_name;Testing chaining of block devices}}, queued...
[1531465438.49][CONN][RXD]
[1531465438.49][CONN][INF] found KV pair in stream: {{__testcase_name;Testing profiling of block devices}}, queued...
[1531465438.56][CONN][RXD] >>> Running case #1: 'Testing slicing of a block device'...
[1531465438.62][CONN][INF] found KV pair in stream: {{__testcase_start;Testing slicing of a block device}}, queued...
[1531465438.72][CONN][RXD] :41::SKIP: Not enough memory for test
[1531465438.72][CONN][INF] found KV pair in stream: {{__testcase_finish;Testing slicing of a block device;1;0}}, queued...
[1531465438.78][CONN][RXD] >>> 'Testing slicing of a block device': 1 passed, 0 failed
[1531465438.78][CONN][RXD]
[1531465438.85][CONN][RXD] >>> Running case #2: 'Testing chaining of block devices'...
[1531465438.91][CONN][INF] found KV pair in stream: {{__testcase_start;Testing chaining of block devices}}, queued...
[1531465439.01][CONN][RXD] :146::SKIP: Not enough memory for test
[1531465439.01][CONN][INF] found KV pair in stream: {{__testcase_finish;Testing chaining of block devices;1;0}}, queued...
[1531465439.08][CONN][RXD] >>> 'Testing chaining of block devices': 1 passed, 0 failed
[1531465439.08][CONN][RXD]
[1531465439.14][CONN][RXD] >>> Running case #3: 'Testing profiling of block devices'...
[1531465439.20][CONN][INF] found KV pair in stream: {{__testcase_start;Testing profiling of block devices}}, queued...
[1531465439.31][CONN][RXD] :216::SKIP: Not enough memory for test
[1531465439.31][CONN][INF] found KV pair in stream: {{__testcase_finish;Testing profiling of block devices;1;0}}, queued...
[1531465439.37][CONN][RXD] >>> 'Testing profiling of block devices': 1 passed, 0 failed
[1531465439.37][CONN][RXD]
[1531465439.41][CONN][RXD] >>> Test cases: 3 passed, 0 failed
[1531465439.44][CONN][INF] found KV pair in stream: {{__testcase_summary;3;0}}, queued...
[1531465439.47][CONN][INF] found KV pair in stream: {{max_heap_usage;0}}, queued...
[1531465439.47][HTST][ERR] orphan event in main phase: {{max_heap_usage;0}}, timestamp=1531465439.466000
[1531465439.49][CONN][INF] found KV pair in stream: {{reserved_heap;0}}, queued...
[1531465439.49][HTST][ERR] orphan event in main phase: {{reserved_heap;0}}, timestamp=1531465439.488000
[1531465439.51][CONN][INF] found KV pair in stream: {{end;success}}, queued...
[1531465439.52][HTST][INF] __notify_complete(True)
[1531465439.52][HTST][INF] __exit_event_queue received
[1531465439.52][HTST][INF] test suite run finished after 1.26 sec...
[1531465439.52][CONN][INF] found KV pair in stream: {{__exit;0}}, queued...
[1531465439.52][CONN][INF] received special event '__host_test_finished' value='True', finishing
[1531465439.53][HTST][INF] CONN exited with code: 0
[1531465439.53][HTST][INF] Some events in queue
[1531465439.53][HTST][INF] stopped consuming events
[1531465439.53][HTST][INF] host test result() call skipped, received: True
[1531465439.53][HTST][WRN] missing __exit event from DUT
[1531465439.53][HTST][INF] calling blocking teardown()
[1531465439.53][HTST][INF] teardown() finished
[1531465439.53][HTST][INF] {{result;success}}
mbedgt: checking for GCOV data...
mbedgt: mbed-host-test-runner: stopped and returned 'OK'
mbedgt: test on hardware with target id: 701100000000000000000000000000000000000097969904
mbedgt: test suite 'features-tests-filesystem-util_block_device' ..................................... OK in 41.27 sec
        test case: 'Testing chaining of block devices' ............................................... OK in 0.10 sec
        test case: 'Testing profiling of block devices' .............................................. OK in 0.11 sec
        test case: 'Testing slicing of a block device' ............................................... OK in 0.10 sec
mbedgt: test case summary: 3 passes, 0 failures
mbedgt: all tests finished!
mbedgt: shuffle seed: 0.8397540719
mbedgt: test suite report:
+-----------------+---------------+---------------------------------------------+--------+--------------------+-------------+
| target          | platform_name | test suite                                  | result | elapsed_time (sec) | copy_method |
+-----------------+---------------+---------------------------------------------+--------+--------------------+-------------+
| TMPM066-GCC_ARM | TMPM066       | features-tests-filesystem-util_block_device | OK     | 41.27              | default     |
+-----------------+---------------+---------------------------------------------+--------+--------------------+-------------+
mbedgt: test suite results: 1 OK
mbedgt: test case report:
+-----------------+---------------+---------------------------------------------+------------------------------------+--------+--------+--------+--------------------+
| target          | platform_name | test suite                                  | test case                          | passed | failed | result | elapsed_time (sec) |
+-----------------+---------------+---------------------------------------------+------------------------------------+--------+--------+--------+--------------------+
| TMPM066-GCC_ARM | TMPM066       | features-tests-filesystem-util_block_device | Testing chaining of block devices  | 1      | 0      | OK     | 0.1                |
| TMPM066-GCC_ARM | TMPM066       | features-tests-filesystem-util_block_device | Testing profiling of block devices | 1      | 0      | OK     | 0.11               |
| TMPM066-GCC_ARM | TMPM066       | features-tests-filesystem-util_block_device | Testing slicing of a block device  | 1      | 0      | OK     | 0.1                |
+-----------------+---------------+---------------------------------------------+------------------------------------+--------+--------+--------+--------------------+
mbedgt: test case results: 3 OK
mbedgt: completed in 44.20 sec
@davidsaada

This comment has been minimized.

Contributor

davidsaada commented Jul 13, 2018

Thanks @soramame21. Also got report that it solves #7439

@davidsaada davidsaada force-pushed the davidsaada:david_tests_not_enough_memory branch from 539ab2a to a224ca7 Jul 16, 2018

@davidsaada

This comment has been minimized.

Contributor

davidsaada commented Jul 16, 2018

Rebased. No longer depending on #7448.

@cmonr

This comment has been minimized.

Contributor

cmonr commented Jul 18, 2018

Fixes #6008

@0xc0170

This comment has been minimized.

Member

0xc0170 commented Jul 25, 2018

@mprse @fkjagodzinski @maciejbocianski Can you review these test changes ?

@0xc0170

This comment has been minimized.

Member

0xc0170 commented Jul 31, 2018

Waiting for core team to review, meanwhile

/morph build

@geky

geky approved these changes Jul 31, 2018 edited

I don't know if I count as core team, but this is a good solution 👍

EDIT: GitHub says I do, but I'm not sure that's correct. @SenRamakri?

@cmonr

cmonr approved these changes Jul 31, 2018

Interesting. Curious to see how well this works. LGTM.

@cmonr

This comment has been minimized.

Contributor

cmonr commented Jul 31, 2018

/morph build

@mbed-ci

This comment has been minimized.

mbed-ci commented Jul 31, 2018

Build : SUCCESS

Build number : 2705
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/7465/

Triggering tests

/morph test
/morph uvisor-test
/morph export-build
/morph mbed2-build

@mbed-ci

This comment has been minimized.

mbed-ci commented Jul 31, 2018

Build : SUCCESS

Build number : 2708
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/7465/

Triggering tests

/morph test
/morph uvisor-test
/morph export-build
/morph mbed2-build

@mbed-ci

This comment has been minimized.

@mbed-ci

This comment has been minimized.

@cmonr

This comment has been minimized.

Contributor

cmonr commented Jul 31, 2018

That's what I get for double building...
/morph export-build

@mbed-ci

This comment has been minimized.

@mbed-ci

This comment has been minimized.

@mbed-ci

This comment has been minimized.

@0xc0170 0xc0170 added ready for merge and removed needs: CI labels Aug 1, 2018

@0xc0170 0xc0170 merged commit d5f70f0 into ARMmbed:master Aug 1, 2018

14 checks passed

AWS-CI uVisor Build & Test Success
Details
ci-morph-build build completed
Details
ci-morph-exporter build completed
Details
ci-morph-mbed2-build build completed
Details
ci-morph-test test completed
Details
continuous-integration/jenkins/pr-head This commit looks good
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
travis-ci/astyle Passed, 794 files
Details
travis-ci/docs Local docs testing has passed
Details
travis-ci/events Passed, runtime is 8836 cycles (-1175 cycles)
Details
travis-ci/gitattributestest Local gitattributestest testing has passed
Details
travis-ci/licence_check Local licence_check testing has passed
Details
travis-ci/littlefs Passed, code size is 9960B (+0.00%)
Details
travis-ci/tools-py2.7 Local tools-py2.7 testing has passed
Details

@0xc0170 0xc0170 removed the ready for merge label Aug 1, 2018

@davidsaada davidsaada deleted the davidsaada:david_tests_not_enough_memory branch Aug 1, 2018

pan- pushed a commit to pan-/mbed that referenced this pull request Aug 22, 2018

Merge pull request ARMmbed#7465 from davidsaada/david_tests_not_enoug…
…h_memory

Skip a few tests if not enough memory can be allocated for them
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment