Problem at running generate_scorecards

[MarceloSalazar]

Few things:

1. It complains about test_results not being defined.
   I think this line requires extra tab.
   https://github.com/maclobdell/simple_mbed_test_runner/blob/create_scorecards/generate_scorecards.py#L96

2. I've specified a folder to analyze files using option "-f .\test_output\master"
   However, it looks only one folder is analyzed.

3. It's hard to get the overall picture of what happened after the test.
   Could we also generate a master json object that can later be exported using json2html module?

@maclobdell

[maclobdell]

for 1 & 2, I've made some fixes and need to verify. For 3, lets talk more.

[MarceloSalazar]

I've updated to master , commit 906e5a2

I see a few things related to my previous comment

1. Running the script inside mbed-os, but outside of test_output folder

C:\mbed\mbed-os-multi>..\simple_mbed_test_runner\generate_scorecards.py
mbed os ver: 2018-July-2
target: DISCO_F413ZH
Traceback (most recent call last):
  File "C:\mbed\simple_mbed_test_runner\generate_scorecards.py", line 106, in <module>
    main()
  File "C:\mbed\simple_mbed_test_runner\generate_scorecards.py", line 50, in main
    generate_scorecard("./", target, mbed_ver)
  File "C:\mbed\simple_mbed_test_runner\generate_scorecards.py", line 96, in generate_scorecard
    scorecard_data[toolchain] = test_results
UnboundLocalError: local variable 'test_results' referenced before assignment

It would be great if the script could search for tests results in json format

2. Running the script and specifying a folder

C:\mbed\mbed-os-multi>..\simple_mbed_test_runner\generate_scorecards.py -f test_output\2018-July-7
mbed os ver: DISCO_F413ZH

Note, there are more folders with multiple json files.
Why did it stop on DISCO_F413ZH ?

3. Not an issue - but an enhancement request - to discuss separately ;-)

4. New (minor) issue - Running the script outside of an mbed-os repo:

C:\mbed\simple_mbed_test_runner>python generate_scorecards.py -h
Traceback (most recent call last):
  File "generate_scorecards.py", line 16, in <module>
    from tools.targets import TARGETS
ImportError: No module named tools.targets

Note: not a valid case, as today we need to use the tools.

[maclobdell]

this ticket is really messy. Too many issues mixed together. I think a lot of it has to do with running the script from the right location, etc. Closing this ticket now. lets re-test the current state and add tickets for any new issues.