Tools: Reoder post-build hooks and bl merging #9738

theotherjimmy · 2019-02-15T14:39:48Z

Description

On current master, post-build hooks are run before any merging that
managed bootloader mode would do. This puts several targets in a hard
place: they need the post-build hook to run to complute a digest of
the ROM, but when using managed bl mode, these digests are incorrect.

This PR reorders the post-build hooks to run after the merging of
managed bootloader mode, and on the "Image" (as opposed to the
"Update Image"). This should allow post build hooks that generate a
digest of the ROM to compute a correct digest.

Pull request type

[x] Fix
[ ] Refactor
[ ] Target update
[ ] Functionality change
[ ] Docs update
[ ] Test update
[ ] Breaking change

Reviewers

@bridadan, For tools changes
@ARMmbed/mbed-os-storage , This team raised the issue

bridadan

Nicely done!

bridadan · 2019-02-15T22:01:40Z

tools/toolchains/__init__.py

        # Initialize memap and process map file. This doesn't generate output.
        self.mem_stats(mapfile)

        self.notify.var("compile_succeded", True)
        self.notify.var("binary", filename)

-        return full_path, needed_update
+        return full_path, updatable


Can we drop needed_update completely then? Doesn't seem to be used in this function.

Also, I can't find a call in our code base that actually uses the updatable result from link_program(). Is it possible to remove it?

Yes. We can remove that too. We may have to update the online compiler to use this new API.

theotherjimmy · 2019-02-19T17:00:05Z

@bridadan Ready for review again, supposing that travis passes.

bridadan

Good stuff!

cmonr · 2019-02-19T20:05:23Z

@ARMmbed/mbed-os-storage Please review.

cmonr · 2019-02-25T17:50:45Z

Still waiting on a review by @ARMmbed/mbed-os-storage

@bulislaw Fyi

cmonr · 2019-02-25T17:50:53Z

@theotherjimmy And now this needs a rebase.

theotherjimmy · 2019-02-25T20:07:32Z

@cmonr Rebase done.

cmonr · 2019-02-25T22:57:39Z

CI started assuming @ARMmbed/mbed-os-storage will eventually_ be ok with this PR.

mbed-ci · 2019-02-26T01:30:22Z

Test run: FAILED

Summary: 3 of 8 test jobs failed
Build number : 2
Build artifacts

Failed test jobs:

jenkins-ci/mbed-os-ci_build-GCC_ARM
jenkins-ci/mbed-os-ci_build-ARM
jenkins-ci/mbed-os-ci_build-IAR

bulislaw · 2019-02-26T15:15:52Z

This PR is at risk of missing 5.12 release as it's marked as "needs: work". Code freeze is coming! On Friday 1st. Please make necessary updates ASAP and make sure the reviewers are aligned for prompt code inspection.

yossi2le · 2019-02-26T15:57:52Z

This PR looks fine for me. @theotherjimmy, I have only one question. The changes here originated from the need to compile and build FUTURE_SEQUANA_PSA with bootloader support. in order to do so @bridadan helped us with some tweaks and changes. are all those changes included in the PR or they are not needed here?

dannybenor · 2019-02-26T16:08:31Z

@bulislaw please be aware that this PR is related to PSA, and we may need additional changes when the platforms code will be available

theotherjimmy · 2019-02-26T17:05:41Z

Errors seem to be CI passing in unsupported platforms.

theotherjimmy · 2019-02-26T17:06:57Z

@dannybenor You may need additional PRs. This change is complete as is, so long as no bugs are found around the reordering. Please don't hold PRs because you might find that you need changes related to them.

theotherjimmy · 2019-02-26T17:08:05Z

@yossi2le You're going to have to remind me what those changes are. They probably should come in as other PRs though.

cmonr · 2019-02-27T05:27:29Z

@bulislaw please be aware that this PR is related to PSA, and we may need additional changes when the platforms code will be available

@dannybenor Which is when? We're running out of days before code freeze...

cmonr · 2019-02-27T05:27:38Z

@theotherjimmy This needs a rebase.

theotherjimmy · 2019-02-27T15:33:04Z

@cmonr rebase done

Aparantely I missed this previously

The 4 remaining are related to the fact that __init__.py defines a base class. They'll be removed next commit

except for the extremely long regex

adbridge · 2019-03-04T17:47:40Z

Ci started

mbed-ci · 2019-03-04T20:03:50Z

Test run: SUCCESS

Summary: 13 of 13 test jobs passed
Build number : 5
Build artifacts

cmonr · 2019-03-04T20:57:24Z

CI job restarted: jenkins-ci/exporter

Job config was updated to properly run against list of targets instead of an empty list.

cmonr · 2019-03-04T21:23:48Z

Hmmm.... Trying a complete rebuild.

Suspect that the export fix that was recently applied didn't quite do the job.

cmonr · 2019-03-04T21:24:10Z

CI restarted

mbed-ci · 2019-03-04T23:23:56Z

Test run: FAILED

Summary: 1 of 13 test jobs failed
Build number : 6
Build artifacts

Failed test jobs:

jenkins-ci/mbed-os-ci_exporter

cmonr · 2019-03-06T05:18:16Z

CI restarted.
Merge commit needs to use updated master

mbed-ci · 2019-03-06T08:17:52Z

Test run: FAILED

Summary: 1 of 13 test jobs failed
Build number : 7
Build artifacts

Failed test jobs:

jenkins-ci/mbed-os-ci_exporter

0xc0170 · 2019-03-06T12:02:09Z

exporters green 👍

cmonr added the needs: review label Feb 15, 2019

cmonr requested review from bridadan and a team February 15, 2019 15:22

bridadan approved these changes Feb 15, 2019

View reviewed changes

bridadan approved these changes Feb 19, 2019

View reviewed changes

cmonr added release-version: 5.12.0-rc1 risk: A labels Feb 19, 2019

cmonr added needs: work and removed needs: review labels Feb 25, 2019

theotherjimmy force-pushed the reorder-bl-hooks branch from 105224d to bedfcff Compare February 25, 2019 20:07

theotherjimmy force-pushed the reorder-bl-hooks branch from bedfcff to b5e79b2 Compare February 26, 2019 17:18

theotherjimmy force-pushed the reorder-bl-hooks branch from b5e79b2 to b7e0102 Compare February 27, 2019 15:32

0xc0170 added needs: review and removed needs: work labels Feb 27, 2019

theotherjimmy added 13 commits March 4, 2019 11:32

Remove use of hooks from IAR

a007212

Aparantely I missed this previously

Correct use before definition bug in ARM

3353722

Drop unused variable from link_program

cd20c37

Add SPDX identifier

9c9e059

Define filename when OUTPUT_EXT is elf

98a7bfc

Prevent "Image: None" in log

7a55bc7

Correct all pylint warnings and errors except 4

b0b43c2

The 4 remaining are related to the fact that __init__.py defines a base class. They'll be removed next commit

Move toolchain base class into another file

a2fcdba

Fix formatting erros in arm.py

5dd149c

except for the extremely long regex

Fix pylint warnings in gcc.py except for the long regex

1712506

Fix pylint in iar.py except for the long regex

c9674dc

Add SPDX identifier to liscense in "new" file

1f4c271

Remove unused import patch

c9b613c

theotherjimmy force-pushed the reorder-bl-hooks branch from 2eea5cc to c9b613c Compare March 4, 2019 17:36

0xc0170 added needs: CI and removed needs: work labels Mar 5, 2019

0xc0170 mentioned this pull request Mar 5, 2019

make armc6: error 8 for assembly file #9938

Closed

0xc0170 added ready for merge and removed needs: CI labels Mar 6, 2019

0xc0170 merged commit 9d72a5c into ARMmbed:master Mar 6, 2019

0xc0170 removed the ready for merge label Mar 6, 2019

Tools: Reoder post-build hooks and bl merging #9738

Tools: Reoder post-build hooks and bl merging #9738

Uh oh!

Conversation

theotherjimmy commented Feb 15, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Pull request type

Reviewers

Uh oh!

bridadan left a comment

Choose a reason for hiding this comment

Uh oh!

bridadan Feb 15, 2019

Choose a reason for hiding this comment

Uh oh!

theotherjimmy Feb 19, 2019

Choose a reason for hiding this comment

Uh oh!

theotherjimmy commented Feb 19, 2019

Uh oh!

bridadan left a comment

Choose a reason for hiding this comment

Uh oh!

cmonr commented Feb 19, 2019

Uh oh!

cmonr commented Feb 25, 2019

Uh oh!

cmonr commented Feb 25, 2019

Uh oh!

theotherjimmy commented Feb 25, 2019

Uh oh!

cmonr commented Feb 25, 2019

Uh oh!

mbed-ci commented Feb 26, 2019

Test run: FAILED

Uh oh!

bulislaw commented Feb 26, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yossi2le commented Feb 26, 2019

Uh oh!

dannybenor commented Feb 26, 2019

Uh oh!

theotherjimmy commented Feb 26, 2019

Uh oh!

theotherjimmy commented Feb 26, 2019

Uh oh!

theotherjimmy commented Feb 26, 2019

Uh oh!

cmonr commented Feb 27, 2019

Uh oh!

cmonr commented Feb 27, 2019

Uh oh!

theotherjimmy commented Feb 27, 2019

Uh oh!

adbridge commented Mar 4, 2019

Uh oh!

mbed-ci commented Mar 4, 2019

Test run: SUCCESS

Uh oh!

cmonr commented Mar 4, 2019

Uh oh!

cmonr commented Mar 4, 2019

Uh oh!

cmonr commented Mar 4, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mbed-ci commented Mar 4, 2019

Test run: FAILED

Uh oh!

cmonr commented Mar 6, 2019

Uh oh!

mbed-ci commented Mar 6, 2019

Test run: FAILED

Uh oh!

0xc0170 commented Mar 6, 2019

Uh oh!

Uh oh!

theotherjimmy commented Feb 15, 2019 •

edited

Loading

bulislaw commented Feb 26, 2019 •

edited

Loading

cmonr commented Mar 4, 2019 •

edited

Loading