Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check-in PSA related auto-generated files to support online compiler. #11685

Merged
merged 4 commits into from Nov 11, 2019

Conversation

@jainvikas8
Copy link
Contributor

jainvikas8 commented Oct 14, 2019

This PR is to add support for an online compiler which is currently failing LPC55S69 (PSA target).

Support the online compiler build for PSA targets, some files are auto-generated (LPC55S69_NS and ARM_MUSCA_A1_NS) by the script from the offline build system. Since these files are identical for these V8-M targets it would be good to check them in to support online compiler as they are COMPONENT_SPE and services related.

Offline build system only for PSA related targets will be affected, as the files are checked in, it will use those files instead of auto-generating them.

@jainvikas8 jainvikas8 changed the title [WIP] Removing the PSA_AUTOGEN dependencies to support online compiler [WIP] Check-in PSA_AUTOGEN to support online compiler for PSA targets. Oct 14, 2019
@ciarmcom ciarmcom requested review from ashok-rao, maclobdell and ARMmbed/mbed-os-maintainers Oct 14, 2019
@ciarmcom

This comment has been minimized.

Copy link
Member

ciarmcom commented Oct 14, 2019

@jainvikas8 jainvikas8 force-pushed the jainvikas8:IOTSEC-1246 branch from 3fe983b to 7451445 Oct 17, 2019
@jainvikas8 jainvikas8 force-pushed the jainvikas8:IOTSEC-1246 branch 5 times, most recently from a8df8dd to 56ef8f3 Oct 31, 2019
@jainvikas8 jainvikas8 changed the title [WIP] Check-in PSA_AUTOGEN to support online compiler for PSA targets. [WIP] Check-in PSA related auto-generated files to support online compiler. Nov 4, 2019
@jainvikas8

This comment has been minimized.

Copy link
Contributor Author

jainvikas8 commented Nov 5, 2019

Needs a preceding PR merge #11775

Tested with:
mbed test --compile -m LPC55S69_NS -n *compliance* -t GCC_ARM --run
mbed test --compile -m LPC55S69_NS -n tests-psa-* -t ARMC6 --run

@jainvikas8 jainvikas8 changed the title [WIP] Check-in PSA related auto-generated files to support online compiler. Check-in PSA related auto-generated files to support online compiler. Nov 5, 2019
@0xc0170

This comment has been minimized.

Copy link
Member

0xc0170 commented Nov 5, 2019

Needs a preceding PR merge #11775

Merged, this can proceed?

@jainvikas8

This comment has been minimized.

Copy link
Contributor Author

jainvikas8 commented Nov 5, 2019

Needs a preceding PR merge #11775

Merged, this can proceed?

Yes, please start CI.

Copy link
Contributor

mark-edgeworth left a comment

This will change both the online and offline (mbed-cli) builds. It looks like the change removes all of the auto generation in favor of checked-in files for both online and offline building (ie the generator will not now be used at all). Is this intentional. If so, can you add a note to the PR description to indicate that offline builds are affected too?

@@ -44,7 +44,6 @@
from tools.utils import argparse_dir_not_parent
from tools.utils import NoValidToolchainException
from tools.utils import print_end_warnings
from tools.psa import generate_psa_sources, clean_psa_autogen
from tools.resources import OsAndSpeResourceFilter

This comment has been minimized.

Copy link
@devran01

devran01 Nov 6, 2019

Contributor

@jainvikas8 Do we still need OsAndSpeResourceFilter ?

This comment has been minimized.

Copy link
@jainvikas8

jainvikas8 Nov 6, 2019

Author Contributor

I'm not sure about this, as the main objective is to not make drastic changes to the offline build system and only remove auto-generation of header files.

source_dirs=options.source_dir,
ignore_paths=[options.build_dir]
)

resource_filter = None
if target.is_PSA_secure_target:

This comment has been minimized.

Copy link
@devran01

devran01 Nov 6, 2019

Contributor

PSA secure target is built by tools/psa/release.py. Can we remove this?

This comment has been minimized.

Copy link
@jainvikas8

jainvikas8 Nov 6, 2019

Author Contributor

Same comment as #11685 (comment)

source_dirs=options.source_dir,
ignore_paths=[options.build_dir]
)

resource_filter = None
if target.is_PSA_secure_target:

This comment has been minimized.

Copy link
@devran01

devran01 Nov 6, 2019

Contributor

PSA secure target is built by tools/psa/release.py right. Can we remove this?

This comment has been minimized.

Copy link
@jainvikas8

jainvikas8 Nov 6, 2019

Author Contributor

Same comment as #11685 (comment)

generate_psa_sources(source_dirs=options.source_dir,
ignore_paths=[]
)

resource_filter = None
if target.is_PSA_secure_target:

This comment has been minimized.

Copy link
@devran01

devran01 Nov 6, 2019

Contributor

PSA secure target is built by tools/psa/release.py right. Can we remove this?

This comment has been minimized.

Copy link
@jainvikas8

jainvikas8 Nov 6, 2019

Author Contributor

Same comment as #11685 (comment)

@@ -244,12 +240,6 @@ def main():
if target.is_PSA_secure_target:

This comment has been minimized.

Copy link
@devran01

devran01 Nov 6, 2019

Contributor

Will mbed test be run on secure targets? If not then can we remove this?

This comment has been minimized.

Copy link
@jainvikas8
@devran01

This comment has been minimized.

Copy link
Contributor

devran01 commented Nov 6, 2019

@jainvikas8 The tools/psa/release.py should be updated to check in the auto-generated files in the respective locations. The tools/psa/release.py is run as part of every mbed-os release, updating it will ensure that auto-generated and template files are in sync in mbed-os.

@mark-edgeworth

This comment has been minimized.

Copy link
Contributor

mark-edgeworth commented Nov 6, 2019

Thank you for adding the comment about offline. I've no further objections.

@Patater
Patater approved these changes Nov 7, 2019
Copy link
Contributor

Patater left a comment

LGTM

@0xc0170 0xc0170 requested a review from devran01 Nov 8, 2019
@0xc0170
0xc0170 approved these changes Nov 8, 2019
@0xc0170

This comment has been minimized.

Copy link
Member

0xc0170 commented Nov 8, 2019

@jainvikas8 Can you update your first comment based on the latest PR template (it should specify PR type) - to be certain about the release version.

@0xc0170

This comment has been minimized.

Copy link
Member

0xc0170 commented Nov 8, 2019

Started CI

@jainvikas8

This comment has been minimized.

Copy link
Contributor Author

jainvikas8 commented Nov 8, 2019

@jainvikas8 The tools/psa/release.py should be updated to check in the auto-generated files in the respective locations. The tools/psa/release.py is run as part of every mbed-os release, updating it will ensure that auto-generated and template files are in sync in mbed-os.

@0xc0170 I'm looking to resolve this above comment, so there will be tool/psa/release.py changes when any secure target is built, it will freshly generate those PSA autogenerate files as part of new mbed-os-release. This would save us time to maintain these files.
TARGET_PSA
--TARGET_MBED_SPM
--COMPONENT_SPE
psa_setup.c
--TARGET_TFM
--COMPONENT_SPE
--inc
tfm_partition_defs.inc
tfm_partition_list.inc
tfm_service_list.inc
tfm_spm_signal_defs.h
--services
--inc
autogen_sid.h
mbed_spm_partitions.h

@mbed-ci

This comment has been minimized.

Copy link

mbed-ci commented Nov 8, 2019

Test run: SUCCESS

Summary: 11 of 11 test jobs passed
Build number : 1
Build artifacts

@0xc0170 0xc0170 added needs: work and removed needs: CI labels Nov 8, 2019
jainvikas8 added 4 commits Oct 14, 2019
To support online compiler build for PSA targets, some files are
auto-generated (LPC55S69_NS and ARM_MUSCA_A1_NS) by the script
from offline build system. Since these files are identical for these
V8-M targets it would be good to check them in to support online
compiler as they are COMPONENT_SPE and services related.

In folder: components/TARGET_PSA/
  --TARGET_MBED_SPM
    --COMPONENT_SPE
      psa_setup.c
  --TARGET_TFM
    --COMPONENT_SPE
      --inc
	tfm_partition_defs.inc
        tfm_partition_list.inc
        tfm_service_list.inc
        tfm_spm_signal_defs.h
  --services
    --inc
      autogen_sid.h
      mbed_spm_partitions.h

Signed-off-by: Vikas Katariya <Vikas.Katariya@arm.com>
Only generate PSA headers/source related to components and services
when Secure build is initiated during compile time of PSA targets.
Let the Non-secure build rely on the checked-in files already present.

Signed-off-by: Vikas Katariya <Vikas.Katariya@arm.com>
Since the offline build is made to auto-generate PSA related components
and services for Secure targets, we can change the output directory to
update the files in the respective locations.
TARGET_PSA
  --TARGET_MBED_SPM
    --COMPONENT_SPE
      psa_setup.c
  --TARGET_TFM
    --COMPONENT_SPE
      --inc
	tfm_partition_defs.inc
        tfm_partition_list.inc
        tfm_service_list.inc
        tfm_spm_signal_defs.h
  --services
    --inc
      autogen_sid.h
      mbed_spm_partitions.h

The release script is been modified to commit these files if there are
any changes detected when `--commit` argument is passed.
Cleaning of auto-generated is been removed as it uses the main directory
for its operations, but PSA auto-generation will work if any of the
service and application-based manifests are updated.

Signed-off-by: Vikas Katariya <Vikas.Katariya@arm.com>
Signed-off-by: Vikas Katariya <Vikas.Katariya@arm.com>
@jainvikas8 jainvikas8 force-pushed the jainvikas8:IOTSEC-1246 branch from 56ef8f3 to cdb5ebd Nov 8, 2019
@jainvikas8 jainvikas8 requested review from 0xc0170, mark-edgeworth and Patater Nov 8, 2019
@jainvikas8

This comment has been minimized.

Copy link
Contributor Author

jainvikas8 commented Nov 8, 2019

@0xc0170 Please, can you start the CI. Thanks.

@Patater
Patater approved these changes Nov 8, 2019
@0xc0170

This comment has been minimized.

Copy link
Member

0xc0170 commented Nov 11, 2019

CI started

@mbed-ci

This comment has been minimized.

Copy link

mbed-ci commented Nov 11, 2019

Test run: SUCCESS

Summary: 11 of 11 test jobs passed
Build number : 2
Build artifacts

@0xc0170 0xc0170 added ready for merge and removed needs: work labels Nov 11, 2019
@0xc0170 0xc0170 merged commit 78d26a7 into ARMmbed:master Nov 11, 2019
26 checks passed
26 checks passed
continuous-integration/jenkins/pr-head This commit looks good
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
jenkins-ci/build-ARM Success
Details
jenkins-ci/build-GCC_ARM Success
Details
jenkins-ci/build-IAR Success
Details
jenkins-ci/cloud-client-test Success
Details
jenkins-ci/dynamic-memory-usage RTOS ROM(+0 bytes) RAM(-72 bytes)
Details
jenkins-ci/exporter Success
Details
jenkins-ci/greentea-test Success
Details
jenkins-ci/mbed2-build-ARM Success
Details
jenkins-ci/mbed2-build-GCC_ARM Success
Details
jenkins-ci/mbed2-build-IAR Success
Details
jenkins-ci/unittests Success
Details
travis-ci/astyle Success!
Details
travis-ci/docs Success!
Details
travis-ci/doxy-spellcheck Success!
Details
travis-ci/events Success! Runtime is 8633 cycles.
Details
travis-ci/gitattributestest Success!
Details
travis-ci/include_check Success!
Details
travis-ci/licence_check Success!
Details
travis-ci/littlefs Success! Code size is 8420B.
Details
travis-ci/psa-autogen Success!
Details
travis-ci/tools-py2.7 Success!
Details
travis-ci/tools-py3.5 Success!
Details
travis-ci/tools-py3.6 Success!
Details
travis-ci/tools-py3.7 Success!
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
7 participants
You can’t perform that action at this time.