Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Armclang / armlink support in Zephyr - Zephyr CMake linker functions #36140

Merged
merged 20 commits into from Aug 30, 2021

Conversation

@tejlmand
Copy link
Collaborator

@tejlmand tejlmand commented Jun 10, 2021

This is the initial support for Zephyr CMake linker functions.

Using CMake for linker script generations allows us to specify memory regions, linker sections, linker symbols in a common way using CMake functions, and then generate toolchain specific linker scripts / scatter files.

This allows users to only write things once and avoid linker scripts to get out of sync when only a single script (for example ld script) is updated, but not other linker scripts.

Besides a unifies Linker script language, another advantage of using CMake is that users can add new sections in their application CMakeLists.txt file.

Using CMake, a user can now write:

zephyr_linker_memory(NAME FOO FLAGS rx START 0x40000000 SIZE 1k)
zephyr_linker_section(NAME bar LMA FOO)
zephyr_linker_section_configure(SECTION bar INPUT baz KEEP)

and this can even be done in the app CMakeLists.txt file, like:

find_package(Zephyr REQUIRED)

target_sources(app PRIVATE baz.c)

zephyr_linker_memory(NAME FOO FLAGS rx START 0x40000000 SIZE 1k)
zephyr_linker_section(NAME bar LMA FOO)
zephyr_linker_section_configure(SECTION bar INPUT baz KEEP)

This will create a memory region named FOO, with a section bar where the input section baz will be placed with the KEEP flag.

Also it is possible to describe a new section in CMake together with the library as:

zephyr_library_named(pinmux)
zephyr_library_sources(pinmux.c)
zephyr_linker_section(NAME .pinmux GROUP ROM_REGION)
or
zephyr_linker_section(NAME .pinmux LMA ROM)

With this PR, the arm architecture has been ported to the new Zephyr CMake linker generator.

When building with armclang, the new linker generator is always used.
When building with zephyr or gnuarmemb, the old system will be used, and the new linker generator must be manually selected using: -DCONFIG_CMAKE_LINKER_GENERATOR=y (or by using menuconfig).

To build using armclang set the toolchain to armclang and the path to the toolchain.

export ZEPHYR_TOOLCHAIN_VARIANT=armclang
export ARMCLANG_TOOLCHAIN_PATH=/opt/armds-2020-1/sw/ARMCompiler6.15/

Building hello_world for qemu_cortex_m3 using armclang / armlink:

$ cmake -GNinja -DBOARD=qemu_cortex_m3 -Bbuild_qemu samples/hello_world/
...
-- The C compiler identification is ARMClang 6.15.2
-- The CXX compiler identification is ARMClang 6.15.2
-- The ASM compiler identification is ARMClang
...
$ ninja -Cbuild_qemu run
[116/117] To exit from QEMU enter: 'CTRL+a, x'[QEMU] CPU: cortex-m3
qemu-system-arm: warning: nic stellaris_enet.0 has no peer
*** Booting Zephyr OS build zephyr-v2.6.0-69-gcfa5059ceaeb  ***
Hello World! qemu_cortex_m3

and for nrf52840dk_nrf52840:

$ cmake -GNinja -DBOARD=nrf52840dk_nrf52840 -Bbuild samples/hello_world/
...
$ ninja -Cbuild run
$ ninja -Cbuild flash
...
...
-- runners.nrfjprog: Board with serial number 683845540 flashed successfully.

Open minicom and see the console output:

*** Booting Zephyr OS build zephyr-v2.6.0-69-gcfa5059ceaeb  ***
Hello World! nrf52840dk_nrf52840

Building with Zephyr toolchain using new linker generator:

$ export ZEPHYR_TOOLCHAIN_VARIANT=zephyr
$ cmake -GNinja -DBOARD=qemu_cortex_m3 -Bbuild_qemu samples/hello_world/ 
 -DCONFIG_CMAKE_LINKER_GENERATOR=y
....
$ ninja -Cbuild_qemu run
[124/125] To exit from QEMU enter: 'CTRL+a, x'[QEMU] CPU: cortex-m3
qemu-system-arm: warning: nic stellaris_enet.0 has no peer
Timer with period zero, disabling
*** Booting Zephyr OS build zephyr-v2.6.0-69-gcfa5059ceaeb  ***
Hello World! qemu_cortex_m3

ToDo:

  • Documentation
  • Kconfig setting for linker script generator
  • Memory sections from Device Tree
  • Linker groups / regions
  • Userspace support
  • Linker symbols for groups
  • offset on section configure

Additional notes:

  • CMake v3.18 or higher required, see #37575
  • Issues seen when using make, so for time being, please use ninja CMake 3.21 with make is supported. Older CMake with make will issue a error message, see: d72ec88
  • Warnings will be seen during linking. Those warnings will be cleaned up before going into mergable PR state.
  • In general, memory regions should be defined using devicetree, and then Zephyr CMake linker template will create corresponding memory regions in the linker script, see #37624

The cleanup commit series can be found here as independent PR:
#37938

Fixes #33435

Copy link
Contributor

@galak galak left a comment

I think some of these commits we could pull out and merge sooner:

  • cmake: removed stray space in readelf outfile flag
  • linker: removing unused _DATA_IN_ROM
  • linker: align __data_ram/rom_start/end linker symbol names
    etc.

Loading

@@ -2245,3 +2279,379 @@ function(target_byproducts)
COMMENT "Logical command for additional byproducts on target: ${TB_TARGET}"
)
endfunction()

########################################################
Copy link
Contributor

@galak galak Jun 10, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should put these in their own file (cmake/linker.cmake) and think about removing the zephyr_ prefix as I think this could be used by other projects.

Loading

Copy link
Collaborator Author

@tejlmand tejlmand Jun 10, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense.

Loading

Copy link
Collaborator Author

@tejlmand tejlmand Jun 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@galak, not sure if linker.cmake is best choice directly at the <zephyr>/cmake/ level as it may not be obvious to people that this particular file contains CMake extension functions / macros, and we also uses linker.cmake elsewhere in this PR, meaning less obvious to people browsing the repo that cmake/linker.make contains extension functions.

Maybe we should refactor and the create:

<zephyr>/cmake/extensions  <-- folder
<zephyr>/cmake/extensions/zephyr.cmake  <-- Very Zephyr specific extensions, functions prefixed with `zephyr_` (taken from extensions.cmake)
<zephyr>/cmake/extensions/generic.cmake <-- More generic CMake extensions (taken from extensions.cmake)
<zephyr>/cmake/extensions/linker.cmake  <-- Linker specific CMake extensions, functions prefixed with `linker_` )

thoughts ?

Loading

@tejlmand
Copy link
Collaborator Author

@tejlmand tejlmand commented Jun 10, 2021

I think some of these commits we could pull out and merge sooner:

I completely agree, and this is also my intention, but I wanted to create this PR first so people could se the reasoning for such independent cleanup.

Loading

@tejlmand tejlmand force-pushed the armclang branch 4 times, most recently from 60f211d to 0dd656f Jun 11, 2021
/*
* Tell armclang that stack alignment are ensured.
*/
.eabi_attribute Tag_ABI_align_preserved, 1
Copy link
Member

@stephanosio stephanosio Jun 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since 8-byte stack alignment currently is/must always be ensured, it would be better to make this part of _ASM_FILE_PROLOGUE.

Loading

Copy link
Collaborator Author

@tejlmand tejlmand Jun 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very good point, thanks

Loading

Copy link
Member

@stephanosio stephanosio Aug 27, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since we are almost out of time, a reminder to myself to refactor this in the future. @stephanosio

Loading

@stephanosio stephanosio requested a review from povergoing Jun 11, 2021
lib/os/cbprintf_packaged.c Outdated Show resolved Hide resolved
Loading
kernel/main_weak.c Outdated Show resolved Hide resolved
Loading
cmake/extensions.cmake Outdated Show resolved Hide resolved
Loading
cmake/extensions.cmake Outdated Show resolved Hide resolved
Loading
cmake/extensions.cmake Outdated Show resolved Hide resolved
Loading
# r: Read-only region
# w: Read-write region
# x: Executable region
# The flags r and x, or w and x may be combined like: rx, wx.
Copy link
Member

@mbolivar-nordic mbolivar-nordic Jun 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not an expert, but are there other flags that might be necessary here besides just these on some linkers? If so, should this be moved into multi_value_keywords instead of single_args? That would let you say FLAGS r x while potentially leaving this a bit more future proof.

Or maybe a comma-separated list?

Loading

Copy link
Collaborator Author

@tejlmand tejlmand Jun 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point.
For a start I will leave it a bit open, to allow some more thoughts on best approach here.

@TTornblom would there be something to consider for IAR in this regard ?

Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tejlmand I will check this out, thanks.

Loading

Copy link
Member

@mbolivar-nordic mbolivar-nordic Aug 19, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will check this out, thanks.

@TTornblom ping

Loading

cmake/extensions.cmake Outdated Show resolved Hide resolved
Loading
cmake/extensions.cmake Outdated Show resolved Hide resolved
Loading
cmake/extensions.cmake Outdated Show resolved Hide resolved
Loading
@tejlmand tejlmand force-pushed the armclang branch 3 times, most recently from 8b44d5c to 3191ee9 Jun 14, 2021
@dleach02
Copy link
Member

@dleach02 dleach02 commented Jul 13, 2021

Addresses #33435

Loading

@cfriedt
Copy link
Collaborator

@cfriedt cfriedt commented Jul 29, 2021

@tejlmand

I wonder if this might tie in with #37123 (macos native_posix support).

The major issue with macos is that the Mach-O binary file format only supports section names 1 <= n <= 16 chars in length, and we generate a number of linker-defined sections that exceed that size.

One possible solution for that would be some kind of mapping for section names. I'll be looking at other solutions as well. Currently macos support is very brute force.

Loading

Tell armlink that files has ensured proper stack alignment.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
@tejlmand tejlmand force-pushed the armclang branch 4 times, most recently from 7c35509 to 125e14a Aug 28, 2021
@tejlmand
Copy link
Collaborator Author

@tejlmand tejlmand commented Aug 28, 2021

@nashif @carlescufi There are two failures:

  1. Compliance, the libc additions for ARMClang contains typedef, just like they do for minimal, newlib, and mwdt files, so those error should be ignored.
  2. The errno.h cause failure because of this:
    /*
    * Copyright (c) 1982, 1986 Regents of the University of California.
    * All rights reserved. The Berkeley software License Agreement
    * specifies the terms and conditions for redistribution.
    *
    * @(#)errno.h 7.1 (Berkeley) 6/4/86
    */

    where the main part of the header originates from here:
    https://github.com/zephyrproject-rtos/zephyr/blob/2bb0ed859c2e2300eaa3a8178c92b2e238e58d74/lib/libc/minimal/include/errno.h
    I think we should ignore this warning, if not, please advice on what to do.

Besides that CI is green.

Loading

Copy link
Member

@stephanosio stephanosio left a comment

For an experimental feature, looks good enough to me.

Any nits/refinements/refactorings can be done later.

Loading

tejlmand added 18 commits Aug 30, 2021
This commit introduces zephyr_linker CMake functions for creation of
linker scripts.

The linker functions supports the following features:
- Configuration of memory sections
- Configuration of memory groups
- Creation of output sections
- Configuration of input section the should go into output section and
  how those should be treated, such as sorting, keep, priority.
- Defining linker symbols
- Specifying Kernel VMA to support virtual linking on x86

Overview of functions and macros introduce with this commit:
- pow2round
- zephyr_linker
- zephyr_linker_memory
- zephyr_linker_memory_ifdef
- zephyr_linker_group
- zephyr_linker_section
- zephyr_linker_section_ifdef
- zephyr_iterable_section
- zephyr_linker_section_obj_level
- zephyr_linker_section_configure
- zephyr_linker_symbol
- zephyr_linker_property_append

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
This commit introduces zephyr_linker_dts CMake functions for creation of
linker scripts based on devicetree nodes.

The linker devicetree functions supports the following features:
- Configuration of memory sections based on devicetree nodes

Overview of functions introduced with this commit:
- zephyr_linker_dts_memory

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Converted existing ld script templates into CMake files.

This commit takes the common-ram.ld, common-rom.ld, debug-sections.ld,
and thread-local-storage.ld and creates corresponding CMake files for
the linker script generator.

The CMake files uses the new Zephyr CMake functions:
- zephyr_linker_section()
- zephyr_linker_section_configure()
- zephyr_linker_section_obj_level()

to generate the same linker result as the existing C preprocessor based
scheme.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
This commit specifies the intList section in the IDT_LIST region in the
arch/common CMakeLists.txt file.

It uses zephyr_linker_section to setup the intList section for first
pass linker file and configures the section to hold irq_info and
intList input section.

For second pass linker file, the irq_info and intList input sections are
placed in the /DISCARD/ section.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
This is the initial version of a Zephyr CMake linker file for the arm
architecture.

This file defines memory regions, groups, linker sections and symbols
for the arm architecture.

It also sources the common common-ram.cmake, common-rom.cmake,
debug-sections,cmake, and thread-local-storage.cmake.

It configure sections for SoC families using zephyr_linker_sources()
functions:
- nxp_imx

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
This commit add devicetree memory regions added to arm/linker.cmake.

The following memory regions are now added to the generated linker
script and scatter file if they exists in the devicetree.
- chosen:    zephyr,itcm
- chosen:    zephyr,dtcm
- nodelabel: ti_ccfg_partition
- nodelabel: sram1
- nodelabel: sram2
- nodelabel: sram3
- nodelabel: sram4
- nodelabel: sdram1
- nodelabel: sdram2
- nodelabel: backup_sram

Also support for the itcm and dtcm section and their placement in
corresponding memory regions has been added.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Adding intial version of linker_script_common.cmake.

This script parses memory regions, groups, sections, settings, and
symbols defined in the Zephyr CMake build system.

The linker script creates objects for each type above and groups
sections, symbols, and groups together based on their configuration.

This creates a hierarchy which the `<linker>_script.cmake`
implementations can then use during linker script generation.

linker_script_common.cmake also provides stubs for <type>_to_string()
functions which are used when generating the output script.

Each specific `<linker>_script.cmake` must implement those to_string()
functions to create the final linker script.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Adding intial version of ld_script.cmake.

This script can generate ld linker script from the Zephyr CMake linker
functions.

This script is called by the build system and all linker settings, such
as memory regions, groups, output sections, linker symbols defined with
Zephyr CMake linker functions are passed to this script in order to
generate an ld linker script.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
A "Linker script" choice group has been added to the linker options
menu.

The existing HAVE_CUSTOM_LINKER_SCRIPT config has been relocated inside
the linker script menu.

Two new entries has been created:
- LD template, for using the old ld template files together with the C
  pre-processor to create the final linker script.
- CMake linker generator, used for ARMClang / armlink toolchain.
  The CMake linker generator is also supported for LD linker scripts
  for the ARM architecture.

In CMake, the file cmake/linker/ld/target.cmake has been updated to
support the CMake LD linker generator.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
The armclang.h includes the toolchain/llvm.h header.

Also it redifines the __GENERIC_SECTION and Z_GENERIC_SECTION so that
they includes the `used` attribute which is needed by armlink.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Initial bintools support for elfconvert
- hex, srec, bin conversion
- strip
- gap fill

Initial bintools support for readelf
Initial bintools support for disaasembly

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
This is the initial support for the armclang compiler together with the
armlink linker.

Introduced in this commit:
- armclang compiler support
- armlink linker support
- armlink scatter file generator for scatter loading
- dual pass linker script generation

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
When using ARMClang linker and scatter files (armlink) then all
libraries are linked as object libraries using `$<TARGET_OBJECTS:lib>`.

CMake version 3.20 only has limited support for such linking:
> Referencing $<TARGET_OBJECTS> in target_link_libraries calls worked
> in versions of CMake prior to 3.21 for some cases, but was not fully
> supported.

One of those cases that do not work is Unix Makefiles generators.

As only Ninja is currently verified to work, this commit will check the
CMake version in use and the generator, and if CMake version <=3.21 is
used with non-Ninja generator then an error is raised informing user
to either use Ninja generator or update CMake.

As the Ninja generator has been confirmed to work as expected, then that
generator is accepted with CMake 3.20 and older.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
The default armlink signature uses `--list=<TARGET_BASE>.map`, but in
Zephyr we uses a different name for the map file, therefore we need to
specify a custom link executable signature.

This is done in the linker specific flags file:
cmake/linker/armlink/linker_flags.cmake

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
This commit introduces armlink steering file.

A steering file in armlink allows Zephyr to keep using existing linker
symbols defined in ld scripts and used throughout the code tree.

The steering file is generated at build time in order to resolve Zephyr
linker symbols to their corresponding armlink symbols.

As example, Zephyr defines __ramfunc_start which corresponds to the
armlink auto defined Image$$ramfunc$$Base symbol.

Or __init_PRE_KERNEL_1_start which corresponds to Image$$init_0$$Base.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Support for ARM Compiler C library.

This commit add support for the ARM Compiler C libary in:
- Kconfig
- libc/armstdc

A new Kconfig symbol is added to allow a toolchain to specify if they
support linking with the minimal C library.
Also the CMake variable `TOOLCHAIN_HAS_NEWLIB` is exported to Kconfig
so that CONFIG_NEWLIB_LIBS can only be enabled if the toolchain has
newlib.

The armclang toolchain selects the CMake scatter file generator and
disables support for the LD linker template which is not supported by
armlink.

For the ARM Compiler C library, a corresponding lib/libc/armstc/ folder
with a minimal implementation to work with the ARM Compiler C library
is added.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
The stub file threading_weak.c has been added containing weak stub
implementation of threading related kernel functions.

The file is needed for armlink.

When linking with armlink the linker will resolve undefined symbols for
all undefined functions even if those functions the reference the
undefined symbol is never actually called.

This file provides weak stub implementations that are compiled when
CONFIG_MULTITHREADING=n to ensure proper linking.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
This commit adds an additional test case for several kernel test suites
to ensure that the linker script generator is working correctly for a
subset of the Zephyr test suites.

The ensures that the basic functionality of the linker script generator
is working while still keep the performance impact on CI at a minimal
level.

Using the kernel tests is a trade-off between testing coverage of the
linker script generator and the time it takes to complete CI.

The kernel tests is considered to have the broadest coverage of various
features important for the generated linker script.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
@nashif nashif merged commit 2760fb9 into zephyrproject-rtos:main Aug 30, 2021
15 of 17 checks passed
Loading
@tejlmand tejlmand mentioned this pull request Aug 31, 2021
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment