Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implementing an alternative build framework with CMake #1776

Merged
merged 30 commits into from
Dec 21, 2022

Conversation

massonal
Copy link
Contributor

Implementing an alternative build framework with CMake

This PR implements an alternative build engine to Arduino IDE or arduino-cli.
Although widely compatible with these tools, a CMake implementation is much faster (especially on Windows, especially behind a proxy/firewall): in the order of 10sec, instead of up to a minute with the IDE on my machine. Other benefits include more flexibility in describing the build, and enhanced support of incremental compilation, including when making changes to the core.

Most features of Arduino IDE are reimplemented here; the most obvious missing feature is the lack of automatic dependency management (libraries). Indeed, the way Arduino implements can not be mirrored cleanly with CMake.

How to use:

Examples can be found at https://github.com/massonal/STM32CMake_workspace.
Briefly, there needs to be a CMakeLists.txt present in the sketch, along the sources. The recommended template makes heavy use of wrapper functions designed to mirror the Arduino automation; these are implemented in the /cmake folder here.
Please read the wiki I started writing on my fork for more details.

How to maintain

  • Arduino doesn't "see" CMake files, so this PR overall should have no effect on the Arduino tools.
  • Most CMake files are autogenerated from a template; this script updates them all. I advise running it automatically as a git hook after this PR gets merged.

@fpistm fpistm added this to In progress in STM32 core based on ST HAL via automation Jul 29, 2022
@fpistm fpistm self-requested a review July 29, 2022 12:16
@massonal massonal force-pushed the cmake_dev branch 4 times, most recently from 2f8ea81 to 290b330 Compare August 3, 2022 07:15
Copy link
Member

@fpistm fpistm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use passive form as far as possible

README_CMAKE.md Outdated Show resolved Hide resolved
STM32 core based on ST HAL automation moved this from In progress to Needs review Aug 3, 2022
@massonal massonal force-pushed the cmake_dev branch 3 times, most recently from c0babcf to f5960ab Compare August 4, 2022 12:53
@massonal massonal marked this pull request as draft August 9, 2022 07:28
@massonal massonal force-pushed the cmake_dev branch 4 times, most recently from ed57b42 to fe5794c Compare August 12, 2022 09:31
@massonal massonal force-pushed the cmake_dev branch 5 times, most recently from 2da70fa to c4b84c4 Compare September 8, 2022 06:44
massonal and others added 14 commits December 20, 2022 14:52
The script uses its location and arduino-cli to write a template
of the sketch's CMakeLists.txt.
* Defines:
  - CORE_PATH
  - USER_LIBS -> for use with `external_library()`
* Includes all user libraries
* Usable on a given sketch or generically (use placeholders)
* Optionally: run CMake (config+build)
I.e. changed some variables from "CACHE" to "regular in the parent scope"

CACHE variables can exhibit some counterintuitive behavior;
switching back to regular variables is thus safer
and does not significantly impact build time.
Using:
- platform.txt to get the version of the core
- the dev version of the JSON describing all the tools+versions+deps
- stand-alone: Python, wrt. version and modules.

Changes:
- The download folder is no longer inside Arduino_Core_STM32, but alongside.
- The options to control the download (clearance + folder) are removed.
- CMake: harmonize the `cmake_minimum_required()`s

CMSIS and xpack are no longer downloaded _in_ the root folder, but alongside.

All this parsing is costly (e.g., on rebuild), so shortcuts have been implemented if
it looks like the dependencies are satisfied already.
- Added a new kind of insight: logic structure
- Added the CORE_CALLBACK overall setting
Important: the changed implemented here _require_ the user
to call `project()` in "step 1", before calling any custom function,
but after defining the toolchain file.

The whole of Arduino_Core_STM32 (variant + core) is now added
automatically when the user calls build_sketch() ; they don't have to
do it themselves anymore.

Also the standard libraries have moved from stm32_runtime to base_config.
Indeed, even the files from the core depend on them.

Also, _all_ the Arduino libraries depend on base_config only (even SrcWrapper).
Additional dependencies were actually not needed.

***Moved the CMake-related files to /cmake***
-- for clarity with the remainder of the project (when using Arduino IDE)
-- Also, this prevents IDEs from building the core "standalone" when seeing a CMakeLists.txt at the root,
--   since that would be meaningless anyway.

Also improve portability by replacing backslashes with forward slashes in paths on Windows.

=======================

Change: moved the download folder away

Having the download folder along the core seemed to
cause malfunctions with some external tools (Arduino plugin on VScode).

Moving it in the user's home folder fixes this issue.

Each core installation nonetheless has its own download subfolder,
to allow different versions with different requirements to coexist.
Using a fuzzy find after parsing boards.txt.

This has required refactoring the board parser code in a dedicated modules,
with minor impacts in the files that depended on it.

=============

Doc changes:

expand on the syntax in easy_setup:
Added a comment block to explain the use of the DEPENDS clause.
Added "usual" values for board settings.
Add a link to the wiki in easy_setup.

Also update link in the README_CMAKE
- update link to point to the new location for the examples;
- unify the displayed text for the links;
- add a link to the wiki.
This adds a new keyword, BOOTLOADER, in set_board().

Cmake-wise, the bootloader targets replace the board targets, they are not feature targets to plug in.
Signed-off-by: Frederic Pillon <frederic.pillon@st.com>
Signed-off-by: Frederic Pillon <frederic.pillon@st.com>
Signed-off-by: Alexis Masson <alexis.masson@st.com>
Mostly to benefit from the built-in "--help",
as the script takes no actual argument.
ASM not being enabled when `external_library()` is called
triggered an error where CMAKE_ASM_COMPILE_OBJECT was undefined.
This commit fixes that by adding the calls to enable_language()
earlier in the config process.
massonal and others added 4 commits December 20, 2022 16:31
Now detects and warns when arduino-cli is misconfigured,
and falls back to a sensible default value for userlibs.
To mirror as closely as possible the behavior of Arduino tools.
Signed-off-by: Frederic Pillon <frederic.pillon@st.com>
Signed-off-by: Frederic Pillon <frederic.pillon@st.com>
@fpistm fpistm marked this pull request as ready for review December 20, 2022 16:11
@fpistm fpistm self-requested a review December 20, 2022 16:13
@fpistm fpistm force-pushed the cmake_dev branch 2 times, most recently from f17548f to 74d524e Compare December 20, 2022 16:16
Signed-off-by: Frederic Pillon <frederic.pillon@st.com>
Copy link
Member

@fpistm fpistm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @massonal for this PR.
As a first implementation it is a good job.
Ready to merge and deploy.

STM32 core based on ST HAL automation moved this from Needs review to Reviewer approved Dec 20, 2022
@fpistm fpistm merged commit 280efe1 into stm32duino:main Dec 21, 2022
STM32 core based on ST HAL automation moved this from Reviewer approved to Done Dec 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Development

Successfully merging this pull request may close these issues.

None yet

2 participants