Enable ARM(SVE) CPU support with reference backend #2614

ajay-fuji · 2023-12-15T17:28:51Z

Proposed changes in this PR

Enabled support for build on ARM CPUs (SVE).
Arm build supports DAL Interface (daal) and oneAPI C++ interface (oneapi_c).
ci/env/tbb.sh helps to build tbb for ARM (SVE) cpu and installs it in the right folder structure for use during compile time.
Make command for arm(sve):
make -f makefile daal oneapi_c BACKEND_CONFIG=ref PLAT=lnxarm -j$(nproc)

Open Issues/Limitations:

oneAPI DPC++ interface (oneapi) is not supported since Intel's DPC++ compiler is not supported on arm.
Bazel build is yet not supported.

keeranroth

For the most part, the structure looks good, and a change for building on a generic (non vendor-specific) CPU looks very similar to what is here. One of the most common comments I have (I didn't put all of them in), is that it's probably good to have x86 branches first, as this is the most likely code path at the moment.

There are a few things to discuss still, which can be quite big in themselves:

Do we target building with Bazel for new architectures? I don't have an opinion yet, as I'm not familiar with Bazel
How do we structure the code for new architectures? At the moment, there are multiple #ifdef in the files, where if this is done for each (micro-)architecture, the files can become cluttered. I would advocate for having target-specific sub-directories, and put target specific code in those, keeping as much common code in the files they are currently in.

.ci/scripts/build.sh

azure-pipelines.yml

deploy/local/vars_lnx.sh

deploy/pkg-config/generate_pkgconfig.py

keeranroth · 2023-12-18T12:36:13Z

cpp/oneapi/dal/algo/subgraph_isomorphism/backend/cpu/compiler_adapt.hpp

@@ -16,7 +17,11 @@

 #pragma once
 #include <cstdint>
-#include <immintrin.h>
+
+#ifndef __ARM_ARCH


Something like #ifdef TARGET_X86_64 - it shouldn't be included if Arm isn't defined. It should be included if x86 is defined.

cpp/oneapi/dal/algo/triangle_counting/backend/cpu/intersection_tc.hpp

cpp/oneapi/dal/backend/primitives/intersection/intersection.hpp

cpp/oneapi/dal/detail/policy.hpp

makefile

rakshithgb-fujitsu · 2023-12-18T13:26:42Z

For the most part, the structure looks good, and a change for building on a generic (non vendor-specific) CPU looks very similar to what is here. One of the most common comments I have (I didn't put all of them in), is that it's probably good to have x86 branches first, as this is the most likely code path at the moment.

There are a few things to discuss still, which can be quite big in themselves:

Do we target building with Bazel for new architectures? I don't have an opinion yet, as I'm not familiar with Bazel

How do we structure the code for new architectures? At the moment, there are multiple #ifdef in the files, where if this is done for each (micro-)architecture, the files can become cluttered. I would advocate for having target-specific sub-directories, and put target specific code in those, keeping as much common code in the files they are currently in.

@keeranroth we din't really change any structure (apart from adding make file extensions for micro-archs) and stuck to what x86 follows so we just added arm compatibility on top of it. My thoughts on further points:

@napetrov prefers the new architectures to support bazel, however x86 itself isn't fully ready yet so we chose to support the make file approach. We are now actively working towards supporting bazel on arm. Bazel is new for us as well and might take time to support it.
This would require a massive code re-architecture since only external APIs are isolated (BLAS/LAPACK etc.), ideally we would need to replicate similar methods for other parts of the algorithms as well. I agree with you on this to isolate the micro-architecture components.

cpp/daal/include/services/env_detect.h

napetrov · 2023-12-18T19:27:12Z

/intelci: run

amgrigoriev · 2023-12-19T07:36:48Z

The approach above involves too many mk files like CMPLR.BACKEND.ARCH.mk. Having "n" backends and "m" archs, it will give [nxm] matrix. For every new arch it will be necessary to add all backends and vise versa and repeat it for all supported compilers. My suggestion is to separate backend and arch definitions to have a single file at least for every backend. For instance, defs.ref.mk for "ref" backend could contain COMPILER.defs += -DDAAL_REF -DONEDAL_REF -DONEAPI_DAAL_NO_MKL_GPU_FUNC while cmplr.clang.x86.mk could use COMPILER.defs above (both are included in main makefile): **COMPILER.mac.clang = clang++ -m64 -fgnu-runtime -stdlib=libc++ -mmacosx-version-min=10.15 -fwrapv
-Werror -Wreturn-type $(COMPILER.defs)

COMPILER.lnx.clang = clang++ -m64
-Werror -Wreturn-type
$(COMPILER.defs)**

aepanchi · 2023-12-19T10:18:51Z

cpp/daal/include/services/env_detect.h

@@ -42,11 +43,16 @@ namespace daal
 */
 enum CpuType
 {
+#ifdef __ARM_ARCH
+    sve         = 0, /*!< ARM processors based on Arm's Scalable Vector Extension (SVE) */


Suggested change

sve = 0, /*!< ARM processors based on Arm's Scalable Vector Extension (SVE) */

sve = 0, /*!< ARM(R) processors based on Arm(R) Scalable Vector Extension (SVE) */

aepanchi · 2023-12-19T10:19:21Z

cpp/daal/include/services/env_detect.h

-        avx512      = 2  /*!< Intel(R) Xeon(R) processors based on Intel(R) Advanced Vector Extensions 512 (Intel(R) AVX-512) \DAAL_DEPRECATED */
+
+#ifdef __ARM_ARCH
+        sve = 2, /*!< ARM processors based on Arm's Scalable Vector Extension (SVE) */


Suggested change

sve = 2, /*!< ARM processors based on Arm's Scalable Vector Extension (SVE) */

sve = 2, /*!< ARM(R) processors based on Arm(R) Scalable Vector Extension (SVE) */

cpp/daal/src/algorithms/kernel_inst_arm.h

dev/make/cmplr.clang.ref.arm.mk

Vika-F

Thank you for adding ARM support to oneDAL! This is a tremendous work.
I have reviewed the code in ./cpp folder and it looks good and mostly correct. Please see the additional comments below.

But my biggest concern is that the Azure pipelines are not updated to test those changes.
.ci/pipelines/ci.yml file needs to be updated to support a new job for ARM.
Can you please update the pipelines so we can see if the changes are working as expected?

cpp/daal/src/algorithms/algorithm_hyperparameter.cpp

Vika-F · 2023-12-19T10:09:17Z

cpp/daal/src/algorithms/covariance/covariance_impl.i

+#ifdef __ARM_ARCH
+    #define CPU_TYPE sve
+#else
+    #define CPU_TYPE avx512
+#endif
+


Please replace CPU_TYPE with DAAL_CPU_TYPE.
Also those changes might negatively impact the performance on ARM because the size of the block was tuned here for AVX512 and the same block size might not be good for other CPU architectures.

It is possible to tune the block size by running performance experiments with the different block sizes using oneDAL API as follows:

const auto input = dal::read<dal::table>(dal::csv::data_source{ input_file_name }); const auto cov_desc = dal::covariance::descriptor{}.set_result_options( dal::covariance::result_options::cor_matrix | dal::covariance::result_options::means); dal::covariance::detail::compute_parameters params{}; params.set_cpu_macro_block(140 /* tested block size */); const auto result = dal::compute(cov_desc, params, input);

It is also possible to use Python API for this purpose.

Thank you for suggestion

Hello @Vika-F,

Can you please share the python equivalent code to generate block size, and what "input_file_name" should be use to get optimal block size ?

Covariance algorithms' hyperparameters can be tuned using PCA, as the Covariance algorithm is used as part of PCA.
Here is the code:

pca = PCA(n_components=2, svd_solver="full") pca.get_hyperparameters("fit").cpu_macro_block = 140 # tested block size pca.fit(X)

The input_file_name (or ndarray X in Python code) should point to the file with the input data for Covariance/PCA algorithm.
You can place there any data which you think is relevant for performance tuning.

cpp/daal/src/algorithms/kernel_inst_arm.h

Vika-F · 2023-12-19T11:10:59Z

cpp/daal/src/services/compiler/generic/env_detect_features.cpp

@@ -41,6 +46,28 @@
 void __daal_serv_CPUHasAVX512f_enable_it_mac();
 #endif

+#if defined(__ARM_ARCH)


Agree. This sounds reasonable. Possible naming for the new directory: compiler/arm.

cpp/daal/src/services/env_detect.cpp

Vika-F · 2023-12-19T11:40:21Z

cpp/oneapi/dal/algo/jaccard/backend/cpu/vertex_similarity_default_kernel_avx512.hpp

+#ifdef __ARM_ARCH
+vertex_similarity_result<task::all_vertex_pairs> jaccard_sve(
+#else
 vertex_similarity_result<task::all_vertex_pairs> jaccard_avx512(
+#endif


I think this will not work like this. Because the code below uses AVX512 intrinsics which are incompatible with SVE.
That's why please use the default code path on ARM.

cpp/oneapi/dal/detail/policy.hpp

dev/download_tbb.sh

dev/make/cmplr.clang.ref.arm.mk

azure-pipelines.yml

KulikovNikita · 2023-12-19T16:23:28Z

cpp/daal/include/algorithms/algorithm_container_base_batch.h

@@ -1,6 +1,7 @@
 /* file: algorithm_container_base_batch.h */
 /*******************************************************************************
 * Copyright 2014 Intel Corporation
+* Copyright 2023-24 FUJITSU LIMITED


Suggested change

* Copyright 2023-24 FUJITSU LIMITED

* Copyright 2024 FUJITSU LIMITED

I believe we have an internal agreement that year in the file header reflect the year of initial file creation or modification (in your case)

KulikovNikita · 2023-12-19T16:25:36Z

cpp/daal/include/algorithms/algorithm_container_base_batch.h

@@ -143,10 +144,15 @@ class AlgorithmContainerImpl<batch> : public AlgorithmContainer<batch>
 * \tparam avx2Container        Implementation for Intel(R) Advanced Vector Extensions 2 (Intel(R) AVX2)
 * \tparam avx512Container      Implementation for Intel(R) Xeon(R) processors based on Intel AVX-512
 */
+#ifdef __ARM_ARCH


What about other macros? I believe I've seen at least __arch64__, _M_ARM64 somewhere in ARM specific code (in other repos)

cpp/daal/include/algorithms/algorithm_container_base_batch.h

KulikovNikita · 2023-12-19T16:32:31Z

cpp/daal/src/services/compiler/generic/env_detect_features.cpp

@@ -1,6 +1,7 @@
 /* file: env_detect_features.cpp */


I would actually suggest to split this particular file into env_detct_x86_features.cpp and ``env_detect_arm_features.cppin order to not create the#ifdef` hell

I would also suggest to have the files live in cpp/daal/src/services/compiler/generic/<arch>/env_detect_features.cpp, and then have the appropriate file built by the build system, depending on compile-time configuration

cpp/daal/src/services/service_topo.h

cpp/daal/src/threading/export_lnxarm.ref.def

cpp/oneapi/dal/algo/subgraph_isomorphism/backend/cpu/compiler_adapt.hpp

Vika-F · 2023-12-20T10:27:16Z

cpp/daal/src/algorithms/kernel_inst_arm.h

@@ -0,0 +1,131 @@
+/* file: kernel_inst_arm.h */


The file "kernel_inst_x86.h" was recently modified - some unused macros were removed. Please see PR #2617 for the details.
Please make the similar removals in this file.
The macros that contain "SYCL" should also be removed as all new SYCL development is moved into oneapi/dal part of the code base.

@Vika-F that PR does not show any SYCL related modifications, could you please help point it out?

@rakshithgb-fujitsu - this is not part of this PR, but a comment for longerterm plans. all sycl related finctionality in DAAL library is deprecated and will be removed. so there is no point in copying DAAL SYCL related defines.

@rakshithgb-fujitsu Yes, my point was about the deprecation of SYCL-related functionality in DAAL. Since it is deprecated, no need to replicate those defines for ARM. They will be removed in near future anyway.

Now I also see that it is not easy to remove those defines at all right now as they are used in various files.

So, let's leave them as is for now in order not to block the development anymore.

napetrov · 2023-12-20T13:23:37Z

.ci/scripts/build.sh

@@ -1,6 +1,7 @@
 #! /bin/bash
 #===============================================================================
 # Copyright 2019 Intel Corporation
+# Copyright 2023-24 FUJITSU LIMITED


Per discussion with UXL - we can start adding "Copyright Contributors to the oneDAL project", while keeping existing Intel copyright that already exist.

There should be some checks fixes done internally to reflect this + we will be starting using new copyright as well

napetrov · 2023-12-22T21:52:03Z

cpp/oneapi/dal/algo/covariance/backend/cpu/partial_compute_kernel_dense.cpp

@@ -53,7 +60,7 @@ static partial_compute_result<Task> call_daal_kernel_partial_compute(
    /// the logic of block size calculation is copied from DAAL,
    /// to be changed to passing the values from the performance model
    std::int64_t blockSize = 140;
-    if (ctx.get_enabled_cpu_extensions() == dal::detail::cpu_extension::avx512) {
+    if (ctx.get_enabled_cpu_extensions() == CPU_EXTENSION) {


not sure if this is right way to handle this - the idea of this block is that this is avx512 specialized code here and as well as in some other algorithms. It assume avx512 isa and i'm not sire that it would work that easily on ARM. We probably can have a better approach defined for architecture/ISA specific implementation in algorithms.

Agreed. One possible approach is to have a function defined which returns the block size, and this can be different per architecture. This is already done in a few locations with templating, but it looks like we want a runtime check here, so a slightly different implementation is required.

deploy/local/vars_lnx.sh

napetrov · 2023-12-22T22:04:40Z

dev/download_tbb.sh

+
+  rm -rf __deps/tbb
+  pushd oneTBB
+    git checkout v2021.11.0


even once this would be in separate script it would worth placing dependency version at least to the header of the file

makefile

.ci/env/tbb.sh

keeranroth · 2024-01-02T09:21:54Z

.ci/env/tbb.sh

+CoreCount=$(lscpu -p | grep -Ev '^#' | wc -l)
+
+rm -rf __deps/tbb
+pushd oneTBB


The name of the directory should reflect that this is a build directory, so onetbb-build or similar would be better. I'm not a fan of mixed-case names, but that is just a personal preference.

@keeranroth thank you for suggestion.
Currently makefile expect TBB dependency to reside inside __deps/tbb/. Changing name of directory would require making changes in other makefiles as well, which at this point I think is not required. @napetrov any comments.

I was referring to the oneTBB directory name, which is put into the root directory at the moment. This can probably be onetbb-src or something similar, so that the clone at the top of this file would be:

git clone https://github.com/oneapi-src/oneTBB.git onetbb-src

and it's probably worth doing a shallow clone so that the switch of the branch isn't necessary

git clone --depth 1 --branch v2021.11.0 https://github.com/oneapi-src/oneTBB.git onetbb-src

i think downloading sources archive is a better option that gitclones here.

In terms of names - i guess the point is to have build directory properly named as build directory and compiler tbb can be put in to dependencies directory. Point is that it would be easier to debug things with clear naming

@napetrov, what if we go with shallow clone as suggested by @keeranroth so that same task can be done in less lines of code as compared to archive download.

git clone --depth 1 --branch v2021.11.0 https://github.com/oneapi-src/oneTBB.git onetbb-src

have nothing against git clone. The only point - it would be good to have single place for versions modification.

Also there is open question on if env/tbb.sh should be merged with download_tbb script from dev folder

The original pull request had both in the same file, which I commented I thought looked weird, in that the download script was also an Arm build recipe. I think that it's better with two different scripts, but happy to go with the majority on that one.

2 different scipts is option, but then tbb build script for arm should be named properly. For now it's just tbb.sh that would mean that this is universal script. So might be just rename it it to reflect that it's intended to work on arm only and potentially add platform check there.

I also think env/tbb.sh should be generic. Only catch is for intel and arm, we know dependency path are __deps/tbb/lnx/lib/intel64/gcc4.8/ and __deps/tbb/lnx/lib/arm/gcc4.8/ respectively. How do we take care for other architecture?
Our suggestion for deciding architecture directory -

arch=$(uname -m) if [ "${arch}" == "x86_64" ]; then arch_dir="intel64" elif [ "${arch}" == "aarch64" ]; then arch_dir="arm" else arch_dir=${arch} fi

yes, we can put things in this way. Although - i thought that arm support would be cross compiled ? i.e. build would happen on x86 machine for arm target. so uname will not necessary return arm platform

Also i would say we can move download_tbb script logic here for Intel and just use prebuild tbb instead of building it from sources.

.ci/env/tbb.sh

.ci/scripts/test.sh

cmake/scripts/generate_config.cmake

keeranroth · 2024-01-02T10:33:11Z

deploy/nuget/prepare_dal_nuget.sh

@@ -101,6 +110,9 @@ create_package() {
        if [ "${platform}" = "linux-x64" ]; then
            dl_postfix=.so.${major_binary_version}.${minor_binary_version}
            sl_postfix=.a
+        elif [ "${platform}" = "linux-aarch64" ]; then
+            dl_postfix=.${major_binary_version}.${minor_binary_version}.dylib


Should this be .so suffix on linux?

deploy/pkg-config/generate_pkgconfig.py

dev/make/identify_os.sh

examples/cmake/setup_examples.cmake

keeranroth · 2024-01-02T11:58:14Z

makefile

@@ -1,5 +1,6 @@
 #===============================================================================


It would be nice to have this file with architecture-specific parts split out into their own files. That requires quite a bit of a refactor of this file so that we can keep as much common code in here as possible. It should be possible to just split out the setting of variables into architecture specific parts at the start of the file, and then use these later on in the makefile. I'm happy to have a look at this, if required.

@keeranroth yup please have a look at this and do make suggestions as needed, this seems a bit tricky to isolate the parts.

Ack. I'll start having a look at this tomorrow.

rageshhajela16 · 2024-02-19T04:47:39Z

The internal CI failure with our examples are still unresolved - anyone else on the intel side able to take a peek at the fails?

@ethanglaser @napetrov can you please help to check if the internal CI is still failing with the recent rebase! Thanks

Alexandr-Solovev · 2024-02-19T08:32:21Z

/intelci: run

Alexandr-Solovev · 2024-02-19T11:10:45Z

Hi @rageshhajela16!
I have checked the last commit. Unfortunately, the red ci steps still exist. After the investigation of internal ci issues, I found the issue.
The true issue log in examples is

I doublecheck how __release_lnx/daal/latest/lib/cmake/oneDAL/oneDALConfig.cmake looks like and it looks

So instead of lib/"arch_name" it uses just lib, and I would like to say that its incorrectly.
In comparison with master(it has green ci) it looks like

I also doublechecked your changes and these changes looks good for me https://github.com/oneapi-src/oneDAL/pull/2614/files#diff-a711276942a2647ec3ee01db428047af7e0efcd9ed25a93f600bc9bf53b70c2dR212
But may be its possible to replace it with ${ARCH_DIR_ONEDAL}, what do you think? Or may be you have some more ideas about what could be the reason. I checked env variables and ARCH_DIR_ONEDAL is set correctly. I also will work on the investigation

ajay-fuji · 2024-02-19T16:11:57Z

Hi @rageshhajela16! I have checked the last commit. Unfortunately, the red ci steps still exist. After the investigation of internal ci issues, I found the issue. The true issue log in examples is

I doublecheck how __release_lnx/daal/latest/lib/cmake/oneDAL/oneDALConfig.cmake looks like and it looks

So instead of lib/"arch_name" it uses just lib, and I would like to say that its incorrectly. In comparison with master(it has green ci) it looks like

I also doublechecked your changes and these changes looks good for me https://github.com/oneapi-src/oneDAL/pull/2614/files#diff-a711276942a2647ec3ee01db428047af7e0efcd9ed25a93f600bc9bf53b70c2dR212 But may be its possible to replace it with ${ARCH_DIR_ONEDAL}, what do you think? Or may be you have some more ideas about what could be the reason. I checked env variables and ARCH_DIR_ONEDAL is set correctly. I also will work on the investigation

From the build directory name it looks like, onedal is built using icc compiler (now depracated in version 2024.0, please confirm!).
We have run tried building onedal and examples using below commands -
make -f makefile onedal_c -j32
.ci/scripts/test.sh --test-kind examples --build-dir__release_lnx --compiler gnu --interface daal/cpp --build_system cmake
.ci/scripts/test.sh --test-kind examples --build-dir__release_lnx --compiler gnu --interface oneapi/cpp --build_system cmake

Everything works fine. All test examples building and passing successfully.

here compiler is automatically taken as icc (/opt/intel/oneapi/compiler/2023.2.1/linux/bin/intel64/icc) (noticed from logs).
icc installed via sudo apt install intel-oneapi-compiler-dpcpp-cpp-and-cpp-classic

oneDALConfig.cmake contain correct prefix -

Just to cross verify, is it possible to share configuration used to generate binaries? More precisely build command.
Thanks!

Alexandr-Solovev · 2024-02-20T08:05:15Z

Hi @ajay-fuji @rageshhajela16 . I also checked it with manually build and it works fine. For me rn it looks like an issue in the CI step command/script, will check, thanks!

napetrov · 2024-02-20T10:29:45Z

Hi @rageshhajela16! I have checked the last commit. Unfortunately, the red ci steps still exist. After the investigation of internal ci issues, I found the issue. The true issue log in examples is
I doublecheck how __release_lnx/daal/latest/lib/cmake/oneDAL/oneDALConfig.cmake looks like and it looks

So instead of lib/"arch_name" it uses just lib, and I would like to say that its incorrectly. In comparison with master(it has green ci) it looks like

I also doublechecked your changes and these changes looks good for me https://github.com/oneapi-src/oneDAL/pull/2614/files#diff-a711276942a2647ec3ee01db428047af7e0efcd9ed25a93f600bc9bf53b70c2dR212 But may be its possible to replace it with ${ARCH_DIR_ONEDAL}, what do you think? Or may be you have some more ideas about what could be the reason. I checked env variables and ARCH_DIR_ONEDAL is set correctly. I also will work on the investigation

From the build directory name it looks like, onedal is built using icc compiler (now depracated in version 2024.0, please confirm!). We have run tried building onedal and examples using below commands - make -f makefile onedal_c -j32 .ci/scripts/test.sh --test-kind examples --build-dir__release_lnx --compiler gnu --interface daal/cpp --build_system cmake .ci/scripts/test.sh --test-kind examples --build-dir__release_lnx --compiler gnu --interface oneapi/cpp --build_system cmake

Everything works fine. All test examples building and passing successfully.

here compiler is automatically taken as icc (/opt/intel/oneapi/compiler/2023.2.1/linux/bin/intel64/icc) (noticed from logs). icc installed via sudo apt install intel-oneapi-compiler-dpcpp-cpp-and-cpp-classic

oneDALConfig.cmake contain correct prefix -

Just to cross verify, is it possible to share configuration used to generate binaries? More precisely build command. Thanks!

on icc usage question - yes, we use it for build currently with plan for migration to icx.
But for icc it's public build, one of latest versions.

Alexandr-Solovev · 2024-02-20T11:02:34Z

@ethanglaser here the job with infra fixes https://intel-ci.intel.com/eecfd78a-97b3-f181-afe8-a4bf010d0e2e
Examples look good now.

napetrov · 2024-02-20T12:35:57Z

@ethanglaser here the job with infra fixes https://intel-ci.intel.com/eecfd78a-97b3-f181-afe8-a4bf010d0e2e Examples look good now.

would suggest running nightly validation to check on broader scope, but so far things looks good and ready for integration

Alexandr-Solovev · 2024-02-20T12:59:39Z

@ethanglaser here the job with infra fixes https://intel-ci.intel.com/eecfd78a-97b3-f181-afe8-a4bf010d0e2e Examples look good now.

would suggest running nightly validation to check on broader scope, but so far things looks good and ready for integration

Previous regular job has successfully finished!
Nightly job with this pr: https://intel-ci.intel.com/eecfef9e-97dc-f1b8-9360-a4bf010d0e2e

Alexandr-Solovev · 2024-02-20T14:21:25Z

/intelci: run

napetrov

Let's wait for CI/Nightly results but overall changes are good to go

Alexandr-Solovev · 2024-02-20T15:16:58Z

/intelci: run

ethanglaser

Good from CI perspective. Thanks for taking a look @Alexandr-Solovev!

Alexandr-Solovev

LGTM! Ci is also green. @napetrov should you merge it?

napetrov · 2024-02-21T08:45:20Z

Thanks everyone for working on this PR!

rageshhajela16 · 2024-02-21T09:06:20Z

Thanks everyone for working on this PR!

Thanks everyone contributing to this PR and for the review!

Following on from pull request oneapi-src#2614, we want to make it easier to distinguish common and platform specific code when building. We take common code from the main makefile, and split this out into arch and uarch specific files. We try and share as much code as possible between uarch files. In this first commit, we make sure that the x86 build works on linux. Next commits will make sure that the Mac, Windows and Arm builds also work. For now these are broken.

* Don't use double brackets in source script Match the same style as used in the rest of the script. I observed a syntax error when running in a Docker container locally as well, so fix this up quickly. * Move arch and uarch specific code out of main Makefile Following on from pull request #2614, we want to make it easier to distinguish common and platform specific code when building. We take common code from the main makefile, and split this out into arch and uarch specific files. We try and share as much code as possible between uarch files. In this first commit, we make sure that the x86 build works on linux. Next commits will make sure that the Mac, Windows and Arm builds also work. For now these are broken. * Update Mac, Windows and Arm makefile configuration Put Mac and Windows makefile code into their own files, and correct some copy-paste errors in the Arm build. This should build on all platforms now, but there are some more commits to go on top before the refactor is done. * Refactor compiler configuration in makefiles We want to have as much common code shared between makefiles for different compilers, as well as for the different backends. We shuffle things around a little. This now passes on x86 Linux as well as aarch64. Needs some internal tests to be run. * Remove recursive definition * Code review copyright notice and new line changes * Remove year from contributors copyright notice * Don't use year in copyright * review comments

ajay-fuji requested review from razdoburdin, ahuber21, avolkov-intel, icfaust, Alexsandruss, samir-nasibli, KulikovNikita, Alexandr-Solovev, napetrov, homksei and ethanglaser as code owners December 15, 2023 17:28

keeranroth suggested changes Dec 18, 2023

View reviewed changes

napetrov reviewed Dec 18, 2023

View reviewed changes

cpp/daal/include/services/env_detect.h Outdated Show resolved Hide resolved

napetrov requested review from Vika-F, maria-Petrova and aepanchi December 18, 2023 19:21

aepanchi reviewed Dec 19, 2023

View reviewed changes

Vika-F reviewed Dec 19, 2023

View reviewed changes

KulikovNikita reviewed Dec 19, 2023

View reviewed changes

Vika-F reviewed Dec 20, 2023

View reviewed changes

napetrov reviewed Dec 20, 2023

View reviewed changes

napetrov reviewed Dec 22, 2023

View reviewed changes

deploy/local/vars_lnx.sh Outdated Show resolved Hide resolved

napetrov reviewed Dec 22, 2023

View reviewed changes

makefile Outdated Show resolved Hide resolved

keeranroth reviewed Jan 2, 2024

View reviewed changes

rakshithgb-fujitsu force-pushed the enable-arm-build-with-ref-backend branch from 71c0021 to ab084d1 Compare January 4, 2024 07:02

rakshithgb-fujitsu force-pushed the enable-arm-build-with-ref-backend branch from 04ee1b0 to b86d408 Compare February 15, 2024 11:52

Merge branch 'oneapi-src:main' into enable-arm-build-with-ref-backend

7fbf4ed

napetrov approved these changes Feb 20, 2024

View reviewed changes

ethanglaser approved these changes Feb 20, 2024

View reviewed changes

Alexandr-Solovev approved these changes Feb 21, 2024

View reviewed changes

napetrov merged commit a76316b into oneapi-src:main Feb 21, 2024
12 of 13 checks passed

This was referenced Feb 21, 2024

Makefile refactoring to factor out common build code #2672

Merged

Move makefiles into x86_64 target directory #2604

Closed

ajay-fuji deleted the enable-arm-build-with-ref-backend branch February 27, 2024 11:36

ajay-fuji restored the enable-arm-build-with-ref-backend branch February 27, 2024 11:36

chandan-fujitsu-research deleted the enable-arm-build-with-ref-backend branch February 29, 2024 04:45

chandan-fujitsu-research restored the enable-arm-build-with-ref-backend branch February 29, 2024 04:45

rakshithgb-fujitsu mentioned this pull request Mar 14, 2024

Enable Cross Compilation of Arm SVE on x86 Cl (Binary only) #2691

Merged

ajay-fuji mentioned this pull request Mar 22, 2024

Enable build on ARM(SVE) intel/scikit-learn-intelex#1771

Merged

	sve = 0, /!< ARM processors based on Arm's Scalable Vector Extension (SVE) /
	sve = 0, /!< ARM(R) processors based on Arm(R) Scalable Vector Extension (SVE) /

	sve = 2, /!< ARM processors based on Arm's Scalable Vector Extension (SVE) /
	sve = 2, /!< ARM(R) processors based on Arm(R) Scalable Vector Extension (SVE) /

	* Copyright 2023-24 FUJITSU LIMITED
	* Copyright 2024 FUJITSU LIMITED

		@@ -1,5 +1,6 @@
		#===============================================================================

Enable ARM(SVE) CPU support with reference backend #2614

Enable ARM(SVE) CPU support with reference backend #2614

Conversation

ajay-fuji commented Dec 15, 2023 • edited Loading

Proposed changes in this PR

keeranroth left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rakshithgb-fujitsu commented Dec 18, 2023 • edited Loading

napetrov commented Dec 18, 2023

amgrigoriev commented Dec 19, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Vika-F left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rageshhajela16 commented Feb 19, 2024 • edited Loading

Alexandr-Solovev commented Feb 19, 2024

Alexandr-Solovev commented Feb 19, 2024 • edited Loading

ajay-fuji commented Feb 19, 2024

Alexandr-Solovev commented Feb 20, 2024 • edited Loading

napetrov commented Feb 20, 2024

Alexandr-Solovev commented Feb 20, 2024 • edited Loading

napetrov commented Feb 20, 2024

Alexandr-Solovev commented Feb 20, 2024

Alexandr-Solovev commented Feb 20, 2024

napetrov left a comment

Choose a reason for hiding this comment

Alexandr-Solovev commented Feb 20, 2024

ethanglaser left a comment

Choose a reason for hiding this comment

Alexandr-Solovev left a comment

Choose a reason for hiding this comment

napetrov commented Feb 21, 2024

rageshhajela16 commented Feb 21, 2024 • edited Loading

ajay-fuji commented Dec 15, 2023 •

edited

Loading

rakshithgb-fujitsu commented Dec 18, 2023 •

edited

Loading

amgrigoriev commented Dec 19, 2023 •

edited

Loading

rageshhajela16 commented Feb 19, 2024 •

edited

Loading

Alexandr-Solovev commented Feb 19, 2024 •

edited

Loading

Alexandr-Solovev commented Feb 20, 2024 •

edited

Loading

Alexandr-Solovev commented Feb 20, 2024 •

edited

Loading

rageshhajela16 commented Feb 21, 2024 •

edited

Loading