Implement Logistic Regression algorithm (#2530)

* Initial commit * Add tests * Add cg_solver primitive to solve equation Ax = b * Move newton_cg primitve to optimizators primitive * Define newton_cg optimization function * Add backtracking algorithm for optimal alpha, implement newton_cg solver * Fix errors, add tests for newton-cg * Remove redundant wait_and_throw, add links to sources * Ensure code stability and fix minor issues - Add control over the number of iterations in while loops - Use l2-norm for convergence checks in cg-solver - Move QuadraticFunction to primitives section * Add sycl::fill, sycl::fabs and add specifiers for virtual functions * Remove redundant package dependency, update default values for Float parameters * Change update_x return type to event_vector, rename test function and minor fix * Initial commit * Split logloss and derivative functions, decrease the number of parameters * Delete redundant compute functions, deselect tests * Add LogLossFunction class and cover it with tests * Fix bugs, rename kernels and remove redundant, update perforamnce tests * Add wait and throw after gemv events * Minor * Fix error and add batch test * Add const qualifier for table with data * Minor * Add LogReg test to newton_cg optimizer * Add LogReg algorithm interface * Add BUILD files, vanila implementation and tmp test * Add test * Minor * Update train kernel * Add GPU infer kernels to LogReg algo * Add logistic regression dpc example * Fix GPU test failures: PCA (#2531) * Fix PCA tests * Apply clang-format * Support building reference backend with clang (#2529) * Add correct compile flags for clang reference build The clang makefile doesn't have the right includes defined for the reference backend implementation. Update this to match the defines for GCC. * Use right include directory for TBB The makefile doesn't use the most up-to-date variable for finding TBB libraries. This meant that a user-installed TBB, specified through TBBROOT, was not picked up during the build process. Now, if there is no TBB in the __deps folder, it is picked up from TBBROOT. * oneDAL development env docker image (#2513) * removing JDK deps * Fixing env and introduce docker * Creating bazel env script * adding readme for docker * updating * adding github validation * new line * remove bash invocation for scripts * fixing ci.yml * add execute permissions * fixing ci.yml * fixing github flow name * adding sudo to docker image * fixing install dpcpp script * fixing base docker deps * fixing bazelisk * fixing ci.yml * fixing windows ci with bash call * fixing bazel ci * updating bazel readme to current state * fixing bazel caching and nuget windows build * adding git binary * adding build validation and conda to env * enable larger validation in CI * fixing CLI * adding conda to docker * fixing docker * fixing docker * adding deps and build * adding docs env and more tests for nightly * not passing param * switching to conda env file * adding conda env * fixign path to env.yml * drop caching version to 1 as bazel version was fixed * bump version for spxinks-prompt to 1.8 * revert version * disabling doc reqs * switchign to latest tag * actually running code in docker * merging docker commands in one * merging docker commands in one * merging docker commands in one * installing bazel in docker image * adding build with OpenBLAS to validation * removing build with OpenBLAS to validation * Apply suggestions from code review Co-authored-by: Alexandra <alexandra.epanchinzeva@intel.com> * Apply suggestions from code review Co-authored-by: Alexander Andreev <alexander.andreev@intel.com> --------- Co-authored-by: Alexandra <alexandra.epanchinzeva@intel.com> Co-authored-by: Alexander Andreev <alexander.andreev@intel.com> * chore(deps): update actions/checkout digest to 8ade135 (#2521) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> * Update version to 2024.0.1 (#2533) * Online interfaces for LOM (#2483) * chore(deps): update dependency urllib3 to v2.0.6 [security] (#2536) * Minor * Change task name to binary classification and update an example * Add template optimizer parameter to descriptor, add optimizer factory, change method to dense_batch * Change unimplemented error name thrown in cpu kernel * Move data to gpu before training * Minor * Change dependency graph to address error, fix tests * Add optimizer descriptor as a template parameter * Update interface, improve primitive stability -Add l1_coef, class_count getters/setters -Throw exception if l1_coef != 0.0 or class_count != 2 -Update example -Add wait_and_throw for gemv events in logloss primitive to avoid sporadic errors * Move the optimizer dispatching logic to minimize function * Minor * Remove debug output, change optimizer_impl minimize interface * Remove redundant header and fix example * Remove redundatn imports * Update makefile, remove dependency from rng_primitive * Clang format * Update makefile * Add iterations number parameter * Update devices list for logreg example * Change fp type to float * Address comments * Adress changes, change iterations_number to iterations_count * Substitute l1, l2 coefs to C - inverse regularization in the interface * Remove class_count from descriptor constructor * Fix typo in result option naming * Add skipif statement to ignore cpu testing * Remove redundant includes --------- Co-authored-by: Dmitriy Ovchinnikov <117165790+inteldimitrius@users.noreply.github.com> Co-authored-by: Keeran Rothenfusser <141222236+keeranroth@users.noreply.github.com> Co-authored-by: Nikolay Petrov <nikolay.a.petrov@intel.com> Co-authored-by: Alexandra <alexandra.epanchinzeva@intel.com> Co-authored-by: Alexander Andreev <alexander.andreev@intel.com> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Aleksandr Solovev <aleksandr.solovev@intel.com>
oneapi-src · Nov 8, 2023 · 286f2f7 · 286f2f7
1 parent e244529
commit 286f2f7
Show file tree

Hide file tree

Showing 67 changed files with 3,739 additions and 173 deletions.
diff --git a/cpp/oneapi/dal/algo/BUILD b/cpp/oneapi/dal/algo/BUILD
@@ -23,6 +23,7 @@ ALGOS = [
     "kmeans_init",
     "knn",
     "linear_kernel",
+    "logistic_regression",
     "logloss_objective",
     "louvain",
     "minkowski_distance",

diff --git a/cpp/oneapi/dal/algo/logistic_regression.hpp b/cpp/oneapi/dal/algo/logistic_regression.hpp
@@ -0,0 +1,20 @@
+/*******************************************************************************
+* Copyright 2023 Intel Corporation
+*
+* Licensed under the Apache License, Version 2.0 (the "License");
+* you may not use this file except in compliance with the License.
+* You may obtain a copy of the License at
+*
+*     http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*******************************************************************************/
+
+#pragma once
+
+#include "oneapi/dal/algo/logistic_regression/infer.hpp"
+#include "oneapi/dal/algo/logistic_regression/train.hpp"
diff --git a/cpp/oneapi/dal/algo/logistic_regression/BUILD b/cpp/oneapi/dal/algo/logistic_regression/BUILD
@@ -0,0 +1,61 @@
+package(default_visibility = ["//visibility:public"])
+load("@onedal//dev/bazel:dal.bzl",
+    "dal_module",
+    "dal_test_suite",
+)
+
+dal_module(
+    name = "core",
+    auto = True,
+    dal_deps = [
+        "@onedal//cpp/oneapi/dal:core",
+        "@onedal//cpp/oneapi/dal/algo/logistic_regression/backend:model_impl",
+        "@onedal//cpp/oneapi/dal/algo/logistic_regression/detail:optimizers",
+    ],
+)
+
+dal_module(
+    name = "parameters",
+    dal_deps = [
+        "@onedal//cpp/oneapi/dal/algo/logistic_regression/parameters",
+    ],
+)
+
+dal_module(
+    name = "logistic_regression",
+    dal_deps = [
+        ":core",
+        ":parameters",
+        "@onedal//cpp/oneapi/dal/algo/logistic_regression/detail",
+        "@onedal//cpp/oneapi/dal/algo/logistic_regression/backend",
+
+    ]
+)
+
+dal_test_suite(
+    name = "interface_tests",
+    framework = "catch2",
+    compile_as = [ "dpc++" ],
+    hdrs = glob([
+        "test/*.hpp",
+    ], exclude=[
+        "test/mpi_*.hpp",
+        "test/ccl_*.hpp"
+    ]),
+    srcs = glob([
+        "test/*.cpp",
+    ], exclude=[
+        "test/mpi_*.cpp",
+        "test/ccl_*.cpp"
+    ]),
+    dal_deps = [
+        ":logistic_regression",
+    ],
+)
+
+dal_test_suite(
+    name = "tests",
+    tests = [
+        ":interface_tests",
+    ],
+)
diff --git a/cpp/oneapi/dal/algo/logistic_regression/backend/BUILD b/cpp/oneapi/dal/algo/logistic_regression/backend/BUILD
@@ -0,0 +1,23 @@
+package(default_visibility = ["//visibility:public"])
+load("@onedal//dev/bazel:dal.bzl",
+    "dal_module",
+    "dal_test_suite",
+)
+
+dal_module(
+    name = "backend",
+    dal_deps = [
+        "@onedal//cpp/oneapi/dal/algo/logistic_regression/backend/cpu",
+        "@onedal//cpp/oneapi/dal/algo/logistic_regression/backend/gpu",
+    ],
+)
+
+dal_module(
+    name = "model_impl",
+    hdrs = glob(["model_impl.hpp"]),
+)
+
+dal_module(
+    name = "optimizer_impl",
+    hdrs = glob(["optimizer_impl.hpp"]),
+)
diff --git a/cpp/oneapi/dal/algo/logistic_regression/backend/cpu/BUILD b/cpp/oneapi/dal/algo/logistic_regression/backend/cpu/BUILD
@@ -0,0 +1,14 @@
+package(default_visibility = ["//visibility:public"])
+load("@onedal//dev/bazel:dal.bzl",
+    "dal_module",
+    "dal_test_suite",
+)
+
+dal_module(
+    name = "cpu",
+    auto = True,
+    dal_deps = [
+        "@onedal//cpp/oneapi/dal/algo/logistic_regression:core",
+        "@onedal//cpp/oneapi/dal/backend/primitives:common",
+    ],
+)
diff --git a/cpp/oneapi/dal/algo/logistic_regression/backend/cpu/infer_kernel.hpp b/cpp/oneapi/dal/algo/logistic_regression/backend/cpu/infer_kernel.hpp
@@ -0,0 +1,31 @@
+/*******************************************************************************
+* Copyright 2023 Intel Corporation
+*
+* Licensed under the Apache License, Version 2.0 (the "License");
+* you may not use this file except in compliance with the License.
+* You may obtain a copy of the License at
+*
+*     http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*******************************************************************************/
+
+#pragma once
+
+#include "oneapi/dal/algo/logistic_regression/infer_types.hpp"
+#include "oneapi/dal/backend/dispatcher.hpp"
+
+namespace oneapi::dal::logistic_regression::backend {
+
+template <typename Float, typename Method, typename Task>
+struct infer_kernel_cpu {
+    infer_result<Task> operator()(const dal::backend::context_cpu& ctx,
+                                  const detail::descriptor_base<Task>& params,
+                                  const infer_input<Task>& input) const;
+};
+
+} // namespace oneapi::dal::logistic_regression::backend
diff --git a/cpp/oneapi/dal/algo/logistic_regression/backend/cpu/infer_kernel_dense_batch.cpp b/cpp/oneapi/dal/algo/logistic_regression/backend/cpu/infer_kernel_dense_batch.cpp
@@ -0,0 +1,40 @@
+/*******************************************************************************
+* Copyright 2023 Intel Corporation
+*
+* Licensed under the Apache License, Version 2.0 (the "License");
+* you may not use this file except in compliance with the License.
+* You may obtain a copy of the License at
+*
+*     http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*******************************************************************************/
+
+#include "oneapi/dal/backend/interop/common.hpp"
+
+#include "oneapi/dal/algo/logistic_regression/common.hpp"
+#include "oneapi/dal/algo/logistic_regression/infer_types.hpp"
+#include "oneapi/dal/algo/logistic_regression/backend/cpu/infer_kernel.hpp"
+
+namespace oneapi::dal::logistic_regression::backend {
+
+using dal::backend::context_cpu;
+
+template <typename Float, typename Task>
+struct infer_kernel_cpu<Float, method::dense_batch, Task> {
+    infer_result<Task> operator()(const context_cpu& ctx,
+                                  const detail::descriptor_base<Task>& desc,
+                                  const infer_input<Task>& input) const {
+        throw unimplemented(
+            dal::detail::error_messages::log_reg_dense_batch_method_is_not_implemented_for_cpu());
+    }
+};
+
+template struct infer_kernel_cpu<float, method::dense_batch, task::classification>;
+template struct infer_kernel_cpu<double, method::dense_batch, task::classification>;
+
+} // namespace oneapi::dal::logistic_regression::backend
diff --git a/cpp/oneapi/dal/algo/logistic_regression/backend/cpu/train_kernel.hpp b/cpp/oneapi/dal/algo/logistic_regression/backend/cpu/train_kernel.hpp
@@ -0,0 +1,32 @@
+/*******************************************************************************
+* Copyright 2023 Intel Corporation
+*
+* Licensed under the Apache License, Version 2.0 (the "License");
+* you may not use this file except in compliance with the License.
+* You may obtain a copy of the License at
+*
+*     http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*******************************************************************************/
+
+#pragma once
+
+#include "oneapi/dal/algo/logistic_regression/train_types.hpp"
+#include "oneapi/dal/backend/dispatcher.hpp"
+
+namespace oneapi::dal::logistic_regression::backend {
+
+template <typename Float, typename Method, typename Task>
+struct train_kernel_cpu {
+    train_result<Task> operator()(const dal::backend::context_cpu& ctx,
+                                  const detail::descriptor_base<Task>& desc,
+                                  const detail::train_parameters<Task>& params,
+                                  const train_input<Task>& input) const;
+};
+
+} // namespace oneapi::dal::logistic_regression::backend
diff --git a/cpp/oneapi/dal/algo/logistic_regression/backend/cpu/train_kernel_dense_batch.cpp b/cpp/oneapi/dal/algo/logistic_regression/backend/cpu/train_kernel_dense_batch.cpp
@@ -0,0 +1,38 @@
+/*******************************************************************************
+* Copyright 2023 Intel Corporation
+*
+* Licensed under the Apache License, Version 2.0 (the "License");
+* you may not use this file except in compliance with the License.
+* You may obtain a copy of the License at
+*
+*     http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*******************************************************************************/
+
+#include "oneapi/dal/algo/logistic_regression/backend/cpu/train_kernel.hpp"
+#include "oneapi/dal/exceptions.hpp"
+
+namespace oneapi::dal::logistic_regression::backend {
+
+using dal::backend::context_cpu;
+
+template <typename Float, typename Task>
+struct train_kernel_cpu<Float, method::dense_batch, Task> {
+    train_result<Task> operator()(const context_cpu& ctx,
+                                  const detail::descriptor_base<Task>& desc,
+                                  const detail::train_parameters<Task>& params,
+                                  const train_input<Task>& input) const {
+        throw unimplemented(
+            dal::detail::error_messages::log_reg_dense_batch_method_is_not_implemented_for_cpu());
+    }
+};
+
+template struct train_kernel_cpu<float, method::dense_batch, task::classification>;
+template struct train_kernel_cpu<double, method::dense_batch, task::classification>;
+
+} // namespace oneapi::dal::logistic_regression::backend
diff --git a/cpp/oneapi/dal/algo/logistic_regression/backend/gpu/BUILD b/cpp/oneapi/dal/algo/logistic_regression/backend/gpu/BUILD
@@ -0,0 +1,18 @@
+package(default_visibility = ["//visibility:public"])
+load("@onedal//dev/bazel:dal.bzl",
+    "dal_module",
+    "dal_test_suite",
+)
+
+dal_module(
+    name = "gpu",
+    auto = True,
+    dal_deps = [
+        "@onedal//cpp/oneapi/dal/backend/primitives:blas",
+        "@onedal//cpp/oneapi/dal/backend/primitives:common",
+        "@onedal//cpp/oneapi/dal/backend/primitives:lapack",
+        "@onedal//cpp/oneapi/dal/backend/primitives:objective_function",
+        "@onedal//cpp/oneapi/dal/backend/primitives:optimizers",
+        "@onedal//cpp/oneapi/dal/algo/logistic_regression:core",
+    ],
+)
diff --git a/cpp/oneapi/dal/algo/logistic_regression/backend/gpu/infer_kernel.hpp b/cpp/oneapi/dal/algo/logistic_regression/backend/gpu/infer_kernel.hpp
@@ -0,0 +1,31 @@
+/*******************************************************************************
+* Copyright 2023 Intel Corporation
+*
+* Licensed under the Apache License, Version 2.0 (the "License");
+* you may not use this file except in compliance with the License.
+* You may obtain a copy of the License at
+*
+*     http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*******************************************************************************/
+
+#pragma once
+
+#include "oneapi/dal/algo/logistic_regression/infer_types.hpp"
+#include "oneapi/dal/backend/dispatcher.hpp"
+
+namespace oneapi::dal::logistic_regression::backend {
+
+template <typename Float, typename Method, typename Task>
+struct infer_kernel_gpu {
+    infer_result<Task> operator()(const dal::backend::context_gpu& ctx,
+                                  const detail::descriptor_base<Task>& params,
+                                  const infer_input<Task>& input) const;
+};
+
+} // namespace oneapi::dal::logistic_regression::backend