DOC: Add instructions for building with ASAN #2473

david-cortes-intel · 2025-05-13T09:33:39Z

Description

This PR adds instructions for building sklearnex with ASAN (assuming a oneDAL build with it too) and running python with it.

PR should start as a draft, then move to ready for review state after CI is passed and all applicable checkboxes are closed.
This approach ensures that reviewers don't spend extra time asking for regular requirements.

You can remove a checkbox as not applicable only if it doesn't relate to this PR in any way.
For example, PR with docs update doesn't require checkboxes for performance while PR with any change in actual code should have checkboxes and justify how this code change is expected to affect performance (or justification should be self-evident).

Checklist to comply with before moving PR from draft:

PR completeness and readability

I have reviewed my changes thoroughly before submitting this pull request.
I have updated the documentation to reflect the changes or created a separate PR with update and provided its number in the description, if necessary.
Git commit message contains an appropriate signed-off-by string (see CONTRIBUTING.md for details).
I have added a respective label(s) to PR if I have a permission for that.
I have resolved any merge conflicts that might occur with the base branch.

Testing

I have run it locally and tested the changes extensively.
All CI jobs are green or I have provided justification why they aren't.

codecov · 2025-05-13T12:57:05Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Flag	Coverage Δ
azure	`?`
github	`71.62% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

see 41 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

napetrov · 2025-05-13T16:07:34Z

INSTALL.md

+Putting it all together, the earlier examples building the library in-place and executing a python file with it become as follows:
+```shell
+source <path to ASAN-enabled oneDAL env.sh>
+CC="icx -fsanitize=address -g" CXX="icpx -fsanitize=address -g" LDFLAGS="-static-libasan" python setup.py build_ext --inplace --force --abs-rpath


should it be ICX?

If you meant about the CXX variable, then: no, on linux the C++ compiler is invoked through icpx.

i mean that we would need icx only for gpu portion of scikit but CPU side of the things should be gcc

Yes, but then you'd be mixing a library compiled with ICX (oneDAL) expecting the clang runtime for ASAN, and a library compiled with GCC expecting the gnu runtime for ASAN.

It does work with swapped runtimes as far as I can tell, but I think the clang runtime has additional things that the gnu one doesn't, so it could potentially lead to incompatibility problems depending on which functionalities/symbols/etc. from ASAN get used.

INSTALL.md

ethanglaser

This looks comprehensive, thanks for creating. General question (since I am not super familiar) - is this something geared more towards users or contributors? INSTALL.md doesn't seem like the right location, maybe CONTRIBUTING.md or within docs/ ? Would welcome other opinions on this

INSTALL.md

Co-authored-by: ethanglaser <42726565+ethanglaser@users.noreply.github.com>

david-cortes-intel · 2025-05-16T05:33:22Z

This looks comprehensive, thanks for creating. General question (since I am not super familiar) - is this something geared more towards users or contributors? INSTALL.md doesn't seem like the right location, maybe CONTRIBUTING.md or within docs/ ? Would welcome other opinions on this

This is more geared towards contributors.

It is indeed the case that our docs have things mixed up throughout different places and would benefit from reordering things and deduplicating sections, but currently this is the only place having instructions for executing the setup.py file, and the instructions here are easier to read when put next to the instructions for building without ASAN.

ethanglaser · 2025-05-16T13:51:52Z

This looks comprehensive, thanks for creating. General question (since I am not super familiar) - is this something geared more towards users or contributors? INSTALL.md doesn't seem like the right location, maybe CONTRIBUTING.md or within docs/ ? Would welcome other opinions on this

This is more geared towards contributors.

It is indeed the case that our docs have things mixed up throughout different places and would benefit from reordering things and deduplicating sections, but currently this is the only place having instructions for executing the setup.py file, and the instructions here are easier to read when put next to the instructions for building without ASAN.

Makes sense, overall I think the content is ready for merge but would be interested if anyone else has thoughts on location of this info.

* add conversion from treelite * add dependency also for conda

…n#2455) * Update .pre-commit-config.yaml * Update .pre-commit-config.yaml * Update pyproject.toml * Update pyproject.toml * Update pyproject.toml * Update .pre-commit-config.yaml * Update .pre-commit-config.yaml * Update pyproject.toml * Update pyproject.toml * Update dispatcher.py * Update dispatcher.py * Update .pre-commit-config.yaml * Update pyproject.toml * Update pyproject.toml * Update incremental_basic_statistics.py * Update incremental_linear.py * Update incremental_linear.py * Update incremental_linear.py * Update _device_offload.py * Update incremental_basic_statistics.py * Update incremental_basic_statistics.py * Update basic_statistics.py * Update incremental_pca.py * Update incremental_linear_model.py * Update incremental_covariance.py * Update incremental_ridge.py * Update incremental_linear.py * Update incremental_ridge.py * make fixes * Update _config.py * Update _config.py * Update _config.py * Update _backend.py * Update _dpep_helpers.py * Update _dpep_helpers.py * Update kernel_functions.py * Update kernel_functions.py * Update kernel_functions.py * Update kernel_functions.py * Update kernel_functions.py * Update kernel_functions.py * Update svm.py * Update _data_conversion.py * Update _data_conversion.py * Update _data_conversion.py * Update __init__.py * Update __init__.py * Update __init__.py * Update _device_offload.py * Update _device_offload.py * Update covariance.py * Update incremental_covariance.py * Update incremental_covariance.py * Update incremental_linear_model.py * Update incremental_linear_model.py * Update incremental_linear_model.py * Update linear_model.py * Update linear_model.py * Update linear_model.py * Update linear_model.py * Update incremental_basic_statistics.py * Update kmeans.py * Update pca.py * Update basic_statistics.py * Update basic_statistics.py * Update basic_statistics.py * Update basic_statistics.py * Update incremental_pca.py * Update basic_statistics.py * Update incremental_pca.py * Update logistic_regression.py * Update incremental_pca.py * Update _config.py * Update _config.py * Update _device_offload.py * Update _device_offload.py * Update _device_offload.py * Update _device_offload.py * Update validation.py * Update _sycl_queue_manager.py * Update _sycl_queue_manager.py * Update _sycl_queue_manager.py * Update _sycl_queue_manager.py * Update _backend.py * Update _utils.py * Update _utils.py * Update _utils.py * Update _utils.py * Update _utils.py * Apply suggestions from code review Co-authored-by: david-cortes-intel <david.cortes@intel.com> * Update _utils.py * Update _device_offload.py * Apply suggestions from code review Co-authored-by: david-cortes-intel <david.cortes@intel.com> --------- Co-authored-by: david-cortes-intel <david.cortes@intel.com>

…utes to python oneDAL table class (uxlfoundation#2430) * initial save * save point * interim stop * check: * fix most problems * save point * formatting * clang-formatting and save point * add testing * address coverage for dlpack::free_capsule * formatting * add gc and static_cast * remove comment * add suggestion

* chore(deps): update dependency xgboost to v3.0.1 * Update requirements-test.txt --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: ethanglaser <42726565+ethanglaser@users.noreply.github.com>

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* clarify that SPMD is not available on windows builds * Update doc/sources/distributed-mode.rst Co-authored-by: yuejiaointel <108152493+yuejiaointel@users.noreply.github.com> --------- Co-authored-by: yuejiaointel <108152493+yuejiaointel@users.noreply.github.com>

* add conda-forge channel for dependencies, mention incompatibilities with anaconda * update commands also in .md file * suggest impi install from conda-forge * remove warning about conda-forge incompatibilities * Update INSTALL.md Co-authored-by: ethanglaser <42726565+ethanglaser@users.noreply.github.com> * Update doc/sources/distributed-mode.rst Co-authored-by: ethanglaser <42726565+ethanglaser@users.noreply.github.com> * Update doc/sources/distributed-mode.rst Co-authored-by: ethanglaser <42726565+ethanglaser@users.noreply.github.com> --------- Co-authored-by: ethanglaser <42726565+ethanglaser@users.noreply.github.com>

…ion#2478) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Migrate to modern NumPy interface Signed-off-by: Emmanuel Ferdman <emmanuelferdman@gmail.com> * Update onedal/utils/validation.py Co-authored-by: david-cortes-intel <david.cortes@intel.com> * Update sklearnex/svm/_common.py Co-authored-by: david-cortes-intel <david.cortes@intel.com> --------- Signed-off-by: Emmanuel Ferdman <emmanuelferdman@gmail.com> Co-authored-by: david-cortes-intel <david.cortes@intel.com>

…2484)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* chore(deps): update dependency numpy to v2.2.6 * Update dependencies-dev * Update dependencies-dev --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: ethanglaser <42726565+ethanglaser@users.noreply.github.com>

…tion#2477) * chore(deps): update dependency array-api-compat to v1.12.0 * Update requirements-test.txt --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: ethanglaser <42726565+ethanglaser@users.noreply.github.com>

* chore(deps): update dependency xgboost to v3.0.2 * Update requirements-test.txt --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: ethanglaser <42726565+ethanglaser@users.noreply.github.com>

* fix: use upload github action to public site * fix: allow tag trigger with also year.month * test: test tag trigger * test: test tag trigger * .github/workflows/docs-release.yml * fix: don't copy gh-pages everytime, only copy parts don't exist * fix:move prepare script to doc_release.sh * fix:test checkout * fix:test upload pages artifact only has v3 * fix: fetch gh-apges if we have one * fix: use eaceiris/actions-gh-pages@v3 to keep old files when upload new ones * fix: use a intermediate unprotected branch for all doc archives * fix: fix archive * fix: fix archive * fix: remove doc from switcher url in conf and archive enture folder * fix: remove +x changes in doc relase before archive step * fix: move permission and use exact commit id for github actions * fix: use env var for _site folder and use doc folders for archive branch * fix: delete everything for new doc arvhice branch * fix: remove untracked files * fix: only add verison folders to git first time doc archive * fix: move TEMP_DOC_FOLDER to yml and make sure it is im tmp * fix: move env to jobs * fix: still need git rm for firs ttime doc archive creation * fix: remove doc * fix: allow manual trigger * fix: workflow dispatch add input requriement * fix: don't force input on workflow dispatch * test: test older python version doc build warning * fix: force doc version input * fix: remove unused dependency --------- Co-authored-by: yuejiaointel <41898282+github-actions[bot]@users.noreply.github.com>

)

…2493) * Update conda-recipe-lnx.yml * Update conda-recipe-win.yml * Update conda-recipe-lnx.yml * Update conda-recipe-win.yml * Update test_model_builders.py * Update test_model_builders.py * Delete tests/test_model_builders.py * Revert "Delete tests/test_model_builders.py" This reverts commit 453dd88. * disable catboost and check * Update test_model_builders.py * Update test_model_builders.py * Update test_model_builders.py * Update test_model_builders.py * Update test_model_builders.py * Update test_model_builders.py * Update test_model_builders.py * Update test_model_builders.py * Update test_model_builders.py

…en CI (uxlfoundation#2500) * Update deselected_tests.yaml * Update deselected_tests.yaml Co-authored-by: ethanglaser <42726565+ethanglaser@users.noreply.github.com> --------- Co-authored-by: ethanglaser <42726565+ethanglaser@users.noreply.github.com>

* create get_policy * formatting * Update policy.cpp

icfaust

I am very thankful for you writing this, as I plan on using this soon. Just some small nitpicks.

INSTALL.md

Co-authored-by: Ian Faust <icfaust@gmail.com>

icfaust

Looks good to go with these last swaps. Sorry about the delay.

INSTALL.md

Co-authored-by: Ian Faust <icfaust@gmail.com>

add instructions for building with ASAN

cd3e1ab

david-cortes-intel requested review from maria-Petrova, napetrov, Alexsandruss and icfaust as code owners May 13, 2025 09:33

david-cortes-intel added documentation infra labels May 13, 2025

david-cortes-intel requested review from Vika-F and Alexandr-Solovev May 13, 2025 09:33

napetrov reviewed May 13, 2025

View reviewed changes

yuejiaointel reviewed May 15, 2025

View reviewed changes

INSTALL.md Outdated Show resolved Hide resolved

remove unneeded variable

c816084

ethanglaser reviewed May 15, 2025

View reviewed changes

INSTALL.md Outdated Show resolved Hide resolved

Update INSTALL.md

f6911cc

Co-authored-by: ethanglaser <42726565+ethanglaser@users.noreply.github.com>

david-cortes-intel and others added 13 commits June 4, 2025 07:32

ENH: Enable model conversion from TreeLite (uxlfoundation#2452)

1ec21ca

* add conversion from treelite * add dependency also for conda

chore(deps): update dependency cython to v3.1.0 (uxlfoundation#2470)

b60e4f6

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

chore(deps): update dependency cmake to v4.0.2 (uxlfoundation#2468)

58329ce

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

chore(deps): update dependency tornado to v6.5 [security] (uxlfoundat…

04f7762

…ion#2478) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

fix: add cstdint include to transceiver for public CI (uxlfoundation#…

8c8f692

…2484)

chore(deps): update dependency cython to v3.1.1 (uxlfoundation#2483)

2d45c60

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

renovate bot and others added 9 commits June 4, 2025 07:32

downgrade setuptools to v79 for develop mode setup (uxlfoundation#2480)

86f87c7

ci(workflows): add OpenSSF Scorecard GitHub Action (uxlfoundation#2499

f744a60

)

[enhancement] unify policy creation in pybind11 (uxlfoundation#2390)

6cb98d7

* create get_policy * formatting * Update policy.cpp

Merge branch 'main' into asan_instr

4dde324

icfaust reviewed Jun 4, 2025

View reviewed changes

INSTALL.md Outdated Show resolved Hide resolved

INSTALL.md Outdated Show resolved Hide resolved

INSTALL.md Outdated Show resolved Hide resolved

INSTALL.md Outdated Show resolved Hide resolved

INSTALL.md Outdated Show resolved Hide resolved

david-cortes-intel and others added 5 commits June 4, 2025 14:24

Update INSTALL.md

30dd68f

Co-authored-by: Ian Faust <icfaust@gmail.com>

Update INSTALL.md

b0fa39b

Co-authored-by: Ian Faust <icfaust@gmail.com>

Update INSTALL.md

e91d686

Co-authored-by: Ian Faust <icfaust@gmail.com>

Update INSTALL.md

8c79933

Co-authored-by: Ian Faust <icfaust@gmail.com>

Update INSTALL.md

87786f9

Co-authored-by: Ian Faust <icfaust@gmail.com>

icfaust approved these changes Jun 5, 2025

View reviewed changes

INSTALL.md Outdated Show resolved Hide resolved

Update INSTALL.md

f3e07a4

Co-authored-by: Ian Faust <icfaust@gmail.com>

david-cortes-intel merged commit be4777e into uxlfoundation:main Jun 5, 2025
7 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

DOC: Add instructions for building with ASAN #2473

DOC: Add instructions for building with ASAN #2473

Uh oh!

david-cortes-intel commented May 13, 2025

Uh oh!

codecov bot commented May 13, 2025 •

edited

Loading

Uh oh!

napetrov May 13, 2025

Uh oh!

david-cortes-intel May 14, 2025

Uh oh!

napetrov May 14, 2025

Uh oh!

david-cortes-intel May 15, 2025

Uh oh!

Uh oh!

ethanglaser left a comment

Uh oh!

Uh oh!

david-cortes-intel commented May 16, 2025

Uh oh!

ethanglaser commented May 16, 2025

Uh oh!

icfaust left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

icfaust left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

DOC: Add instructions for building with ASAN #2473

DOC: Add instructions for building with ASAN #2473

Uh oh!

Conversation

david-cortes-intel commented May 13, 2025

Description

Uh oh!

codecov bot commented May 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

napetrov May 13, 2025

Choose a reason for hiding this comment

Uh oh!

david-cortes-intel May 14, 2025

Choose a reason for hiding this comment

Uh oh!

napetrov May 14, 2025

Choose a reason for hiding this comment

Uh oh!

david-cortes-intel May 15, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ethanglaser left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

david-cortes-intel commented May 16, 2025

Uh oh!

ethanglaser commented May 16, 2025

Uh oh!

icfaust left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

icfaust left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov bot commented May 13, 2025 •

edited

Loading