[python-package] prevent build-python.sh from modifying already-installed dependencies #6403
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The recommended workflow for building the Python package from source here uses the
build-python.sh
script, like this:cmake -B build -S . cmake --build build --target _lightgbm -j4 sh ./build-python.sh install --precompile
The version for the library from any two unreleased commits is the same (e.g.
4.3.0.99
). To ensure thatpip
does not give up on successive runs and say "ohlightgbm==4.3.0.99
is already installed, I'm not installing this wheel", in #5907 we added--force-reinstall
to thepip install dist/*
inbuild-python.sh
.That has the unfortunate side-effect of also force-reinstalling all of
lightgbm
's strong runtime dependencies (numpy
andscipy
). That's undesirable for 2 reasons:numpy
version upgraded by installinglightgbm
, even whenlightgbm
is compatible with the version ofnumpy
you already haveconda
environment, this can result in anumpy
conda package being replaced by anumpy
wheel, which can cause a variety of linking and loading problemsThis PR proposes avoiding that, by instead having
build-python.sh install
runpip uninstall --yes lightgbm
.How I tested this
In a conda environment without
lightgbm
installed, but withscipy
andnumpy
installed, ran the following.cmake -B build -S . cmake --build build --target _lightgbm -j4 sh ./build-python.sh install --precompile
Saw this in the logs, confirming that it didn't change my
numpy
orscipy
versions.