title | titleSuffix | description | author | ms.author | ms.topic | ms.service | ms.subservice | ms.date | ms.custom |
---|---|---|---|---|---|---|---|---|---|
Known issue - Slowness in compute instance terminal from a mounted path |
Azure Machine Learning |
While using the compute instance terminal inside a mounted path of data folder, any commands executed from the terminal result in slowness. |
s-polly |
scottpolly |
troubleshooting |
machine-learning |
core |
08/04/2023 |
known-issue |
[!INCLUDE dev v2]
While using the compute instance terminal inside a mounted path of a data folder, any commands executed from the terminal result in slowness. This issue is restricted to the terminal; running the commands from SDK using a notebook works as expected.
Status: Open
Problem area: Compute
While using the compute instance terminal inside a mounted path of a data folder, any commands executed from the terminal result in slowness. This issue is restricted to the terminal; running the commands from SDK using a notebook works as expected.
The LD_LIBRARY_PATH
contains an empty string by default, which is treated as the current directory. This causes many library lookups on remote storage, resulting in slowness.
As an example:
LD_LIBRARY_PATH /opt/intel/compilers_and_libraries_2018.3.222/linux/mpi/intel64/lib:/opt/intel/compilers_and_libraries_2018.3.222/linux/mpi/mic/lib::/anaconda/envs/azureml_py38/lib/:/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64/
Notice the ::
in the path. This is the empty string, which is treated as the current directory.
When one of the paths in a list is "" - every executable tries to find the dynamic libraries it needs relative to current working directory.
On the CI set the path making sure that LD_LIBRARY_PATH
doesn't contain an empty string.
export LD_LIBRARY_PATH="$(echo $LD_LIBRARY_PATH | sed 's/\(:\)\1\+/\1/g')"