Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

conda info Performance regression in 23.7.3 on WSL (find_commands slowdown) #13033

Closed
otaithleigh opened this issue Aug 25, 2023 · 0 comments · Fixed by #13035
Closed

conda info Performance regression in 23.7.3 on WSL (find_commands slowdown) #13033

otaithleigh opened this issue Aug 25, 2023 · 0 comments · Fixed by #13035
Labels
duplicate::primary if an issue/PR has duplicates, this is the consolidated, primary issue/PR source::community catch-all for issues filed by community members tag::performance related to degraded performance type::feature request for a new feature or capability

Comments

@otaithleigh
Copy link
Contributor

otaithleigh commented Aug 25, 2023 •
edited by kenodegard

What happened?

#13006 causes conda info (and anything else that ends up calling find_commands()) to take an extremely long time to execute on WSL (30 seconds on my machine), since WSL inherits all of the Windows PATH environment variable. The slowness is caused by the repeated isfile calls; it probably ends up delegating to networking calls or something because of WSL weirdness? I do not see the same performance issue on plain Windows, so its not an issue with a long PATH.

Delaying the isfile check until after pattern matching succeeds speeds things up considerably (30s to 1s for me).

Debug

active environment : None
            shell level : 0
       user config file : /home/ptalley2/.condarc
 populated config files : /home/ptalley2/mambaforge/.condarc
                          /home/ptalley2/dotfiles/conda/.condarc
          conda version : 23.7.3
    conda-build version : 3.24.0
         python version : 3.10.12.final.0
       virtual packages : __archspec=1=x86_64
                          __glibc=2.35=0
                          __linux=5.15.90.1=0
                          __unix=0=0
       base environment : /home/ptalley2/mambaforge  (writable)
      conda av data dir : /home/ptalley2/mambaforge/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/numba/linux-64
                          https://conda.anaconda.org/numba/noarch
                          https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://conda.anaconda.org/otaithleigh/linux-64
                          https://conda.anaconda.org/otaithleigh/noarch
          package cache : /home/ptalley2/mambaforge/pkgs
                          /home/ptalley2/.conda/pkgs
       envs directories : /home/ptalley2/mambaforge/envs
                          /home/ptalley2/.conda/envs
               platform : linux-64
             user-agent : conda/23.7.3 requests/2.31.0 CPython/3.10.12 Linux/5.15.90.1-microsoft-standard-WSL2 ubuntu/22.04.2 glibc/2.35
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False

Conda Details

conda info 
active environment : base
    active env location : /home/ptalley2/mambaforge
            shell level : 1
       user config file : /home/ptalley2/.condarc
 populated config files : /home/ptalley2/mambaforge/.condarc
                          /home/ptalley2/dotfiles/conda/.condarc
          conda version : 23.7.3
    conda-build version : 3.24.0
         python version : 3.10.12.final.0
       virtual packages : __archspec=1=x86_64
                          __glibc=2.35=0
                          __linux=5.15.90.1=0
                          __unix=0=0
       base environment : /home/ptalley2/mambaforge  (writable)
      conda av data dir : /home/ptalley2/mambaforge/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/numba/linux-64
                          https://conda.anaconda.org/numba/noarch
                          https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://conda.anaconda.org/otaithleigh/linux-64
                          https://conda.anaconda.org/otaithleigh/noarch
          package cache : /home/ptalley2/mambaforge/pkgs
                          /home/ptalley2/.conda/pkgs
       envs directories : /home/ptalley2/mambaforge/envs
                          /home/ptalley2/.conda/envs
               platform : linux-64
             user-agent : conda/23.7.3 requests/2.31.0 CPython/3.10.12 Linux/5.15.90.1-microsoft-standard-WSL2 ubuntu/22.04.2 glibc/2.35
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False
conda config 
==> /home/ptalley2/mambaforge/.condarc <==
channels:
  - conda-forge

==> /home/ptalley2/dotfiles/conda/.condarc <==
auto_activate_base: False
channels:
  - numba
  - conda-forge
  - otaithleigh
anaconda_upload: False

==> cmd_line <==
debug: False
json: False
conda list 
# packages in environment at /home/ptalley2/mambaforge:
#
# Name                    Version                   Build  Channel
_ipython_minor_entry_point 8.7.0                hb6b4a82_0    conda-forge
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
anaconda-client           1.11.1             pyhd8ed1ab_0    conda-forge
anaconda-project          0.11.1             pyhd8ed1ab_0    conda-forge
anyio                     3.7.1              pyhd8ed1ab_0    conda-forge
asttokens                 2.2.1              pyhd8ed1ab_0    conda-forge
attrs                     23.1.0             pyh71513ae_1    conda-forge
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                pyhd8ed1ab_3    conda-forge
backports.functools_lru_cache 1.6.5              pyhd8ed1ab_0    conda-forge
beautifulsoup4            4.12.2             pyha770c72_0    conda-forge
bleach                    6.0.0              pyhd8ed1ab_0    conda-forge
boa                       0.14.0             pyhd8ed1ab_4    conda-forge
boltons                   23.0.0             pyhd8ed1ab_0    conda-forge
brotli                    1.0.9                h166bdaf_9    conda-forge
brotli-bin                1.0.9                h166bdaf_9    conda-forge
brotli-python             1.0.9           py310hd8f1fbe_9    conda-forge
brotlipy                  0.7.0           py310h5764c6d_1005    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.19.1               hd590300_0    conda-forge
ca-certificates           2023.7.22            hbcca054_0    conda-forge
certifi                   2023.7.22          pyhd8ed1ab_0    conda-forge
cffi                      1.15.1          py310h255011f_3    conda-forge
chardet                   5.2.0           py310hff52083_0    conda-forge
charset-normalizer        3.2.0              pyhd8ed1ab_0    conda-forge
clyent                    1.2.2                      py_1    conda-forge
cmarkgfm                  0.8.0           py310h5764c6d_2    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
comm                      0.1.4              pyhd8ed1ab_0    conda-forge
commonmark                0.9.1                      py_0    conda-forge
conda                     23.7.3          py310hff52083_0    conda-forge
conda-build               3.24.0          py310hff52083_1    conda-forge
conda-pack                0.7.1              pyhd8ed1ab_0    conda-forge
conda-package-handling    2.2.0              pyh38be061_0    conda-forge
conda-package-streaming   0.9.0              pyhd8ed1ab_0    conda-forge
cryptography              41.0.3          py310h75e40e8_0    conda-forge
dataclasses               0.8                pyhc8e2a94_3    conda-forge
dbus                      1.13.6               h5008d03_3    conda-forge
debugpy                   1.6.8           py310hc6cd4ac_0    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
defusedxml                0.7.1              pyhd8ed1ab_0    conda-forge
docutils                  0.20.1          py310hff52083_0    conda-forge
entrypoints               0.4                pyhd8ed1ab_0    conda-forge
exceptiongroup            1.1.3              pyhd8ed1ab_0    conda-forge
executing                 1.2.0              pyhd8ed1ab_0    conda-forge
expat                     2.5.0                hcb278e6_1    conda-forge
filelock                  3.12.2             pyhd8ed1ab_0    conda-forge
fmt                       9.1.0                h924138e_0    conda-forge
freetype                  2.12.1               hca18f0e_1    conda-forge
future                    0.18.3             pyhd8ed1ab_0    conda-forge
gettext                   0.21.1               h27087fc_0    conda-forge
glob2                     0.7                        py_0    conda-forge
icu                       72.1                 hcb278e6_0    conda-forge
idna                      3.4                pyhd8ed1ab_0    conda-forge
importlib-metadata        6.8.0              pyha770c72_0    conda-forge
importlib_metadata        6.8.0                hd8ed1ab_0    conda-forge
importlib_resources       6.0.1              pyhd8ed1ab_0    conda-forge
invoke                    2.2.0              pyhd8ed1ab_0    conda-forge
ipykernel                 6.25.1             pyh71e2992_0    conda-forge
ipython                   8.7.0              pyh41d4057_0    conda-forge
jaraco.classes            3.3.0              pyhd8ed1ab_0    conda-forge
jedi                      0.19.0             pyhd8ed1ab_0    conda-forge
jeepney                   0.8.0              pyhd8ed1ab_0    conda-forge
jinja2                    3.1.2              pyhd8ed1ab_1    conda-forge
joblib                    1.3.2              pyhd8ed1ab_0    conda-forge
jpeg                      9e                   h0b41bf4_3    conda-forge
json5                     0.9.14             pyhd8ed1ab_0    conda-forge
jsonpatch                 1.32               pyhd8ed1ab_0    conda-forge
jsonpointer               2.0                        py_0    conda-forge
jsonschema                4.19.0             pyhd8ed1ab_1    conda-forge
jsonschema-specifications 2023.7.1           pyhd8ed1ab_0    conda-forge
jupyter_client            8.3.0              pyhd8ed1ab_0    conda-forge
jupyter_core              5.3.1           py310hff52083_0    conda-forge
keyring                   24.2.0          py310hff52083_0    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
krb5                      1.21.2               h659d440_0    conda-forge
lcms2                     2.15                 hfd0df8a_0    conda-forge
ld_impl_linux-64          2.40                 h41732ed_0    conda-forge
lerc                      4.0.0                h27087fc_0    conda-forge
libarchive                3.6.2                h039dbb9_1    conda-forge
libbrotlicommon           1.0.9                h166bdaf_9    conda-forge
libbrotlidec              1.0.9                h166bdaf_9    conda-forge
libbrotlienc              1.0.9                h166bdaf_9    conda-forge
libcurl                   8.2.1                hca28451_0    conda-forge
libdeflate                1.17                 h0b41bf4_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libexpat                  2.5.0                hcb278e6_1    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-ng                 13.1.0               he5830b7_0    conda-forge
libglib                   2.76.4               hebfc3b9_0    conda-forge
libgomp                   13.1.0               he5830b7_0    conda-forge
libiconv                  1.17                 h166bdaf_0    conda-forge
liblief                   0.12.3               h27087fc_0    conda-forge
libmamba                  1.4.9                h658169a_0    conda-forge
libmambapy                1.4.9           py310h8aae740_0    conda-forge
libnghttp2                1.52.0               h61bc06f_0    conda-forge
libnsl                    2.0.0                h7f98852_0    conda-forge
libpng                    1.6.39               h753d276_0    conda-forge
libsodium                 1.0.18               h36c2ea0_1    conda-forge
libsolv                   0.7.24               hfc55251_1    conda-forge
libsqlite                 3.42.0               h2797004_0    conda-forge
libssh2                   1.11.0               h0841786_0    conda-forge
libstdcxx-ng              13.1.0               hfd8a6a1_0    conda-forge
libtiff                   4.5.0                h6adf6a1_2    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libwebp-base              1.3.1                hd590300_0    conda-forge
libxcb                    1.13              h7f98852_1004    conda-forge
libxml2                   2.11.5               h0d562d8_0    conda-forge
libzlib                   1.2.13               hd590300_5    conda-forge
line_profiler             4.1.0           py310hd41b1e2_0    conda-forge
lz4-c                     1.9.4                hcb278e6_0    conda-forge
lzo                       2.10              h516909a_1000    conda-forge
mamba                     1.4.9           py310h51d5547_0    conda-forge
markdown-it-py            3.0.0              pyhd8ed1ab_0    conda-forge
markupsafe                2.1.3           py310h2372a71_0    conda-forge
matplotlib-inline         0.1.6              pyhd8ed1ab_0    conda-forge
mdurl                     0.1.0              pyhd8ed1ab_0    conda-forge
more-itertools            10.1.0             pyhd8ed1ab_0    conda-forge
nbformat                  5.9.2              pyhd8ed1ab_0    conda-forge
ncurses                   6.4                  hcb278e6_0    conda-forge
nest-asyncio              1.5.6              pyhd8ed1ab_0    conda-forge
openjpeg                  2.5.0                hfec8fc6_2    conda-forge
openssl                   3.1.2                hd590300_0    conda-forge
packaging                 23.1               pyhd8ed1ab_0    conda-forge
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
patch                     2.7.6             h7f98852_1002    conda-forge
patchelf                  0.18.0               h59595ed_0    conda-forge
pcre                      8.45                 h9c3ff4c_0    conda-forge
pcre2                     10.40                hc3806b6_0    conda-forge
pep517                    0.13.0             pyhd8ed1ab_0    conda-forge
pexpect                   4.8.0              pyh9f0ad1d_2    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pillow                    9.4.0           py310h023d228_1    conda-forge
pip                       23.2.1             pyhd8ed1ab_0    conda-forge
pkginfo                   1.9.6              pyhd8ed1ab_0    conda-forge
pkgutil-resolve-name      1.3.10             pyhd8ed1ab_0    conda-forge
platformdirs              3.10.0             pyhd8ed1ab_0    conda-forge
pluggy                    1.2.0              pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.39             pyha770c72_0    conda-forge
prompt_toolkit            3.0.39               hd8ed1ab_0    conda-forge
psutil                    5.9.5           py310h1fa729e_0    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
py-lief                   0.12.3          py310hd8f1fbe_0    conda-forge
pybind11-abi              4                    hd8ed1ab_3    conda-forge
pycosat                   0.6.4           py310h5764c6d_1    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pygments                  2.16.1             pyhd8ed1ab_0    conda-forge
pyopenssl                 23.2.0             pyhd8ed1ab_1    conda-forge
pyparsing                 3.1.1              pyhd8ed1ab_0    conda-forge
pyproject_hooks           1.0.0              pyhd8ed1ab_0    conda-forge
pyrsistent                0.19.3          py310h1fa729e_0    conda-forge
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
python                    3.10.12         hd12c33a_0_cpython    conda-forge
python-build              0.10.0             pyhd8ed1ab_1    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-fastjsonschema     2.18.0             pyhd8ed1ab_0    conda-forge
python-libarchive-c       5.0             py310hff52083_1    conda-forge
python_abi                3.10                    3_cp310    conda-forge
pytz                      2023.3             pyhd8ed1ab_0    conda-forge
pyyaml                    6.0             py310h5764c6d_5    conda-forge
pyzmq                     25.1.1          py310h5bbb5d0_0    conda-forge
readline                  8.2                  h8228510_1    conda-forge
readme_renderer           40.0               pyhd8ed1ab_0    conda-forge
referencing               0.30.2             pyhd8ed1ab_0    conda-forge
reproc                    14.2.4               h0b41bf4_0    conda-forge
reproc-cpp                14.2.4               hcb278e6_0    conda-forge
requests                  2.31.0             pyhd8ed1ab_0    conda-forge
requests-toolbelt         1.0.0              pyhd8ed1ab_0    conda-forge
rfc3986                   2.0.0              pyhd8ed1ab_0    conda-forge
rich                      13.5.1             pyhd8ed1ab_0    conda-forge
ripgrep                   13.0.0               h2f28480_2    conda-forge
rpds-py                   0.9.2           py310hcb5633a_0    conda-forge
ruamel.yaml               0.17.32         py310h2372a71_0    conda-forge
ruamel.yaml.clib          0.2.7           py310h1fa729e_1    conda-forge
ruamel_yaml               0.15.80         py310h5764c6d_1008    conda-forge
secretstorage             3.3.3           py310hff52083_1    conda-forge
setuptools                68.0.0             pyhd8ed1ab_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
sniffio                   1.3.0              pyhd8ed1ab_0    conda-forge
soupsieve                 2.3.2.post1        pyhd8ed1ab_0    conda-forge
sqlite                    3.42.0               h2c6b66d_0    conda-forge
stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
tk                        8.6.12               h27826a3_0    conda-forge
toml                      0.10.2             pyhd8ed1ab_0    conda-forge
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
toolz                     0.12.0             pyhd8ed1ab_0    conda-forge
tornado                   6.3.3           py310h2372a71_0    conda-forge
tqdm                      4.66.1             pyhd8ed1ab_0    conda-forge
traitlets                 5.9.0              pyhd8ed1ab_0    conda-forge
twine                     4.0.2              pyhd8ed1ab_0    conda-forge
typing-extensions         4.7.1                hd8ed1ab_0    conda-forge
typing_extensions         4.7.1              pyha770c72_0    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
urllib3                   2.0.4              pyhd8ed1ab_0    conda-forge
watchgod                  0.8.2              pyhd8ed1ab_0    conda-forge
wcwidth                   0.2.6              pyhd8ed1ab_0    conda-forge
webencodings              0.5.1                      py_1    conda-forge
wheel                     0.41.1             pyhd8ed1ab_0    conda-forge
xorg-libxau               1.0.11               hd590300_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
yaml                      0.2.5                h7f98852_2    conda-forge
yaml-cpp                  0.7.0                h27087fc_2    conda-forge
yapf                      0.40.1             pyhd8ed1ab_0    conda-forge
zeromq                    4.3.4                h9c3ff4c_1    conda-forge
zipp                      3.16.2             pyhd8ed1ab_0    conda-forge
zlib                      1.2.13               hd590300_5    conda-forge
zstandard                 0.19.0          py310h1275a96_2    conda-forge
zstd                      1.5.2                hfc55251_7    conda-forge
@otaithleigh otaithleigh added tag::performance related to degraded performance type::feature request for a new feature or capability labels Aug 25, 2023
@otaithleigh otaithleigh mentioned this issue Aug 25, 2023
Merged
3 tasks
@travishathaway travishathaway added the source::community catch-all for issues filed by community members label Aug 26, 2023
This was referenced Sep 7, 2023
Closed
Merged
@kenodegard kenodegard added the duplicate::primary if an issue/PR has duplicates, this is the consolidated, primary issue/PR label Sep 8, 2023
@kenodegard kenodegard mentioned this issue Sep 11, 2023
Merged
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
duplicate::primary if an issue/PR has duplicates, this is the consolidated, primary issue/PR source::community catch-all for issues filed by community members tag::performance related to degraded performance type::feature request for a new feature or capability
Projects
🧭 Planning
Archived in project
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Speed up find_commands() on WSL
3 participants
@travishathaway @kenodegard @otaithleigh