[solution] Habitat_sim installs failing in Colab #1818

srkiyengar · 2022-07-19T00:28:20Z

Recently, habitat_sim colab installs seem to fail in "import habitat-sim" after successful installtion. It seems to be a complaining about missing library:
ImportError: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by /usr/local/lib/python3.7/dist-packages/scipy/optimize/_group_columns.cpython-37m-x86_64-linux-gnu.so)

The two colab habitat-sim install that I have tried are,

Error details while importing habitat-sim in ECCV_2020_Interactivity.ipynb

ImportError Traceback (most recent call last)
in ()
17 from PIL import Image
18
---> 19 import habitat_sim
20 from habitat_sim.utils import common as ut
21 from habitat_sim.utils import viz_utils as vut

11 frames
/usr/local/lib/python3.7/dist-packages/scipy/optimize/_numdiff.py in ()
6 from scipy.sparse.linalg import LinearOperator
7 from ..sparse import issparse, csc_matrix, csr_matrix, coo_matrix, find
----> 8 from ._group_columns import group_dense, group_sparse
9
10

ImportError: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by /usr/local/lib/python3.7/dist-packages/scipy/optimize/_group_columns.cpython-37m-x86_64-linux-gnu.so)

Error details of "import habitat-sim" after install from "https://raw.githubusercontent.com/facebookresearch/habitat-sim/master/examples/colab_utils/colab_install.sh | NIGHTLY=false bash -s"

/content/oreo-pybullet/agent_oreo.py in ()
----> 1 import habitat_sim
2 import numpy as np
3 import quaternion
4 import math
5 import cv2

/usr/local/lib/python3.7/dist-packages/habitat_sim/init.py in ()
11 if not getattr(builtins, "HSIM_SETUP", False):
12 # TODO: kept only for compatibiliy with existing code. Please gradually remove
---> 13 from habitat_sim import (
14 agent,
15 attributes,

/usr/local/lib/python3.7/dist-packages/habitat_sim/agent/init.py in ()
7 # type: ignore
8
----> 9 from .agent import * # noqa: F403
10 from .controls import * # noqa: F403
11

/usr/local/lib/python3.7/dist-packages/habitat_sim/agent/agent.py in ()
29 )
30
---> 31 from .controls import ActuationSpec, ObjectControls
32
33 all = ["ActionSpec", "SixDOFPose", "AgentState", "AgentConfiguration", "Agent"]

/usr/local/lib/python3.7/dist-packages/habitat_sim/agent/controls/init.py in ()
8 from habitat_sim.agent.controls.default_controls import * # noqa: F401, F403
9 from habitat_sim.agent.controls.object_controls import ObjectControls
---> 10 from habitat_sim.agent.controls.pyrobot_noisy_controls import PyRobotNoisyActuationSpec
11
12 all = [

/usr/local/lib/python3.7/dist-packages/habitat_sim/agent/controls/pyrobot_noisy_controls.py in ()
18 import magnum as mn
19 import numpy as np
---> 20 import scipy.stats
21 from attr import Attribute
22 from numpy import ndarray

/usr/local/lib/python3.7/dist-packages/scipy/stats/init.py in ()
439 """
440
--> 441 from .stats import *
442 from .distributions import *
443 from .morestats import *

/usr/local/lib/python3.7/dist-packages/scipy/stats/stats.py in ()
41 import scipy.special as special
42 from scipy import linalg
---> 43 from . import distributions
44 from . import mstats_basic
45 from ._stats_mstats_common import (_find_repeats, linregress, theilslopes,

/usr/local/lib/python3.7/dist-packages/scipy/stats/distributions.py in ()
6 # instead of git blame -Lxxx,+x.
7 #
----> 8 from ._distn_infrastructure import (rv_discrete, rv_continuous, rv_frozen)
9
10 from . import _continuous_distns

/usr/local/lib/python3.7/dist-packages/scipy/stats/_distn_infrastructure.py in ()
22 # for root finding for continuous distribution ppf, and max likelihood
23 # estimation
---> 24 from scipy import optimize
25
26 # for functions of continuous distributions (e.g. moments, entropy, cdf)

/usr/local/lib/python3.7/dist-packages/scipy/optimize/init.py in ()
398 """
399
--> 400 from .optimize import *
401 from ._minimize import *
402 from ._root import *

/usr/local/lib/python3.7/dist-packages/scipy/optimize/optimize.py in ()
34 line_search_wolfe2 as line_search,
35 LineSearchWarning)
---> 36 from ._numdiff import approx_derivative
37 from scipy._lib._util import getfullargspec_no_self as _getfullargspec
38 from scipy._lib._util import MapWrapper

/usr/local/lib/python3.7/dist-packages/scipy/optimize/_numdiff.py in ()
6 from scipy.sparse.linalg import LinearOperator
7 from ..sparse import issparse, csc_matrix, csr_matrix, coo_matrix, find
----> 8 from ._group_columns import group_dense, group_sparse
9
10

ImportError: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by /usr/local/lib/python3.7/dist-packages/scipy/optimize/_group_columns.cpython-37m-x86_64-linux-gnu.so)

Hopefully the details are sufficient to troubleshoot the colab environment working.The installs used to work nicely the last time I checked 3-4 weeks ago.
Thanks.

The text was updated successfully, but these errors were encountered:

dhruvbatra · 2022-07-19T01:04:59Z

@Skylion007 -- thoughts?

srkiyengar · 2022-07-22T03:12:21Z

Strings command of /usr/lib/x86_64-linux-gnu/libstdc++.so.6 for the colab install.

arjunmajum · 2022-07-22T21:57:19Z

Here's temporary fix that worked for me from the second solution here.

Run this one time:

!add-apt-repository ppa:ubuntu-toolchain-r/test -y
!apt-get update --fix-missing
!apt-get dist-upgrade

Then, restart the runtime -- or -- modify the imports to automatically restart:

try:
  import habitat_sim
except ImportError:
  exit()

srkiyengar · 2022-07-23T00:52:11Z

Thank you @arjunmajum.

Skylion007 · 2022-07-23T03:47:17Z

@Skylion007 -- thoughts?

Trying to find a long term fix here that doesn't require a Colab restart. I am really confused where the new LIBCXX is being installed / overwritten from.

srkiyengar · 2022-08-02T14:08:04Z

@arjunmajum's suggestion of restart worked well and was very usable. But as of last evening, executing the install script to install habitat-sim is running into another issue.

PackagesNotFoundError: The following packages are not available from current channels:

numba=0.56.0

Any suggestion or work around will be appreciated.

!curl -L https://raw.githubusercontent.com/facebookresearch/habitat-sim/master/examples/colab_utils/colab_install.sh | NIGHTLY=false bash -s

--2022-08-02 13:43:47-- https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
Resolving repo.continuum.io (repo.continuum.io)... 104.18.200.79, 104.18.201.79, 2606:4700::6812:c94f, ...
Connecting to repo.continuum.io (repo.continuum.io)|104.18.200.79|:443... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh [following]
--2022-08-02 13:43:47-- https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
Resolving repo.anaconda.com (repo.anaconda.com)... 104.16.130.3, 104.16.131.3, 2606:4700::6810:8203, ...
Connecting to repo.anaconda.com (repo.anaconda.com)|104.16.130.3|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 76607678 (73M) [application/x-sh]
Saving to: ‘Miniconda3-latest-Linux-x86_64.sh’

Miniconda3-latest-L 100%[===================>] 73.06M 234MB/s in 0.3s

2022-08-02 13:43:48 (234 MB/s) - ‘Miniconda3-latest-Linux-x86_64.sh’ saved [76607678/76607678]

PREFIX=/usr/local
Unpacking payload ...
Collecting package metadata (current_repodata.json): done
Solving environment: done

Package Plan

environment location: /usr/local

added / updated specs:
- _libgcc_mutex==0.1=main
- _openmp_mutex==4.5=1_gnu
- brotlipy==0.7.0=py39h27cfd23_1003
- ca-certificates==2022.3.29=h06a4308_1
- certifi==2021.10.8=py39h06a4308_2
- cffi==1.15.0=py39hd667e15_1
- charset-normalizer==2.0.4=pyhd3eb1b0_0
- colorama==0.4.4=pyhd3eb1b0_0
- conda-content-trust==0.1.1=pyhd3eb1b0_0
- conda-package-handling==1.8.1=py39h7f8727e_0
- conda==4.12.0=py39h06a4308_0
- cryptography==36.0.0=py39h9ce1e76_0
- idna==3.3=pyhd3eb1b0_0
- ld_impl_linux-64==2.35.1=h7274673_9
- libffi==3.3=he6710b0_2
- libgcc-ng==9.3.0=h5101ec6_17
- libgomp==9.3.0=h5101ec6_17
- libstdcxx-ng==9.3.0=hd4cf53a_17
- ncurses==6.3=h7f8727e_2
- openssl==1.1.1n=h7f8727e_0
- pip==21.2.4=py39h06a4308_0
- pycosat==0.6.3=py39h27cfd23_0
- pycparser==2.21=pyhd3eb1b0_0
- pyopenssl==22.0.0=pyhd3eb1b0_0
- pysocks==1.7.1=py39h06a4308_0
- python==3.9.12=h12debd9_0
- readline==8.1.2=h7f8727e_1
- requests==2.27.1=pyhd3eb1b0_0
- ruamel_yaml==0.15.100=py39h27cfd23_0
- setuptools==61.2.0=py39h06a4308_0
- six==1.16.0=pyhd3eb1b0_1
- sqlite==3.38.2=hc218d9a_0
- tk==8.6.11=h1ccaba5_0
- tqdm==4.63.0=pyhd3eb1b0_0
- tzdata==2022a=hda174b7_0
- urllib3==1.26.8=pyhd3eb1b0_0
- wheel==0.37.1=pyhd3eb1b0_0
- xz==5.2.5=h7b6447c_0
- yaml==0.2.5=h7b6447c_0
- zlib==1.2.12=h7f8727e_1

The following NEW packages will be INSTALLED:

_libgcc_mutex pkgs/main/linux-64::_libgcc_mutex-0.1-main
_openmp_mutex pkgs/main/linux-64::_openmp_mutex-4.5-1_gnu
brotlipy pkgs/main/linux-64::brotlipy-0.7.0-py39h27cfd23_1003
ca-certificates pkgs/main/linux-64::ca-certificates-2022.3.29-h06a4308_1
certifi pkgs/main/linux-64::certifi-2021.10.8-py39h06a4308_2
cffi pkgs/main/linux-64::cffi-1.15.0-py39hd667e15_1
charset-normalizer pkgs/main/noarch::charset-normalizer-2.0.4-pyhd3eb1b0_0
colorama pkgs/main/noarch::colorama-0.4.4-pyhd3eb1b0_0
conda pkgs/main/linux-64::conda-4.12.0-py39h06a4308_0
conda-content-tru~ pkgs/main/noarch::conda-content-trust-0.1.1-pyhd3eb1b0_0
conda-package-han~ pkgs/main/linux-64::conda-package-handling-1.8.1-py39h7f8727e_0
cryptography pkgs/main/linux-64::cryptography-36.0.0-py39h9ce1e76_0
idna pkgs/main/noarch::idna-3.3-pyhd3eb1b0_0
ld_impl_linux-64 pkgs/main/linux-64::ld_impl_linux-64-2.35.1-h7274673_9
libffi pkgs/main/linux-64::libffi-3.3-he6710b0_2
libgcc-ng pkgs/main/linux-64::libgcc-ng-9.3.0-h5101ec6_17
libgomp pkgs/main/linux-64::libgomp-9.3.0-h5101ec6_17
libstdcxx-ng pkgs/main/linux-64::libstdcxx-ng-9.3.0-hd4cf53a_17
ncurses pkgs/main/linux-64::ncurses-6.3-h7f8727e_2
openssl pkgs/main/linux-64::openssl-1.1.1n-h7f8727e_0
pip pkgs/main/linux-64::pip-21.2.4-py39h06a4308_0
pycosat pkgs/main/linux-64::pycosat-0.6.3-py39h27cfd23_0
pycparser pkgs/main/noarch::pycparser-2.21-pyhd3eb1b0_0
pyopenssl pkgs/main/noarch::pyopenssl-22.0.0-pyhd3eb1b0_0
pysocks pkgs/main/linux-64::pysocks-1.7.1-py39h06a4308_0
python pkgs/main/linux-64::python-3.9.12-h12debd9_0
readline pkgs/main/linux-64::readline-8.1.2-h7f8727e_1
requests pkgs/main/noarch::requests-2.27.1-pyhd3eb1b0_0
ruamel_yaml pkgs/main/linux-64::ruamel_yaml-0.15.100-py39h27cfd23_0
setuptools pkgs/main/linux-64::setuptools-61.2.0-py39h06a4308_0
six pkgs/main/noarch::six-1.16.0-pyhd3eb1b0_1
sqlite pkgs/main/linux-64::sqlite-3.38.2-hc218d9a_0
tk pkgs/main/linux-64::tk-8.6.11-h1ccaba5_0
tqdm pkgs/main/noarch::tqdm-4.63.0-pyhd3eb1b0_0
tzdata pkgs/main/noarch::tzdata-2022a-hda174b7_0
urllib3 pkgs/main/noarch::urllib3-1.26.8-pyhd3eb1b0_0
wheel pkgs/main/noarch::wheel-0.37.1-pyhd3eb1b0_0
xz pkgs/main/linux-64::xz-5.2.5-h7b6447c_0
yaml pkgs/main/linux-64::yaml-0.2.5-h7b6447c_0
zlib pkgs/main/linux-64::zlib-1.2.12-h7f8727e_1

Preparing transaction: done
Executing transaction: done
installation finished.
WARNING:
You currently have a PYTHONPATH environment variable set. This may cause
unexpected behavior when running the Python interpreter in Miniconda3.
For best results, please verify that your PYTHONPATH only points to
directories of packages that are compatible with the Python interpreter
in Miniconda3: /usr/local
Collecting package metadata (current_repodata.json): done
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed

PackagesNotFoundError: The following packages are not available from current channels:

numba=0.56.0

Current channels:

To search for alternate channels that may provide the conda package you're
looking for, navigate to

https://anaconda.org/

and use the search bar at the top of the page.

An error occured during the installation of Habitat-sim or Habitat-Lab.

pkachana3 · 2022-08-04T12:37:16Z

@srkiyengar Were you able to get this installation issue resolved? I am having the same problem as well.

srkiyengar · 2022-08-06T00:42:12Z

Not properly @pkachana3.
"conda search -f numba" stops at numba version 0.55.1 where as the colab default seems to be 0.56.0

I harded coded numba to 0.51.2. In the install script:

conda install -S -y --prefix /usr/local -c "${CHANNEL}" -c conda-forge habitat-sim=0.2.0 headless withbullet "python=${PYTHON_VERSION}" "numpy=${NUMPY_VERSION}" "pillow=${PIL_VERSION}" "cffi=${CFFI_VERSION}" "scipy=${SCIPY_VERSION}" "numba=0.51.2"

It seems to install the habitat.

aclegg3 · 2022-08-12T23:40:42Z

Thanks for the temporary solutions here, folks! With these I'm able to run the colabs successfully. @Skylion007 and I will work on a real fix.

For now, you can:

modify the install cell to use the installer on a branch with the change suggested by @srkiyengar:

https://raw.githubusercontent.com/facebookresearch/habitat-sim/colab-numba-version/examples/colab_utils/colab_install.sh

Edited path 10/11

apply the fix suggested by @arjunmajum in this comment:

Run this one time:

!add-apt-repository ppa:ubuntu-toolchain-r/test -y
!apt-get update --fix-missing
!apt-get dist-upgrade

Then, restart the runtime

rpartsey · 2022-12-15T16:44:14Z

@aclegg3 thank you for advice!
works for me

aclegg3 · 2024-01-16T17:29:20Z

Note we have stopped supporting colab and converted tutorials to local Jupyter notebooks instead.

aclegg3 added the bug Something isn't working label Aug 12, 2022

aclegg3 assigned Skylion007 and aclegg3 Aug 12, 2022

aclegg3 added the solution proposed A solution or workaround is proposed but may not be merged to main. label Sep 1, 2022

aclegg3 mentioned this issue Oct 11, 2022

Habitat_sim install script for Colab is failing - new error #1884

Closed

NakuraMino mentioned this issue Nov 2, 2022

Make_cfg refactoring in examples/tutorial/ #1924

Closed

11 tasks

aclegg3 pinned this issue Dec 14, 2022

aclegg3 changed the title ~~Habitat_sim installs failing in Colab~~ [solution] Habitat_sim installs failing in Colab Jan 11, 2023

This was referenced Jan 26, 2023

Failed to import habitat_sim following tutorial ECCV 2020: Navigation facebookresearch/habitat-lab#1089

Closed

Habitat.config error in Google Colab facebookresearch/habitat-lab#1074

Closed

aclegg3 mentioned this issue Mar 22, 2023

Replica-CAD QuickStart Collab Not Working #2025

Closed

aclegg3 added the wontfix This will not be worked on label Jan 16, 2024

aclegg3 closed this as completed Jan 16, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[solution] Habitat_sim installs failing in Colab #1818

[solution] Habitat_sim installs failing in Colab #1818

srkiyengar commented Jul 19, 2022

dhruvbatra commented Jul 19, 2022

srkiyengar commented Jul 22, 2022

arjunmajum commented Jul 22, 2022

srkiyengar commented Jul 23, 2022

Skylion007 commented Jul 23, 2022

srkiyengar commented Aug 2, 2022

pkachana3 commented Aug 4, 2022

srkiyengar commented Aug 6, 2022 •

edited

aclegg3 commented Aug 12, 2022 •

edited

rpartsey commented Dec 15, 2022

aclegg3 commented Jan 16, 2024

[solution] Habitat_sim installs failing in Colab #1818

[solution] Habitat_sim installs failing in Colab #1818

Comments

srkiyengar commented Jul 19, 2022

dhruvbatra commented Jul 19, 2022

srkiyengar commented Jul 22, 2022

arjunmajum commented Jul 22, 2022

srkiyengar commented Jul 23, 2022

Skylion007 commented Jul 23, 2022

srkiyengar commented Aug 2, 2022

Package Plan

pkachana3 commented Aug 4, 2022

srkiyengar commented Aug 6, 2022 • edited

aclegg3 commented Aug 12, 2022 • edited

rpartsey commented Dec 15, 2022

aclegg3 commented Jan 16, 2024

srkiyengar commented Aug 6, 2022 •

edited

aclegg3 commented Aug 12, 2022 •

edited