Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI setup that runs tests e2e testing using GPU machines #63

Merged
merged 1 commit into from
Jun 1, 2019

Conversation

mathfac
Copy link
Contributor

@mathfac mathfac commented May 31, 2019

Motivation and Context

CI setup that runs tests using GPU machines:

  • Builds and installs habitat-sim with python setup.py install and ./build.sh --headless
  • Runs pytest tests on habitat-sim with testing data
  • Added speed regression test with FPS threshold (900 FPS) , thanks @erikwijmans for advice how to do it in elegant way.
  • Added run of habitat-api tests on master branch. That will guarantee that HSIM changes won't break HAPI
  • Added pytorch installation and running habitat baseline test in prediction mode. That maybe too much of testing, but decided to keep it for now.

Have ideas how to speed up and parallelize jobs. Will follow up in next PRs.

How Has This Been Tested

Tests time estimations:

Install conda and dependencies 02:56
Install cuda 04:32
Build, install habitat-sim and run benchmark 03:13
Run sim tests 03:08
Run api tests with pytorch 08:14

Types of changes

  • Docs change / refactoring / dependency upgrade
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have completed my CLA (see CONTRIBUTING)
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@facebook-github-bot facebook-github-bot added the CLA Signed Do not delete this pull request or issue due to inactivity. label May 31, 2019
export PATH=$HOME/miniconda/bin:$PATH
conda create -y -n habitat python=3.6
. activate habitat
conda install -y numpy pyyaml scipy ipython mkl mkl-include pytest
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add a conda install -y -c conda-forge ninja -- This should help with build speed

python examples/example.py --scene data/scene_datasets/habitat-test-scenes/van-gogh-room.glb
export PYTHONPATH=$PYTHONPATH:$(pwd)
./build.sh --headless
pip install -e .
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possibly bad things will happen as the pip install isn't headless.

Suggested change
pip install -e .
HEADLESS=True pip install -e .

Copy link
Collaborator

@msavva msavva left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great to see CI setup landing! 🙂 Left a minor typo comment, otherwise looks good.

sudo apt-get --yes --force-yes install cuda
nvidia-smi
- run:
name: Install habtiat-sim
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo: habtiat -> habitat

@mathfac mathfac force-pushed the circleci_initial branch 7 times, most recently from b61d14f to f016ebb Compare June 1, 2019 10:45
@mathfac
Copy link
Contributor Author

mathfac commented Jun 1, 2019

  • Added speed regression test, thanks @erikwijmans for advice how to do it in elegant way
  • Added run of habitat-api tests on master branch. That will guarantee that HSIM changes won't break HAPI
  • Added pytorch installation and running habitat baseline test in prediction mode. That maybe too much of testing, but decided to keep it for now.

@mathfac mathfac changed the title [WIP] Initial version of CI setup that runs tests using GPU machines CI setup that runs tests e2e testing using GPU machines Jun 1, 2019
@mathfac
Copy link
Contributor Author

mathfac commented Jun 1, 2019

CI tests passed, merging.

@mathfac mathfac merged commit 6f5de31 into master Jun 1, 2019
@erikwijmans erikwijmans deleted the circleci_initial branch June 1, 2019 17:48
Ram81 pushed a commit to Ram81/habitat-web-sim that referenced this pull request Dec 10, 2020
…ookresearch#63)

CI setup that runs tests using GPU machines:

Builds and installs habitat-sim with python setup.py install and ./build.sh --headless
Runs pytest tests on habitat-sim with testing data
Added speed regression test with FPS threshold (900 FPS) , thanks @erikwijmans for advice how to do it in elegant way.
Added run of habitat-api tests on master branch. That will guarantee that HSIM changes won't break HAPI
Added pytorch installation and running habitat baseline test in prediction mode. That maybe too much of testing, but decided to keep it for now.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed Do not delete this pull request or issue due to inactivity.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants