Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable minimal CircleCI setup #552

Merged
merged 11 commits into from Jul 13, 2021
Merged

Enable minimal CircleCI setup #552

merged 11 commits into from Jul 13, 2021

Conversation

shwang
Copy link
Member

@shwang shwang commented Jul 9, 2021

Configures a minimal CircleCI setup. I uploaded to DockerHub
a minimal Docker image bundling the Ubuntu and Java dependencies
that are required to pip install -e . MineRL. CircleCI pulls
this Docker image and runs pytest on top of it.

Because the CircleCI setup is not complete, we should run both BuildKite and
CircleCI for "hybrid" testing until we complete address the issues below:

This PR adds conditional skips to some data pytests when MINERL_DATA_ROOT is
not set. This is because the minimal CircleCI set up doesn't download
and cache any MineRL datasets right now. I'm leaving that for future
work.

Other future work:

  • Pytests are not parallelized yet.
  • Maybe transfer ownership of springulum/minerl-circleci-base Docker image
    to a shared MineRL account. (But I don't expect that we actually need to
    update this image for a long time, since its just a bunch of Java / Ubuntu packages.)

@shwang
Copy link
Member Author

shwang commented Jul 9, 2021

oof, it's failing on CircleCI's server (during MineRL Gradle build), but not on my local CircleCi runs. Maybe this is because we don't have enough memory in the free Runners offered to open source projects.

@shwang
Copy link
Member Author

shwang commented Jul 9, 2021

ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526 FAILURE: Build failed with an exception.
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526 
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526 * What went wrong:
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526 Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed)
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526 
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526 * Try:
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526 Run with --info or --debug option to get more log output. Run with --scan to get full insights.
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526 
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526 * Exception is:
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526 org.gradle.launcher.daemon.client.DaemonDisappearedException: Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed)
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526 	at org.gradle.launcher.daemon.client.DaemonClient.handleDaemonDisappearance(DaemonClient.java:242)
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526 	at org.gradle.launcher.daemon.client.DaemonClient.monitorBuild(DaemonClient.java:218)
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526 	at org.gradle.launcher.daemon.client.DaemonClient.executeBuild(DaemonClient.java:179)
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526 	at org.gradle.launcher.daemon.client.DaemonClient.execute(DaemonClient.java:142)
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526 	at 
[...]
org.gradle.wrapper.BootstrapMainStarter.start(BootstrapMainStarter.java:30)
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526 	at org.gradle.wrapper.WrapperExecutor.execute(WrapperExecutor.java:127)
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526 	at org.gradle.wrapper.GradleWrapperMain.main(GradleWrapperMain.java:61)
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526 
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526 
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526 * Get more help at https://help.gradle.org
DEBUG    minerl.env._multiagent:_multiagent.py:660 Closing MineRL env...

@shwang shwang mentioned this pull request Jul 9, 2021
@shwang
Copy link
Member Author

shwang commented Jul 13, 2021

OK, we are ready to push this -- we've got CircleCI running on tests that don't launch Minecraft, and it takes about 2.5 minutes to finish those tests. We can try launching Minecraft when we figure out memory requirements later.

@brandonhoughton
Copy link
Member

WOO! This is great =)

@shwang shwang merged commit 606548c into dev Jul 13, 2021
@shwang shwang deleted the circle_ci branch July 13, 2021 02:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants