Skip to content
This repository has been archived by the owner on Oct 22, 2024. It is now read-only.

Fix wrong dependencies #29

Merged
merged 11 commits into from
Jan 26, 2023
Merged

Fix wrong dependencies #29

merged 11 commits into from
Jan 26, 2023

Conversation

h-vetinari
Copy link
Member

@h-vetinari h-vetinari commented Jul 17, 2022

Starting from 0.22, gym depends on pygame, which hasn't been packaged yet (pending conda-forge/staged-recipes#19665).

The upgrade PRs were never merged, until #26, where I didn't realize that this had changed. Since we had to mark those builds as broken, we can still continue with 0.22.0 (and then 0.23.x, and so on).

ale-py is also still missing, pending conda-forge/staged-recipes#19664

Closes #33

@conda-forge-linter
Copy link

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@Tobias-Fischer
Copy link

@conda-forge-admin please restart ci

@h-vetinari
Copy link
Member Author

@conda-forge-admin, please rerender

@Tobias-Fischer
Copy link

I'm a bit lost. It seems like we need mujoco_py, which is deprecated and not supported for mujoco>2.1 (ref: https://github.com/openai/mujoco-py/) (and we only have >2.1 on conda-forge). Then it also seems like gym is now gymnasium, and gym does not get any fixes anymore (ref: openai/gym#2524 (comment)).

That said, even in this gymnasium fork, they still import mujoco_py: https://github.com/Farama-Foundation/Gymnasium/blob/main/gymnasium/envs/mujoco/mujoco_env.py

@h-vetinari
Copy link
Member Author

Mujoco used to be closed-source software until it got bought(?) by google and opensourced. That's why there's probably a split between import patterns. Previously we didn't distribute gym-mujoco due to these reasons. In this case I think we could just add a sed script that replaces import mujoco_py with import mujoco.

Maybe @pseudo-rnd-thoughts can help us explain the split between gym/gymnasium. IIUC correctly, we should be able to just switch sources to gymnasium here at a certain version (and probably switch to the new name then, keeping the old outputs for compatibility for a while)

@h-vetinari
Copy link
Member Author

BTW, do you want to join as a maintainer here @Tobias-Fischer - seeing that you were instrumental in pushing ale-py and pygame over the finishline? 🙃

@h-vetinari
Copy link
Member Author

This is weird. The code wants to import gym.envs.atari, but there's no such module in the sources...?? Does this somehow get added in the process of packaging the pypi wheels?

@h-vetinari
Copy link
Member Author

Great, starting off with a segfault right off the bat after trying to run the test suite... 😑

+ pytest -v tests/
============================= test session starts ==============================
platform linux -- Python 3.8.15, pytest-7.2.0, pluggy-1.0.0 -- $PREFIX/bin/python3.8
cachedir: .pytest_cache
rootdir: $SRC_DIR
Fatal Python error: Segmentation fault

@pseudo-rnd-thoughts
Copy link

Hi @h-vetinari and @Tobias-Fischer, thanks for your work on this repo.
Answers to a couple of questions

  1. The difference between mujoco-py and mujoco = While MuJoCo was closed source and a license was required, I don't believe that it had any python bindings therefore OpenAI created mujoco-py. However OpenAI don't maintain the project anymore and when Deepmind brought MuJoCo they released mujoco which contains python bindings natively (+ a conda project). We do have plans to retire mujoco-py (along with all of the v2 / v3 mujoco environment) however that will not happen for a while. However mujoco-py and mujoco are not equivalent due to the binding, therefore, I would say we just support mujoco and not mujoco-py on conda.
  2. Why the split of Gym and Gymnasium = Read our announcement post and specifically the Gymnasium section https://farama.org/Announcing-The-Farama-Foundation
  3. Atari looking for gym.envs.atari = My understanding was that atari was hacked into gym such that on install, it would install files into gym, in particular gym.envs.atari. In v0.8.0 (and gym v0.26.0) this was fixed to be less hacky and use entry_points.

Im not familiar with conda but Im happy to set up a conda-dev channel on the gym discord in order to talk about getting gym (and hopefully gymnasium) on conda if either of you are interested

@h-vetinari
Copy link
Member Author

h-vetinari commented Dec 13, 2022

Thanks for the inputs @pseudo-rnd-thoughts

  1. However mujoco-py and mujoco are not equivalent due to the binding, therefore, I would say we just support mujoco and not mujoco-py on conda.

Yeah, we cannot support the older mujoco-py (unless it somehow became license-free), but what I'm wondering is what you mean by "not equivalent due to the binding". Are they equivalent enough that we can drop in mujoco for mujoco-py?

2. Why the split of Gym and Gymnasium = Read our announcement post and specifically the Gymnasium section

The way I read the announcement is that at 0.26.2 we could switch over sources and then continue with gymnasium, right?

3. Atari looking for gym.envs.atari = My understanding was that atari was hacked into gym such that on install, it would install files into gym, in particular gym.envs.atari

This used to work in previous gym versions on conda, but for now I'm fine with dropping gym-atari from our packaging, until we find (or the upstream version supports) a sane way to install it properly.

Im not familiar with conda but Im happy to set up a conda-dev channel on the gym discord in order to talk about getting gym (and hopefully gymnasium) on conda if either of you are interested

Gym <=0.21 is already in conda-forge, but we've been stuck on the lack of pygame ever since. We didn't run the test suite previously, but with all the changes, I thought it prudent to do so. The results are not looking great (in particular it's segfaulting on Linux), so your help in analysing those would be very welcome. If it helps, I can provide a way to install the artefacts being created in this PR in a way that you can install them.

It would be fine to skip tests that simply cannot run in our CI, i.e. if they need an actual screen or GPU, but generally it would be good to run as much as we can.

It's also possible that we skip some versions (e.g. go straight to 0.26), but I was looking to fill in the gaps and publish all 0.x versions since 0.21, now that we have pygame. However that was conditional on a passing test suite, and if it's easier to get it to pass with a new version, I'm happy to skip some minor versions in between.

@pseudo-rnd-thoughts
Copy link

  1. I believe not. While mujoco-py is not license free, the license is freely available so it might be possible to create a conda mujoco-py module
  2. Yes, Gymnasiun v0.26.1 == Gym v0.26.1 except for import gym == import gymnasium as gym
  3. Ok, that makes sense. Gym haven't changed pygame since 0.21 so that should be fine to use the conda 3.1.2.dev8 release however ale-py has changed a couple of times, in particular 0.8.0 uses the new step function used in gym 0.26.0

My time is restricted as Im trying to finish a paper for IJCAI in mid-January but if there is an easy way of installing / testing I can try to find the time to help.

@Tobias-Fischer
Copy link

Many thanks for the insights, @pseudo-rnd-thoughts!

Two questions/suggestions:

  • @h-vetinari - while I, in principle, agree that having all versions of gym available here would be nice, skipping to the latest version might save us lots of trouble and time? Especially considering the jump from gym to gymnasium, too, I wonder how many users would actually want to use an "in-between" version.
  • @pseudo-rnd-thoughts - indeed https://github.com/openai/mujoco-py is available under MIT License so we could package it here. Do you know if mujoco-py v2.1.12.14 (the latest version) supports mujoco 2.3.1 (the latest version we package here on conda-forge)? The release notes only mention support for version 2.1.

@h-vetinari
Copy link
Member Author

  • Especially considering the jump from gym to gymnasium,

I'm thinking we could go straight to gymnasium. This feedstock has been in a coma for a while, perhaps it would be cleaner to let it lie. Users can switch their dependency to gymnasium (perhaps we can communicate this upstream with the help of @pseudo-rnd-thoughts once ready)

@h-vetinari
Copy link
Member Author

I'm thinking we could go straight to gymnasium.

And by that I mean open a PR to staged-recipes, which takes this recipe and adapts it to gymnasium. We can even keep the history of this feedstock with some surgery afterwards (same was done e.g. for qt->qt-main)

@Tobias-Fischer
Copy link

Agreed that this is probably the best way forward!

@pseudo-rnd-thoughts
Copy link

Without contacting the development team, someone has already made a Gymnasium feedstock. https://github.com/conda-forge/gymnasium-feedstock
I would love the help to update the feedstock so I don't mind what you do with this PR

@h-vetinari
Copy link
Member Author

Thanks for the pointer @pseudo-rnd-thoughts!

We managed to backport the history to that feedstock and now work continues in conda-forge/gymnasium-feedstock#9

@Tobias-Fischer, the invitation to join the maintenance team is still open ;-)

Also, it looks like we'll have our work cut out for us with the next big ecosystem that joins the gym dependencies as of 0.27: conda-forge/gymnasium-feedstock#8

@conda-forge-webservices
Copy link

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants