How can I get started using the UpkieGroundVelocity environment?

[stephane-caron]

Asked by @perrin-isir in #11:

if I want to develop my own python code that would import UpkieGroundVelocity, how do I do?

[stephane-caron]

Can I use bazel to avoid having to install all the dependencies (gin-config, vulp, ...) manually?

That's one way to go. I can imagine two ways for now:

1. Start from a separate upkie_my_agent repository and use Bazel for everything
2. Make a standalone PyPI package for the gym environments. This alternative wouldn't require preparing a Bazel repo.

The idea of option 2 is that, once the spine is running, the Python gym env runs in a separate process that doesn't depend on much (all the C++ dependencies that lead to vulp are managed by Bazel on the spine side). We could therefore have a workflow like:

• Terminal 1:

cd upkie
./tools/bazelisk run //spines:bullet -c opt -- --nb-substeps 5 --show

• Terminal 2:

pip install upkie
cd my_python_code
python ./train.py  # only depends on upkie_envs

Which option do you think is preferable?

[stephane-caron]

@perrin-isir You can give it a try if you'd like 😃

pip install upkie_locomotion==0.3.0rc0

There are PyPI packages for both vulp and upkie_locomotion. Using them I'm able to test a trained policy from the PPO balancer without going through Bazel:

git:upkie_locomotion/agents/ppo_balancer$ python test.py Pacino

It yields the same result as

git:upkie_locomotion$ ./tools/bazelisk run //agents/ppo_balancer:test -- Pacino

Where Pacino.zip is a random policy I have lying around in the policies/ directory.

(Just a reminder: in both cases I have a simulation spine running as-is from the repository:)

git:upkie_locomotion$ ./tools/bazelisk run //spines:bullet -c opt -- --nb-substeps 5 --show

[stephane-caron]

(This is the work-in-progress discussion thread.) I posted the simplest example I can come up with in this answer below. It works currently from this PR by installing:

pip install -i https://test.pypi.org/simple/ upkie_locomotion==0.3.0rc2

Let me know if this works for you.

[stephane-caron]

Here is a simple example (edit: updated to Upkie 3.4.0):

import gymnasium as gym
import upkie.envs

upkie.envs.register()

if __name__ == "__main__":
    env = gym.make("UpkieGroundVelocity-v3")
    observation = env.reset(seed=42)

    action = env.action_space.sample()
    for step in range(1_000_000):
        observation, reward, done, _, _ = env.step(action)
        if done:
            observation = env.reset()
        pitch = observation[0]
        action[0] = 10.0 * pitch

    env.close()

Start the pi3hat/simulation spine in a separate process before running this script:

$ ./tools/bazelisk run //spines:bullet -c opt -- --nb-substeps 5 --show

Otherwise the script will let you know that the spine is not running.