Any good and free continuous action space environment aside from OpenAI or Universe?


I would like to try out my RL model on different continuous action space env that don’t require licensing.

I have already tried Bipedal Walker.

Any ideas?


DeepMind has recently released their Control Suite:


PyBullet has some very similar envs to the MuJoCo envs in Gym and uses the same API. For example:

$ pip install pybullet
$ python
> import gym, pybullet_envs
> env = gym.make('AntBulletEnv-v0')

Roboschool is also built around PyBullet and has additional tasks but can be a bit complicated to set up.


You could use the torcs driving simulator.


Thanks a bunch Danijar.
This is precisely the type of environment I was looking for.

Looking further I noticed that OpenAI did build additional environments on top of pyBullet called Roboschool.

I will try both.




Hey Danijar,
Were you able to make pybullet work.
pip install pybullet keeps crasing


Yes, it works well for me. I installed it a few months ago though.


I finally got pyBullet to work.
It was a bit of a pain, but so is anything ML related.

I still can’t get Roboschsool to work, but pyBullet will do for now.