OpenAI sure did a great job with its release of Baselines. But there are somethings that I think could be done to make them easier to use:
Only the DQN has its own README explaining the first steps to see the algorithm in action.
It would be nice to have a simple way to run each of the algorithms on the CartPole problem. Being such an easy environment its a nice place to run quick tests and compare algorithms at first.
Can these algorithms be run on the GPU? Since the models for atari ar CNN it would make sense to use the GPU, wouldn’t it?
These are things that I think would help me kick-start my research. I would be happy to contribute myself as long as I could get some pushes in the right direction