Your experience on AI training speed


Hi, I’m new to this so there is a risk I might ask dumb questions.

I have set up default AI platform and started my first agent training. I choose to run Coaster racer flash game. Since first launch it went through game in 4 min and earning about 8000-10000 points, but after a day or so of non-stop running I have not noticed any progress. It’s still 4 min and same points and it is still hitting side walls and jumping off cliffs.


It seems likely that you have a problem, but it is hard to tell from your description what that could be. “My agent is not learning” could be anything from a trivial typo to a fundamental misunderstanding of how the algorithm is supposed to be applied. It could also just be you have low resources and time to learn is much longer on your hardware than expected.

You should start with the simpler problems. Training reinforcement learning agents is not straightforward, and you need the experience and insight from solving things like CartPole and MountainCar before you tackle a more complex environment.


As mentioned by @NeilSlater you should start with a simpler problem in order to validate your assumptions, model, and approach.

That said, you could be facing a problem of overfitting, i.e. your model has converged too much toward the initial training set. Could you describe more what your approach is, what training strategies and what parameters (functions, learning rate, type of NN…) you are using.


In the end I setup universe-starter-agent. After a week of tinkering I solved pong in an hour. I’ll try other environments. My mistake was that I installed new Ubuntu 17, that caused a lot of compatibility issues.