Render window crashes, game keeps on playing


#1

Hi,

I am trying to run the simplest possible example:

import gym
import universe
from universe import wrappers

env = gym.make(‘flashgames.NeonRace-v0’)
#env = wrappers.experimental.SafeActionSpace(env)
env.configure(remotes=1)
observation_n = env.reset()
while True:
** action_n = [[(‘KeyEvent’, ‘ArrowUp’, True)] for ob in observation_n] # your agent here**
** observation_n, reward_n, done_n, info = env.step(action_n)**
** env.render()**

The window pops up but after a while disappears and I get the following error:

universe-rMYIDn-0 | [nginx] 172.17.0.1 - openai [07/Mar/2017:11:04:10 +0000] “GET / HTTP/1.1” 101 4619 “-” "AutobahnPython/0.17.2"
universe-rMYIDn-0 | [2017-03-07 11:04:10,364] [INFO:universe.rewarder.remote] WebSocket connection closed: connection was closed uncleanly (peer dropped the TCP connection without previous WebSocket closing handshake)
universe-rMYIDn-0 | [2017-03-07 11:04:10,370] [INFO:universe.rewarder.remote] [Twisted] Active client disconnected (sent 11 messages). Still have 0 active clients left

[2017-03-07 12:04:10,373] [0] Closing rewarder connection

universe-rMYIDn-0 | [tigervnc] Connections: closed: 172.17.0.1::33794 (Clean disconnection)
universe-rMYIDn-0 | [tigervnc] EncodeManager: Framebuffer updates: 479
universe-rMYIDn-0 | [tigervnc] EncodeManager: Tight:
universe-rMYIDn-0 | [tigervnc] EncodeManager: Solid: 105 rects, 3.74915 Mpixels
universe-rMYIDn-0 | [tigervnc] EncodeManager: 1.64062 KiB (1:8927.31 ratio)
universe-rMYIDn-0 | [tigervnc] EncodeManager: Bitmap RLE: 291 rects, 5.25657 Mpixels
universe-rMYIDn-0 | [tigervnc] EncodeManager: 36.3008 KiB (1:565.742 ratio)
universe-rMYIDn-0 | [tigervnc] EncodeManager: Indexed RLE: 849 rects, 20.2569 Mpixels
universe-rMYIDn-0 | [tigervnc] EncodeManager: 526.603 KiB (1:150.281 ratio)
universe-rMYIDn-0 | [tigervnc] EncodeManager: Tight (JPEG):
universe-rMYIDn-0 | [tigervnc] EncodeManager: Full Colour: 699 rects, 24.1827 Mpixels
universe-rMYIDn-0 | [tigervnc] EncodeManager: 2.96385 MiB (1:31.1276 ratio)
universe-rMYIDn-0 | [tigervnc] EncodeManager: Total: 1.944 krects, 53.4453 Mpixels
universe-rMYIDn-0 | [tigervnc] EncodeManager: 3.51517 MiB (1:58.0058 ratio)

---------------------------------------------------------------------------
Error Traceback (most recent call last)
in ()
** 14 while True:**
** 15 action_n = [[(‘KeyEvent’, ‘ArrowUp’, True)] for ob in observation_n] # your agent here**
—> 16 observation_n, reward_n, done_n, info = env.step(action_n)
** 17 env.render()**

/media/iulialexandra/Storage/code/RL/gym/gym/core.py in step(self, action)
** 108 info (dict): contains auxiliary diagnostic information (helpful for debugging, and sometimes learning)**
** 109 “”"**
–> 110 observation, reward, done, info = self._step(action)
** 111 return observation, reward, done, info**
** 112 **

/media/iulialexandra/Storage/code/RL/universe/universe/wrappers/timer.py in _step(self, action_n)
** 18 start = time.time()**
** 19 with pyprofile.push(‘vnc_env.Timer.step’):**
—> 20 observation_n, reward_n, done_n, info = self.env.step(action_n)
** 21 **
** 22 # Calculate how much time was spent actually doing work**

/media/iulialexandra/Storage/code/RL/gym/gym/core.py in step(self, action)
** 108 info (dict): contains auxiliary diagnostic information (helpful for debugging, and sometimes learning)**
** 109 “”"**
–> 110 observation, reward, done, info = self._step(action)
** 111 return observation, reward, done, info**
** 112 **

/media/iulialexandra/Storage/code/RL/universe/universe/wrappers/render.py in _step(self, action_n)
** 28 **
** 29 def _step(self, action_n):**
—> 30 observation_n, reward_n, done_n, info_n = self.env.step(action_n)
** 31 self._observation = observation_n[0]**
** 32 return observation_n, reward_n, done_n, info_n**

/media/iulialexandra/Storage/code/RL/gym/gym/core.py in step(self, action)
** 108 info (dict): contains auxiliary diagnostic information (helpful for debugging, and sometimes learning)**
** 109 “”"**
–> 110 observation, reward, done, info = self._step(action)
** 111 return observation, reward, done, info**
** 112 **

/media/iulialexandra/Storage/code/RL/universe/universe/wrappers/throttle.py in _step(self, action_n)
** 115 action_n.append([])**
** 116 **
–> 117 observation_n, reward_n, done_n, info = self._substep(action_n)
** 118 **
** 119 # Merge observation, rewards and metadata.**

/media/iulialexandra/Storage/code/RL/universe/universe/wrappers/throttle.py in _substep(self, action_n)
** 130 start = time.time()**
** 131 # Submit the action ASAP, before the thread goes to sleep.**
–> 132 observation_n, reward_n, done_n, info = self.env.step(action_n)
** 133 **
** 134 available_at = info[‘throttle.observation.available_at’] = time.time()**

/media/iulialexandra/Storage/code/RL/gym/gym/core.py in step(self, action)
** 108 info (dict): contains auxiliary diagnostic information (helpful for debugging, and sometimes learning)**
** 109 “”"**
–> 110 observation, reward, done, info = self._step(action)
** 111 return observation, reward, done, info**
** 112 **

/media/iulialexandra/Storage/code/RL/universe/universe/envs/vnc_env.py in _step(self, action_n)
** 462 self._handle_initial_n(observation_n, reward_n)**
** 463 self._handle_err_n(err_n, vnc_err_n, info_n, observation_n, reward_n, done_n)**
–> 464 self._handle_crashed_n(info_n)
** 465 **
** 466 return observation_n, reward_n, done_n, {‘n’: info_n}**

/media/iulialexandra/Storage/code/RL/universe/universe/envs/vnc_env.py in _handle_crashed_n(self, info_n)
** 535 raise error.Error(’{}/{} environments have crashed. No error key in info_n: {}’.format(len(self.crashed), self.n, info_n))**
** 536 else:**
–> 537 raise error.Error(’{}/{} environments have crashed! Most recent error: {}’.format(len(self.crashed), self.n, errors))
** 538 **
** 539 def _propagate_obs_info(self, info_n, obs_info_n):**

Error: 1/1 environments have crashed! Most recent error: {‘0’: ‘Rewarder session failed: Lost connection: connection was closed uncleanly (peer dropped the TCP connection without previous WebSocket closing handshake) (clean=False code=1006)’}

However, after this error the game seems to keep on running in the background, because this keep on being printed:

universe-lmV8Qd-0 | [2017-03-07 11:25:42,706] [INFO:universe.wrappers.logger] Stats for the past 5.01s: vnc_updates_ps=5.0 n=1 reaction_time=None observation_lag=None action_lag=None reward_ps=0.0 reward_total=0.0 vnc_bytes_ps[total]=6089.9 vnc_pixels_ps[total]=45010.2 reward_lag=None rewarder_message_lag=None fps=57.70
universe-XUQA0t-0 | [2017-03-07 11:25:43,130] [play_vexpect] Advancing to the next hopeful state (1/2): MaskState
universe-XUQA0t-0 | [2017-03-07 11:25:43,934] [play_vexpect] Applying transition: ClickTransition[‘initializing2’] x=571 y=512 buttonmask=1> for active state MaskState. (Summary: plausible_states=[MaskState, MaskState] distance_m=[0.0365, 0.7507] match_time_m=[‘273us’, ‘290us’])
universe-XUQA0t-0 | [2017-03-07 11:25:43,951] [play_vexpect] Waiting for any of [MaskState] to activate (or whether any of [MaskState] are still active)
universe-jBHeNf-0 | [2017-03-07 11:25:44,298] [play_vexpect] Applying transition: ClickTransition[‘ready0’, ‘ready1’, ‘ready2’] x=107 y=234 buttonmask=1> for active state initialize2. (Summary: plausible_states=[initialize2, initialize1] distance_m=[0.0, 1.1360776] match_time_m=[‘208us’, ‘74us’])
universe-jBHeNf-0 | [2017-03-07 11:25:44,317] [play_vexpect] Waiting for any of [ready0, ready1, ready2] to activate (or whether any of [initialize2] are still active)
universe-XUQA0t-0 | [2017-03-07 11:25:44,998] [play_vexpect] Fell behind by 0.13258123397827148s from target; losing 7 frames
universe-XUQA0t-0 | [2017-03-07 11:25:45,422] [play_vexpect] Fell behind by 0.11572432518005371s from target; losing 6 frames
universe-XUQA0t-0 | [2017-03-07 11:25:45,590] [play_vexpect] Fell behind by 0.1460554599761963s from target; losing 8 frames
universe-jBHeNf-0 | [2017-03-07 11:25:45,669] [play_vexpect] Applying transition: ClickTransition[] x=0 y=0 buttonmask=0> for active state ready1. (Summary: plausible_states=[ready0, ready1, ready2, initialize2] distance_m=[2.5031936, 0.0046253935, 0.0062009618, 1.1523193] match_time_m=[‘210us’, ‘83us’, ‘80us’, ‘75us’])
universe-jBHeNf-0 | [2017-03-07 11:25:45,692] [play_vexpect] Reaching start state: ready1
universe-jBHeNf-0 | [2017-03-07 11:25:45,692] [play_vexpect] vexpect macro complete in 50.758966s
**universe-jBHeNf-0 | [tigervnc] **
universe-jBHeNf-0 | [tigervnc] Tue Mar 7 11:25:45 2017
universe-jBHeNf-0 | [tigervnc] Connections: closed: 127.0.0.1::56756 (Clean disconnection)
universe-jBHeNf-0 | [tigervnc] EncodeManager: Framebuffer updates: 195
universe-jBHeNf-0 | [tigervnc] EncodeManager: ZRLE:
universe-jBHeNf-0 | [tigervnc] EncodeManager: Solid: 8 rects, 2.299 kpixels
universe-jBHeNf-0 | [tigervnc] EncodeManager: 240 B (1:38.7167 ratio)
universe-jBHeNf-0 | [tigervnc] EncodeManager: Bitmap RLE: 65 rects, 47.14 kpixels
universe-jBHeNf-0 | [tigervnc] EncodeManager: 9.13281 KiB (1:20.2459 ratio)
universe-jBHeNf-0 | [tigervnc] EncodeManager: Indexed RLE: 283 rects, 764.826 kpixels
universe-jBHeNf-0 | [tigervnc] EncodeManager: 253.806 KiB (1:11.7843 ratio)
universe-jBHeNf-0 | [tigervnc] EncodeManager: Full Colour: 28 rects, 411.592 kpixels
universe-jBHeNf-0 | [tigervnc] EncodeManager: 1.17857 MiB (1:1.33248 ratio)
universe-jBHeNf-0 | [tigervnc] EncodeManager: Total: 384 rects, 1.22586 Mpixels
universe-jBHeNf-0 | [tigervnc] EncodeManager: 1.43557 MiB (1:3.26049 ratio)
universe-lmV8Qd-0 | [2017-03-07 11:25:46,082] [INFO:universe.pyprofile] [pyprofile] period=5.02s timers={“score.crop_cache.get.OCRScorerV0”: {“std”: “46.20us”, “mean”: “112.55us”, “calls”: 24}, “reward.parsing.gameover”: {“std”: “2.29ms”, “mean”: “594.99us”, “calls”: 24}, “vnc_env.VNCEnv.vnc_session.step”: {“std”: “4.83ms”, “mean”: “1.92ms”, “calls”: 291}, “rewarder.sleep.missed”: {“std”: “8.25ms”, “mean”: “10.53ms”, “calls”: 17}, “score.crop_cache.get.MatchImage”: {“std”: “22.22us”, “mean”: “61.39us”, “calls”: 24}, “rewarder.compute_reward”: {“std”: “5.03ms”, “mean”: “2.32ms”, “calls”: 291}, “rewarder.sleep”: {“std”: “3.10ms”, “mean”: “14.01ms”, “calls”: 274}, “reward.parsing.score”: {“std”: “1.57ms”, “mean”: “672.14us”, “calls”: 24}, “rewarder.frame”: {“std”: “4.59ms”, “mean”: “18.76ms”, “calls”: 291}} counters={“agent_conn.reward”: {“std”: 0, “mean”: 0.0, “calls”: 1}, “reward.vnc.updates.n”: {“std”: 0.2755597347888479, “mean”: 0.08247422680412371, “calls”: 291}, “score.crop_cache.hit.MatchImage”: {“std”: 0.0, “mean”: 1.0, “calls”: 24}, “score.crop_cache.hit.OCRScorerV0”: {“std”: 0.0, “mean”: 1.0, “calls”: 24}} gauges={“reward_parser.score.last_score”: {“value”: 41.0, “std”: 0.0, “mean”: 41.0, “calls”: 24}} (export_time=136.61us)
universe-lmV8Qd-0 | [2017-03-07 11:25:46,085] [INFO:universe.rewarder.remote] [Rewarder] Over past 5.01s, sent 1 reward messages to agent: reward=0 reward_min=0 reward_max=0 done=False info={‘rewarder.vnc.updates.bytes’: 0, ‘rewarder.vnc.updates.n’: 0, ‘rewarder.vnc.updates.pixels’: 0, ‘rewarder.profile’: ‘<1828 bytes>’}
universe-XUQA0t-0 | [2017-03-07 11:25:46,515] [play_vexpect] Applying transition: ClickTransition[‘ready0’, ‘ready1’, ‘ready2’, ‘ready3’] x=216 y=296 buttonmask=1> for active state MaskState. (Summary: plausible_states=[MaskState, MaskState] distance_m=[0.0361, 0.9961] match_time_m=[‘346us’, ‘262us’])

I have tried the same code on Dusk Drive and the window does not disappear, but it gets stuck and after a while the game restarts only to get stuck again.

I am using jupyter notebok with python 3.6 on Ubuntu 16.04.

Could anybody please help me figure this out?

Thanks a lot!