OpenAI Gym in Jupyter Notebook


I am using Gym in Jupyter notebook. My question is, how can I force notebook to show output (video) in the notebook itself rather than creating a new window? Is it possible? If yes, then how?

Also, I am unable to close the output window when it is created by a notebook.