Neural episodic control


Since around the year 2017 some papers were published at Arxiv with the above mentioned subject and with the aim to play OpenAI gym -games with it. For all, how have no access to Arxiv the concept in summary:

At first, a human expert is playing the game. The sensors and actions are recorded into a database. This is called “episodic memory” or “case based reasoning”. Then a feature vector is constructed. That is a blueprint for a similarity search which is done with neural networks. If the agent plays the game, he searches in the episodic memory for a similar scene and use the information for guiding the search in the game-tree.

The concept of case-based learning is not new. But what “neural episodic control” has done is to give a concrete tutorial how to use a feature vector together with neural networks to retrieve information from the past. The authors call the concept model-free episodic memory, because the neural networks searches through the raw data, which is not very effective. The advantage is, that such a solution can be easy programmed and tested on real OpenAI gym environments which results into nice looking performance graphs. It is likely, that future papers will explore the idea further, which results into high quality AI-controller.