Week 11: Final Project Progress

This week I spent my time learning about the q-learning algorithm and implementing it using javascript.  I found Arthur Juliani's article series to be the best explanation of simple q learning.  Using his python script and Aidan Nelson’s implementation of this code in javascript I implemented a simple q-learning table for my own purposes (Figure 1).

A big thank you to Aidan Nelson for his example as it provided a baseline for my implementation.  To view my code visit this link and to view my example visit this link (press t on the keyboard to train and then r on the keyboard to run).

Now that I better understand the q-table algorithm I have a few steps that I would like to make in the coming weeks.  First I would like to implement an interactive component to the sketch so that before training the agent, the user can change the maze environment.  Then I would like to use this method to visualize data.  I would like to develop a series of environments based on real data and run the agent through that environment.  For example the data could be information about my gym habits.  If I go to the gym on one day, then the maze would show a safe tile at the corresponding spot in the maze.  If I do not go to the gym on one day, the maze would show an unsafe tile at the corresponding spot in the maze.  If I went to the gym a lot, there would be many safe spots and the agent would have an easy time completing the maze (less training requires).  If I skipped the gym a lot, there would be many unsafe spots and the agent would have a very difficult time completing the maze (more training required).  I would like to play around with this idea of how the maze can visualize your data in an interesting way and motivate your actions.  Ultimately, I would like to scale this up so that the maze could be much larger than 5X5 or 4X4.  In this case I may need to implement a neural network to more efficiently work through the q table. 


References:

  1. https://github.com/AidanNelson/reinforcement-learning-experiments/tree/master/simple-rl-tutorials/0-q-learning-agents

  2. https://gist.github.com/awjuliani/9024166ca08c489a60994e529484f7fe#file-q-table-learning-clean-ipynb

  3. https://medium.com/emergent-future/simple-reinforcement-learning-with-tensorflow-part-0-q-learning-with-tables-and-neural-networks-d195264329d0