Learning Resources

In addition to the documentation provided in this project, there are many valuable resources available for those who wish to deepen their understanding of reinforcement learning and related topics. We have compiled a list of recommended papers, textbooks, and blog posts that we have found particularly helpful. These resources range from beginner-friendly introductions to more advanced topics, and we hope that they will be useful to you in your learning journey. In no particular order, we liked the following resources:

• Sutton & Barto, RL Textbook Bible

• Implicit Quantile Networks for Distributional Reinforcememnt Learning

• Revisiting Fundamentals of Experience Replay

• Playing Atari with Deep Reinforcement Learning

• Rainbow: Combining Improvements in Deep Reinforcement Learning

While less central to this project, we also liked:

• Trackmania - The History of Machine Learning in Trackmania, A really cool blog post

• The Phenomenon of Policy Churn, Intuitions about established knowledge can be very very wrong

• Return-based Scaling: Yet Another Normalisation Trick for Deep RL, An actual formalization of a recommended practice

• Bigger, Better, Faster: Human-level Atari with human-level efficiency, Cool tricks that don’t seem to help, but cool anyway.

• Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learning, same as above

• Stabilizing Off-Policy Deep Reinforcement Learning from Pixels, same as above

• Understanding Plasticity in Neural Networks, The reason why we train from scratch on a new map… But we haven’t tested much of this.