Haskell & Open AI Gym

Our Machine Learning series explores both basic and advanced topics when it comes to using Haskell and TensorFlow. This series expands on that knowledge and demonstrates how we can use Haskell with another practical AI framework. We'll explore some of the ideas behind the Open AI Gym, which provides a Python API to make agents for simple games. We can use this agent development process to teach ourselves more about AI and machine Learning. In this series, we'll replicate some of the simplest games and ideas using Haskell. We'll get an opportunity to use TensorFlow, both in Python and in Haskell.

Part 1: Frozen Lake Primer

One of the simplest games in the whole Open AI framework is the Frozen Lake scenario. In this first part, we'll learn the basic rules of this scenario and see how we can use it in Python.

Part 2: Frozen Lake in Haskell

Part 2 will provide a basic implementation for Frozen Lake in Haskell. We'll start to establish some of the core ideas we'll use in our games. In particular, we'll see our first example of an "Environment".

Part 3: Blackjack

In part 3, we'll make a second basic game, a simple implementation of Blackjack. We'll observe the concepts our two games have in common and use this to start building an idea of a common typeclass.

Part 4: Basic Q-Learning

In the fourth part of the series, we'll apply our first machine learning method. We'll take the concept of Q-Learning and apply it to both of our games.

Part 5: Generalizing Our Environments

We've made some good progress with both games separately. But now it's time to bring them together. Part 5 of the series shows how we can make a type family to generalize the common elements of our two games. This will allow us to run both games with nearly identical code.

Part 6: Q-Learning with TensorFlow (Python)

The initial Q-Learning approach from Part 4 has a limited scope. In part 6, we'll see a more general Q-Learning approach that uses TensorFlow. We'll start out in Python, since the code is a bit simpler to follow.

Part 7: Q-Learning with TensorFlow (Haskell)

Armed with the basic ideas of Q-Learning and tensors, we'll move onto part 7, where we'll apply these ideas in Haskell.

Part 8: Rendering with Gloss

Our only form of rendering so far is text-based. In the final part of this series, we'll see how we can render the Frozen Lake game in a more advanced way using the Gloss library. While the integration isn't perfect, we'll see that it still fits in reasonably well with a lot of our ideas so far.

Reference: TensorFlow Guide

This series culminates with using the Haskell TensorFlow library to provide a learning mechanism for our games. It's very tricky to get this library working, and we don't dwell on the details in this series. Download this guide to get yourself going!