Servant Testing Helpers!

We've been looking at Haskell and HTML code for a few weeks now. Last time, we introduced the idea of serving HTML content from a Servant server. Testing any server can be a little tricky. In our Real World Haskell Series, we explored some of the complications in testing such a server.

This week, we're going to look at a couple shortcuts we can take that will make testing our server a little easier. We'll examine some helpers from the Hspec.Wai library, as well as some Quickcheck expressions we can use with Servant.

For more useful Haskell tools, download our Haskell Production Checklist. It goes through a few other libraries you can use besides Servant for your web apps!

Simple Server

Let's start with the same basic user endpoint we had from last time. This takes a User ID, and returns a User if the ID exists in our "database" (an in-memory map for our example).

data User = ...

instance ToJSON User where
  ... 

type MyAPI = "users"  :> Capture "uid" Int :> Get '[JSON] (Maybe User)

userHandler :: Int -> Handler (Maybe User)
userHandler uid = return $ M.lookup uid userDB

We've got a server to run this API, which we can use to make an Application.

myServer :: Server MyAPI
myServer = userHandler

myApi :: Proxy MyAPI
myApi = Proxy

myApp :: Application
myApp = serve myApi myServer

runServer :: IO ()
runServer = run 8080 myApp

But, as with any IO action, testing the behavior of this server is tricky. In our earlier article, we had to jump through several hoops to run some tests using the Hspec library. As a pre-condition of the test, we had to start the IO action. Then we had to construct requests using Servant's client functionality. Unwrapping and analyzing the responses was a bit annoying. Finally, we had to ensure we killed the server at the end. Luckily, there's a simpler way to go through this process.

Test Assertions

The HSpec Wai library attempts to simplify this process. As before, we write our tests in conjunction with Hspec syntax. This gives us the opportunity to put more descriptive information into our tests. Even better, we can use simple functions to send network requests to our server.

The expression we'll construct is a normal Spec, so it can fit right in with any other tests we write using Hspec. The key to this process is the with function, which takes our Application as a parameter:

apiSpec :: Spec
apiSpec = with (return myApp) $ do
  ...

The next key is the get combinator. This creates a test assertion we can use within one of our it statements. It takes a string for the "route" of the request we are sending to our server. The simplest assertion we can make is to check the status code of the reply.

apiSpec :: Spec
apiSpec = with (return myApp) $ do
  Describe "GET /users/1" $ do
    it "responds with 200" $ do
      get "users/1" `shouldRespondWith` 200

The assertion statement shouldRespondWith takes a ResponseMatcher type. We can make this matcher in a few different ways. Passing a simple number will make it only worry about the status code. If we pass a string, it will verify that the response body matches that string. Here's how we can verify that we receive a full user string when the user exists, and a "null" result when they don't.

apiSpec :: Spec
apiSpec = with (return myApp) $ do
  describe "GET /users/1" $ do
    it "responds with 200" $ do
      get "/users/1" `shouldRespondWith` 200
    it "responds with a user" $ do
      get "/users/1" `shouldRespondWith` 
        "{'email':'james@test.com','age':25,'name':'James','id':1}"
  describe "GET /users/5" $ do
    it "responds with null" $ do
      get "/users/5" `shouldRespondWith` "null"

Unfortunately there don't seem to be great mechanisms for verifying individual fields. You'll have to build any kind of custom matcher from scratch. It's certainly sub-optimal to have to match the JSON string exactly. We can reduce this burden a bit by using the JSON helper library. This let's us specify the object within a quasi-quoter so we don't have to be as precise in specifying the string:

{-# LANGUAGE QuasiQuotes #-}

apiSpec :: Spec
apiSpec = with (return myApp) $ do
  describe "GET /users/1" $ do
    ...
    it "responds with a user" $ do
      get "/users/1" `shouldRespondWith`
        [json|
        {email: "james@test.com", age: 25, name: "James", id: 1}
        |]

So there are definitely areas to improve this library. But it does provide some useful functionality.

Servant Quick Check

Another helpful way to test our API is to incorporate some of the techniques of Quickcheck. The servant-quickcheck library allows us to make blanket assertions about our API . It does this by sending many arbitrary requests to it.

We can actually incorporate these assertions into Hspec code as well. We start with a single assertion and withServantServer:

quickcheckSpec :: Spec
quickcheckSpec =
  it "API has good properties" $
    withServantServer myApi (return myServer) ...

Our key function takes a proxy for our API as well as an action returning our server. The next part is a function taking a "URL" parameter. We'll then use the serverSatisfies argument with our API and some defaultArgs.

quickcheckSpec :: Spec
quickcheckSpec =
  it "API has good properties" $
    withServantServer myApi (return myServer) $ \burl ->
      serverSatisfies myApi burl defaultArgs
        ...

The final piece is to build our actual assertions. We combine these with <%> and need to use mempty as a base. For a simple example, we can test that our endpoint never returns a 500 status code. We can also check that it never takes longer than a second (1e9 nanoseconds). Here's our complete assertion:

quickcheckSpec :: Spec
quickcheckSpec =
  it "API has good properties" $
    withServantServer myApi (return myServer) $ \burl ->
      serverSatisfies myApi burl defaultArgs
        (not500 <%> notLongerThan 1e9 <%> mempty)

Another assertion we can make is that our API only returns full JSON objects. The client code might depend on parsing these, rather than loose strings or some other format. In our case, this will actually fail with our API, because it can return null.

quickcheckSpec :: Spec
quickcheckSpec =
  it "API has good properties" $
    withServantServer myApi (return myServer) $ \burl ->
      serverSatisfies myApi burl defaultArgs
        -- Check JSON as well!
        (onlyJsonObjects <%> 
         not500 <%> 
         notLongerThan 1000000000 <%> 
         mempty)

This suggests we could reconsider how our API works. We could, for example, have it return a 404 instead of a null object if the user doesn't exist. These are some of the simplest functions in the You can take a look at the documentation for a complete listing.

Conclusion

Next week will be our last article on web applications for a little while. We'll explore what it will take to have "functional frontend" code. We'll use Purescript to generate Javascript and use this code within the HTML we send from our server!

Don't forget to subscribe to Monday Morning Haskell! This will give you access to our monthly newsletter as well as our subscriber resources!