Deeper Stack Knowledge

This week we'll look at another way we can "level up" our Haskell skills. We'll look at some of the details around how Stack determines package versions. This will help us explain the nuances of when you need "extra deps" and why. We'll also explore some ways to bring in non-standard Haskell code.

But of course you need to know the basics before you can really start going! So if you've never used Stack before, take our free Stack mini-course!

Adding Libraries (Basics)

When I'm writing a small project for one of these articles, I don't have to think much about library versions. Generally, anything recent is fine. Let's take a simple example using Servant. I can start the project with stack new ServantExample. Then I can add servant as a dependency for my library by modifying the .cabal file:

build-depends:
  servant

When we run stack build, it'll install a whole bunch of dependencies for our project. We can do stack ls dependencies (or stack list-dependencies if you're on an older version of Stack). We'll see a list with many libraries, because servant has a lot of dependencies. In this article, we'll explore a few questions. How does Stack know which versions of these to get? Is it always finding the latest version? What happens if we need a different version? Do we ever need to look elsewhere?

Well first, we can, if we want, specify manual constraints on libraries within the .cabal file. To start, we can observe that Stack has downloaded servant-0.14.1 and directory-1.3.1.5 for our program. What happens if we add constraints like so:

build-depends:
  servant >= 0.14.1
  directory <= 1.2.0.0

We'll find that we can't build, because no version of directory matches our constraints. The error message will suggest adding it to extra dependencies in stack.yaml (we'll talk about this later). But this will cause dependency conflicts. So how do we avoid these conflicts? To understand this, let's examine the concept of resolvers.

Resolvers

Resolvers are one of the big things that separate Stack from Cabal on its own. A resolver is a series of libraries that have no conflicts among their dependencies. Each resolver has a version number.

If we go into our stack.yaml file, we'll see that we have a field that relates to the lts version number of our resolver. When we invoked the stack new command, this chose the latest lts, or "Long-Term Support" resolver. If there's an issue with this resolver, we can ask the great people at Stackage what's going wrong. At the time of writing, this version was 12.9:

resolver: lts-12.9

There are other kinds of resolvers we can use, as the comments in our auto-generated file will tell us. There are nightly builds, and resolvers that map to particular versions of GHC.

But let's stick to the idea of lts resolvers for now. A resolver gives us a big set of packages that work together and have no dependency conflicts. This prevents some of the more annoying issues that can come along when we try to have a lot of libraries.

For lts resolvers, the package directory lives on Stackage, and we can examine it if we like. We can see, for instance, on the site that there's a page dedicated to listing everything for lts-12.9. And we can compare the library versions on this site to what we've already got in our directory. And we'll see they're the same! For example, it lists version 0.14.1 of servant and version 1.3.1.5 of directory.

So when we write our cabal file, we don't need to list version constraints on our packages. Stack will find the matching version from the resolver. Then we'll know that we can meet dependency constraints with other packages there!

Resolvers and GHC

Now, our resolver has to work with whatever compiler we use. Each lts resolver links to a specific GHC version. We can't use our lts-12.9 resolver with GHC 7.10.3, because many of the library versions it links to only work for GHC 8+. If we are intent on using an older version of GHC, we'll have to use a resolver version that corresponds to it.

We can also get this kind of information by going onto the Stackage website. Let's lookup GHC 7.10.3, and we'll find that the last resolver for that was 6.35. Given this information, we can then set the resolver in our stack.yaml file. We'll then run stack build again. We'll find it uses different versions of certain packages for servant! For instance, servant itself is now version 0.7.1. Meanwhile the dependency on directory is gone entirely!

Extra Dependencies

Now let's suppose we don't want to write our program using the Servant library. Let's suppose we want to use Spock instead, like we did recently. When we first try to add Spock as a dependency in our .cabal file, we'll actually get an error. It looks like this:

In the dependencies for SpockExample-0.1.0.0:                                                                                     Spock needed, but the stack configuration has no specified version  (latest matching version                                                is 0.13.0.0)

Stack then recommends we add Spock as an extra dependency in stack.yaml. Why do we need to do this? (Get ready for some rhyming). We can't depend on Stackage to contain every package that lives on Hackage. After all, pretty much anyone can publish to Hackage! As you add more libraries, it's more work to ensure they are conflict free.

Often, updates will introduce new conflicts. And often, the original library's authors are no longer maintaining the package. This means they won't release an update to fix it. Thus, the package gets dropped from the latest resolver. And many packages aren't used enough to justify the effort of keeping them in the resolver.

But this is OK! We can still introduce Spock into our Stack program. We'll go to our stack.yaml file and add it under our extra-deps part of the file:

extra-deps:
- Spock-0.13.0.0

This however, leads us to more dependencies we must add:

Spock-core-0.13.0.0                                                                                                - reroute-0.5.0.0

After adding these to our extra packages, everything will build!

Unfortunately, it can be an arduous process to slog through ALL the extra deps you need as a result of one library. You can use the stack solver command to list them all. If you add the --update-config flag, it will even add them to your file for you! At the time of writing though, there seems to be a bug in this feature, as it fails whenever I try to use it on Spock.

Be warned though. Extra dependencies have no guards against conflicts. Packages within the resolver still won't conflict. But every new extra package you introduce brings some more risk. Sometimes you'll need to play version tetris to get things to work. Sometimes you may need to try a different library altogether.

Different Kinds of Packages

Changing gears a bit, the stack.yaml file allows you to specify different packages within your project. Each package is a self-contained Haskell unit containing its own .cabal file. The auto-generated stack.yaml always has this simple packages section:

packages:
- .

One option for what to use as a package is a local directory, relative to wherever the stack.yaml file lives. So the default is the directory itself. But at a certain point, it might make sense for you to break your project into more pieces. This way, they can be independently maintained and tested. You might have sub-packages that look a bit like this:

packages:
- ‘./my-project-core'
- ‘./my-project-server'
- ‘./my-project-client'
- ‘./my-project-db'

You can use other options besides local directories as well. If you have a package stored on a remote server as a tar file, you can reference that:

packages:
- ‘.'
- https://mysite.com/my-project-client-1.0.0.tar.gz

Stack will download the code as necessary and build it. The other common option you'll use is a Github repository. You'll often want to reference a specific commit hash to use. Here's what that would look like:

packages:
- ‘.'
- location:
  git: https://github.com/my-user/my-project.git
  commit: b7deadc0def7128384

This technique is especially useful when you need to fix bugs on a dependency. The normal release process on a library can take a long time. And the library's maintainers might not have time to review your fix. But you can supply your own code and then reference it through Github. Say you want to fix something in Servant. You can make your own fork of the repository, fix the bug, and use that as a package in your project:

packages:
- ‘.'
- location:
  git: https://github.com/jhb563/servant.git
  commit: b7deadc0def7128384

Other Fields

That covers most of what you'll want to do in the Stack file. There are other fields. For instance, the flags field allows you to override certain build flags for packages. Here's an example covered in the docs. The yackage package typically builds with the flag upload. If you're using it as a package or a dependency, you can set this flag in the stack.yaml file:

flags:
  yackage:
    upload: true

But if you want to set it to false, you can do this as well by flipping the flag there.

You can also use the extra-package-dbs field. This is necessary if you need a specialized set of libraries that aren't on Hackage. You can create your own local database if you like and store modified versions of packages there. This feature is pretty advanced so it's unlikely most of you will need it.

Conclusion

Using Stack is easy at a basic level. For starter projects, you probably won't have to change the stack.yaml file much at all. At most you'll add a couple extra dependencies. But as you make more complicated things, you'll need some extra features. You'll need to know how Stack resolves conflicts and how you can bring in code from different places. These small extra features are important to your growth as a Haskell developer.

If you've never learned the basics of Stack, you're in luck! You can take our free Stack mini-course! If you've never learned Haskell at all, nows the time to start! Download our Beginners Checklist to start your journey!