modd: a flexible tool for responding to filesystem change

2016-02-11

I've just released modd, a new1 project of mine. Like its sister project devd, it's distributed as a single, self-contained binary for all major platforms - get it while it's fresh.

Modd is a simple tool that's hard to explain pithily. It triggers commands and manages daemons in response to filesystem changes - but that is a technically-correct mouthful that doesn't really convey how it is used. Part of the problem is that it is extremely flexible. In my projects it runs linters, does live code compiles, manages infrastructure daemons like databases, runs test instances of projects and is even rendering and live-reloading this blog post as I type. Modd replaces parts of tools like Gulp, Grunt, Foreman and make, but it can also augment them. For instance, one of my projects is entirely driven by a Makefile, with tasks invoked by modd on change.

At modd's core is a a file change detection library that tries to get things right for most developer work patterns. It handles temporary files, VCS directories and many pathological behaviors shown by common editors correctly (or at least tries really hard to). The change detection algorithm waits for a lull in activity, so that jobs aren't triggered in the middle of progressive processes like renders and compiles that may touch many files. The result is change detection that is less surprising and more consistent than similar projects out there. The output of the change detection algorithm is then hooked up to a very flexible way to specify commands and manage daemons, letting you specify shell scripts that trigger on file match patterns in a single config file. Finally, there are a few mod-cons. A custom terminal logging module lets modd sensibly interleave the output of possibly concurrent daemons and commands, with headings showing which command was responsible for what. Modd also has support for desktop notifications (Growl on OSX, libnotify on Linux), letting you see things like linter output and compile editors immediately.

Below, I'm going to show one quick example of how I use modd to do a live build/compile cycle for devd, a pretty standard Go project. In a future post, I'll show how I've replaced Gulp entirely for a Javascript-heavy front-end project.

Please see the modd documentation for a complete explanation of the syntax and for more examples.

Test-compile cycle for Go

On startup, modd looks for a file called modd.conf in the current directory. This file has a simple but powerful syntax - one or more blocks of commands, each of which can be triggered on changes to files matching a set of file patterns. Commands have two flavors: prep commands that run and terminate (e.g. compiling, running test suites or running linters), and daemon commands that run and keep running (e.g databases or webservers). Daemons are restarted when their block is triggered, after all prep commands have run successfully. Commands are embedded shell scripts, so shell features like redirection work, and compound, multi-step commands are common.

Here is the simple modd.conf I use to drive the test cycle for devd:

**/*.go {
    prep: go test @dirmods
}

**/*.go !**/*_test.go {
    prep: go install ./cmd/devd
    daemon +sigterm: devd -ml ./tmp
}

After the modd command, the commands execute for the first time, and modd is then ready to respond to changes. The initial output looks like this:

The config file does three things:

The one subtlety here is the @dirmods tag, which is replaced with a shell-escaped list of all directories that contain modified files. There's a similar tag - @mods - that is replaced with all matching modified files. When first run, both of these tags are replaced by all possible matches - that is, all directories containing matching files, and all matching files respectively. This means that the test suite for all the Go modules in the project is run on startup, and only for modified modules after that.

1

In fact, this is release v0.2, which slipped in before I had time to announce v0.1 on my blog.