Watching the Filesystem in Racket

Update - changed the tool used and watch code

Originally, this post used a tool called fswatch to monitor the file system. Unfortunately, fswatch has a bug on some Linux systems where it detects file reads as file changes. The bug has gone unaddressed for several months. It causes a lot of problems - in my static site generator, the -w option will build once and wait for an event fine, but as soon as it sees a change it will start building the site over and over again without any changes, as it detects the generator reading the source files and triggers a new build, which repeats ad infinitum.

I’ve thus replaced the tool used for monitoring directory changes with watchexec, a similar tool, also cross-platform, written in Rust. While I haven’t been able to test this on platforms other than Linux, it doesn’t have any known bugs of that variety, and fswatch's bug made it completely unusable on what has become my primary platform.

Updated Post

I’ve been building a new site lately, which will likely be inaugurated with this very post. My tech decisions for the site are a whole post of their own, but suffice it to say I’m using a static site generator I wrote in Racket, and it’s going swimmingly.

As I’m working on both the generator and the site at the same time, most of the time I have a pain point while writing I can fix it myself. (This is not necessarily a good thing, as I’d likely get more writing done if I wasn’t spending so much time on the generator). One of the biggest things I want out of the generator is live file watching, so when I change files it automatically updates the site.

There aren’t any existing libraries that I could find for watching a filesystem for changes, but this presented an opportunity to learn how to implement an oft-used pattern of mine: shelling out.

You see, there’s a cross-platform, very simple file change monitor called watchexec. Simple as in "given a directory (searched recursively), whenever a file is changed, run a command". Simple is particularly nice in this case because it means it will be easy to interpret the output – all I need to do is figure out how to watch a path.

Spawning the process

Looking at the documentation for process, it appears that we can spawn a process and get its input and output ports^[1]. Let’s write a function that will let us watch a list of paths (this requires racket/string and racket/system):

;; `watch' requires that the `watchexec' command is installed.
(define (watch-paths paths)
  (let ((paths-string
         (string-join
          (map (lambda (path)
                 ;; Quote paths to avoid escaping problems
                 (string-append
                  "-w \""
                  ;; Accept either strings or path objects
                  (cond [(string? path) path]
                        [(path? path) (path->string path)]
                        [else ""])
                  "\" "))
               paths)
          " ")))
    (process (string-append
              "watchexec " paths-string "-- echo \"File changed.\""))))

We can run watchexec at the command line to check the output[^2]:

$ watchexec -w ~/dev/personal-site -- echo "File changed."
File changed.
File changed.

Now we need to kick off a build when the watch tools produces output. Fortunately Racket makes it very easy to call a callback when a port produces output, using handle-evt. It’s as simple as (handle-evt <event object> <event handler). We do need to explicitly create a loop and a channel we can use to break from it, as well.

(define (watch site-directory)
  (letrec (;; Start the watch process
           (watcher (watch-paths (list (build-path site-directory "src"))))

           ;; This is the port we use to read the watch tool's output
           (watcher-output (list-ref watcher 0))

           ;; Putting a message to this channel allows us to exit the
           ;; watch loop programatically
           (exit-channel (make-channel))

           ;; `process' requires that its IO ports are manually
           ;; closed, so we do so in the exit channel's handler
           (exit-handler
            (lambda (val)
              (close-input-port watcher-output)
              (close-output-port (list-ref watcher 1))))

           ;; This function handles the events fired by the watcher
           ;; output port
           (input-handler
            (lambda (val)
              (let ((output (read-line watcher-output)))
                (cond
                 ;; If the watch tool stops, we'll get an EOF in the
                 ;; port. When this happens we need to quit
                 [(eof-object? output)
                  (channel-put exit-channel 'done)
                  (loop "")]
                 [else (loop output)]))))

           ;; This is the function that will loop and wait for events
           (loop
            (lambda ([files "No"])

              ;; We don't want to build the site when exiting, where
              ;; we'll be passed an empty string
              (and (not (string=? files ""))
                   ;; Build the site
                   (displayln "")
                   (displayln (string-append files " files changed, rebuilding..."))
                   (build site-directory)
                   (displayln "Built!"))

              ;; `sync' waits for either the input port or the exit
              ;; channel to send an event
              (sync (handle-evt watcher-output input-handler)
                    (handle-evt exit-channel exit-handler))))

           ;; Kick off the loop
           (watch-thread (thread (lambda () (loop)))))

    ;; Callers may need to be able to access both the thread (so they
    ;; can wait on it if necessary), and the exit channel (so they can
    ;; programatically stop the loop).
    `((thread . ,watch-thread)
      (exit-ch . ,exit-channel))))

Watching from the command line

We’re almost done! At this point we can call (watch <site-directory>) and it will call the build function every time a file changes. However, we’d rather be able to do this without having to enter the Racket REPL every time.

Fortunately creating a command line wrapper for this function is quite easy. All we need to do is create a new file with no .rkt extension and put a hashbang line that points to the racket executable at the top:

#!/usr/local/bin/racket
#lang racket

(require "build.rkt")

(define watch? (make-parameter #f))

(define args
  (command-line
   #:once-each
   [("-w") "watch" (watch? #t)]
   #:args
   (str) str))

(if (watch?)
    (let ((watcher (watch args)))
      (thread-wait (dict-ref watcher 'thread)))
    (build args))

The thread-wait waits for the loop thread we spawn before exiting the process. Without this, the script would exit immediately, which defeats the purpose.

If you’d like to leave a comment, please email benaiah@mischenko.com