Watching the Filesystem in Racket
Update - changed the tool used and watch code
Originally, this post used a tool called
fswatch to monitor the file
fswatch has a
crippling bug on
some Linux systems where it detects file reads as file changes.
The bug has gone unaddressed for several months. It causes a lot of
problems - in my static site generator, the
-w option will build
once and wait for an event fine, but as soon as it sees a change it
will start building the site over and over again without any changes,
as it detects the generator reading the source files and triggers a
new build, which repeats ad infinitum.
I’ve thus replaced the tool used for monitoring directory changes with
watchexec, a similar tool,
also cross-platform, written in Rust. While I haven’t been able to
test this on platforms other than Linux, it doesn’t have any known
bugs of that variety, and
fswatch's bug made it completely unusable
on what has become my primary platform.
I’ve been building a new site lately, which will likely be inaugurated with this very post. My tech decisions for the site are a whole post of their own, but suffice it to say I’m using a static site generator I wrote in Racket, and it’s going swimmingly.
As I’m working on both the generator and the site at the same time, most of the time I have a pain point while writing I can fix it myself. (This is not necessarily a good thing, as I’d likely get more writing done if I wasn’t spending so much time on the generator). One of the biggest things I want out of the generator is live file watching, so when I change files it automatically updates the site.
There aren’t any existing libraries that I could find for watching a filesystem for changes, but this presented an opportunity to learn how to implement an oft-used pattern of mine: shelling out.
You see, there’s a cross-platform, very simple file change monitor called watchexec. Simple as in "given a directory (searched recursively), whenever a file is changed, run a command". Simple is particularly nice in this case because it means it will be easy to interpret the output – all I need to do is figure out how to watch a path.
Spawning the process
Looking at the documentation for process, it appears
that we can spawn a process and get its input and output ports.
Let’s write a function that will let us watch a list of paths (this
;; `watch' requires that the `watchexec' command is installed. (define (watch-paths paths) (let ((paths-string (string-join (map (lambda (path) ;; Quote paths to avoid escaping problems (string-append "-w \"" ;; Accept either strings or path objects (cond [(string? path) path] [(path? path) (path->string path)] [else ""]) "\" ")) paths) " "))) (process (string-append "watchexec " paths-string "-- echo \"File changed.\""))))
We can run
watchexec at the command line to check the output[^2]:
$ watchexec -w ~/dev/personal-site -- echo "File changed." File changed. File changed.
Now we need to kick off a build when the watch tools produces output.
Fortunately Racket makes it very easy to call a callback when a port
produces output, using handle-evt. It’s as simple as
(handle-evt <event object> <event handler). We do need to explicitly
create a loop and a channel we can use to break from it, as well.
(define (watch site-directory) (letrec (;; Start the watch process (watcher (watch-paths (list (build-path site-directory "src")))) ;; This is the port we use to read the watch tool's output (watcher-output (list-ref watcher 0)) ;; Putting a message to this channel allows us to exit the ;; watch loop programatically (exit-channel (make-channel)) ;; `process' requires that its IO ports are manually ;; closed, so we do so in the exit channel's handler (exit-handler (lambda (val) (close-input-port watcher-output) (close-output-port (list-ref watcher 1)))) ;; This function handles the events fired by the watcher ;; output port (input-handler (lambda (val) (let ((output (read-line watcher-output))) (cond ;; If the watch tool stops, we'll get an EOF in the ;; port. When this happens we need to quit [(eof-object? output) (channel-put exit-channel 'done) (loop "")] [else (loop output)])))) ;; This is the function that will loop and wait for events (loop (lambda ([files "No"]) ;; We don't want to build the site when exiting, where ;; we'll be passed an empty string (and (not (string=? files "")) ;; Build the site (displayln "") (displayln (string-append files " files changed, rebuilding...")) (build site-directory) (displayln "Built!")) ;; `sync' waits for either the input port or the exit ;; channel to send an event (sync (handle-evt watcher-output input-handler) (handle-evt exit-channel exit-handler)))) ;; Kick off the loop (watch-thread (thread (lambda () (loop))))) ;; Callers may need to be able to access both the thread (so they ;; can wait on it if necessary), and the exit channel (so they can ;; programatically stop the loop). `((thread . ,watch-thread) (exit-ch . ,exit-channel))))
Watching from the command line
We’re almost done! At this point we can call
(watch <site-directory>) and it will call the
build function every time
a file changes. However, we’d rather be able to do this without having
to enter the Racket REPL every time.
Fortunately creating a command line wrapper for this function is quite
easy. All we need to do is create a new file with no
and put a hashbang line that points to the racket executable at the
#!/usr/local/bin/racket #lang racket (require "build.rkt") (define watch? (make-parameter #f)) (define args (command-line #:once-each [("-w") "watch" (watch? #t)] #:args (str) str)) (if (watch?) (let ((watcher (watch args))) (thread-wait (dict-ref watcher 'thread))) (build args))
thread-wait waits for the loop thread we spawn before exiting
the process. Without this, the script would exit immediately, which
defeats the purpose.
If you’d like to leave a comment, please email firstname.lastname@example.org
If you’re not familiar with ports in the Scheme family, they’re essentially data queues that have a first-class representation. ↩