<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://bridgetownrb.com/" version="2.2.2">Bridgetown</generator><link href="https://davidrunger.com/blog/feed.xml" rel="self" type="application/atom+xml" /><link href="https://davidrunger.com/blog/" rel="alternate" type="text/html" /><updated>2026-10-04T11:37:00-05:00</updated><id>https://davidrunger.com/blog/feed.xml</id><title type="html">David Runger : Blog</title><subtitle>I&apos;m a full stack web developer focusing on Ruby on Rails, and this is my tech blog.</subtitle><entry><title type="html">My decorator-style memoization shard for Crystal</title><link href="https://davidrunger.com/blog/my-decorator-style-memoization-shard-for-crystal" rel="alternate" type="text/html" title="My decorator-style memoization shard for Crystal" /><published>2026-08-23T00:00:00-05:00</published><updated>2026-08-23T00:00:00-05:00</updated><id>repo://posts.collection/_posts/2026-08-23-my-decorator-style-memoization-shard-for-crystal.md</id><content type="html" xml:base="https://davidrunger.com/blog/my-decorator-style-memoization-shard-for-crystal">&lt;p&gt;&lt;img src=&quot;https://david-runger-public-uploads.s3.us-east-1.amazonaws.com/memoization-fox.jpg&quot; alt=&quot;A fox memoizing stuff&quot; /&gt;&lt;/p&gt;

&lt;p&gt;I was surprised that I couldn’t find an established Crystal shard for memoization with the API I wanted. Rubyists have plenty of options, including &lt;a href=&quot;https://github.com/panorama-ed/memo_wise&quot;&gt;MemoWise&lt;/a&gt;, &lt;a href=&quot;https://github.com/matthewrudy/memoist&quot;&gt;Memoist&lt;/a&gt;, and the memoization support in &lt;a href=&quot;https://github.com/dry-rb/dry-core&quot;&gt;&lt;code class=&quot;highlighter-rouge&quot;&gt;dry-core&lt;/code&gt;&lt;/a&gt;. In Crystal, my searches mostly turned up snippets or code embedded in a larger framework.&lt;/p&gt;

&lt;p&gt;So I made &lt;a href=&quot;https://github.com/davidrunger/memoization&quot;&gt;memoization&lt;/a&gt;, a small Crystal shard that lets me write this:&lt;/p&gt;

&lt;div class=&quot;language-crystal highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;n&quot;&gt;memoize&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;def&lt;/span&gt; &lt;span class=&quot;nf&quot;&gt;current_content_sha&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;file_or_directory&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;no&quot;&gt;String&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;no&quot;&gt;String&lt;/span&gt;
  &lt;span class=&quot;c1&quot;&gt;# Calculate and return a SHA-256 digest...&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;end&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Each distinct argument gets its own cached result, and repeated calls reuse it.&lt;/p&gt;

&lt;h2 id=&quot;why-i-wanted-a-memoization-shard&quot;&gt;Why I wanted a memoization shard&lt;/h2&gt;

&lt;p&gt;Memoization is an easy optimization to understand: save a method’s result and reuse it when the same arguments are passed again.&lt;/p&gt;

&lt;p&gt;For a method without arguments, I could manage that myself with an instance variable:&lt;/p&gt;

&lt;div class=&quot;language-crystal highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;k&quot;&gt;def&lt;/span&gt; &lt;span class=&quot;nf&quot;&gt;expensive_value&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;no&quot;&gt;String&lt;/span&gt;
  &lt;span class=&quot;vi&quot;&gt;@expensive_value&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;||=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;calculate_expensive_value&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;end&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;However, that adds state-management boilerplate to a method whose meaningful job is simply to calculate a value. It also requires more careful handling when &lt;code class=&quot;highlighter-rouge&quot;&gt;nil&lt;/code&gt; or &lt;code class=&quot;highlighter-rouge&quot;&gt;false&lt;/code&gt; is valid; otherwise, the calculation can run again on every call.&lt;/p&gt;

&lt;p&gt;Arguments make manual memoization more cumbersome. Remembering &lt;code class=&quot;highlighter-rouge&quot;&gt;current_content_sha(&quot;README.md&quot;)&lt;/code&gt; independently from &lt;code class=&quot;highlighter-rouge&quot;&gt;current_content_sha(&quot;src&quot;)&lt;/code&gt; requires a hash keyed by the method’s arguments, plus lookup and assignment logic around the actual implementation.&lt;/p&gt;

&lt;p&gt;One reason I enjoy Ruby and Crystal is that they strip away boilerplate and leave expressive code. Building and maintaining a cache by hand felt like a step backward. I wanted the method definition to say what it calculates, with &lt;code class=&quot;highlighter-rouge&quot;&gt;memoize&lt;/code&gt; as the one clear declaration that its results should be reused.&lt;/p&gt;

&lt;h2 id=&quot;the-code-i-found&quot;&gt;The code I found&lt;/h2&gt;

&lt;p&gt;My path to the shard included a &lt;a href=&quot;https://forum.crystal-lang.org/t/decorator-for-caching/6221&quot;&gt;Crystal Forum thread about a caching decorator&lt;/a&gt;. Someone pointed to the &lt;a href=&quot;https://github.com/luckyframework/lucky/blob/v1.3.0/src/lucky/memoizable.cr&quot;&gt;&lt;code class=&quot;highlighter-rouge&quot;&gt;memoize&lt;/code&gt; macro in the Lucky framework&lt;/a&gt;, whose syntax was almost exactly what I wanted:&lt;/p&gt;

&lt;div class=&quot;language-crystal highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;n&quot;&gt;memoize&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;def&lt;/span&gt; &lt;span class=&quot;nf&quot;&gt;somefun&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;a&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;b&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;no&quot;&gt;Whatever&lt;/span&gt;
  &lt;span class=&quot;c1&quot;&gt;# ...&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;end&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;There was one important limitation: for methods with arguments, a call with different arguments replaced the previous cached value. The forum thread’s original poster described the same problem:&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;I’m looking to use it for dynamic programming, so I’d want to use it for all param sets, not just the latest.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;I adapted Lucky’s code so that methods with arguments use a hash keyed by argument tuples, preserving results for many sets of arguments instead of only the latest one.&lt;/p&gt;

&lt;p&gt;I was very inexperienced with Crystal macros, and getting this working was difficult and fairly bumbling. The macro has to inspect the method definition at compile time and generate a typed cache, an uncached implementation, and a public wrapper. Somewhat to my surprise, I figured it out.&lt;/p&gt;

&lt;p&gt;The code first lived in my &lt;a href=&quot;https://github.com/davidrunger/dotfiles&quot;&gt;dotfiles&lt;/a&gt;. Git history tells me that I added it in August 2024 while &lt;a href=&quot;https://github.com/davidrunger/dotfiles/commit/c710419e839fcdf25a977ffc9185dcd1ee6342c6&quot;&gt;converting a personal tool from Ruby to Crystal&lt;/a&gt;. In March 2025, I extracted it into the standalone &lt;code class=&quot;highlighter-rouge&quot;&gt;memoization&lt;/code&gt; shard so that I could reuse it elsewhere.&lt;/p&gt;

&lt;h2 id=&quot;installation-and-use&quot;&gt;Installation and use&lt;/h2&gt;

&lt;p&gt;Add the shard to &lt;code class=&quot;highlighter-rouge&quot;&gt;shard.yml&lt;/code&gt;:&lt;/p&gt;

&lt;div class=&quot;language-yaml highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;na&quot;&gt;dependencies&lt;/span&gt;&lt;span class=&quot;pi&quot;&gt;:&lt;/span&gt;
  &lt;span class=&quot;na&quot;&gt;memoization&lt;/span&gt;&lt;span class=&quot;pi&quot;&gt;:&lt;/span&gt;
    &lt;span class=&quot;na&quot;&gt;github&lt;/span&gt;&lt;span class=&quot;pi&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;davidrunger/memoization&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Then install it and require it:&lt;/p&gt;

&lt;div class=&quot;language-console highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;go&quot;&gt;shards install
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;div class=&quot;language-crystal highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;nb&quot;&gt;require&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;memoization&quot;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Add &lt;code class=&quot;highlighter-rouge&quot;&gt;memoize&lt;/code&gt; before a method definition. Both the return type and every argument type must be explicit so that the macro can construct the correctly typed cache:&lt;/p&gt;

&lt;div class=&quot;language-crystal highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;k&quot;&gt;class&lt;/span&gt; &lt;span class=&quot;nc&quot;&gt;Greeting&lt;/span&gt;
  &lt;span class=&quot;n&quot;&gt;memoize&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;def&lt;/span&gt; &lt;span class=&quot;nf&quot;&gt;message&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;nb&quot;&gt;name&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;no&quot;&gt;String&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;no&quot;&gt;String&lt;/span&gt;
    &lt;span class=&quot;nb&quot;&gt;puts&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;Generating a greeting for &lt;/span&gt;&lt;span class=&quot;si&quot;&gt;#{&lt;/span&gt;&lt;span class=&quot;nb&quot;&gt;name&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;
    &lt;span class=&quot;s2&quot;&gt;&quot;Hello, &lt;/span&gt;&lt;span class=&quot;si&quot;&gt;#{&lt;/span&gt;&lt;span class=&quot;nb&quot;&gt;name&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;!&quot;&lt;/span&gt;
  &lt;span class=&quot;k&quot;&gt;end&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;end&lt;/span&gt;

&lt;span class=&quot;n&quot;&gt;greeting&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;no&quot;&gt;Greeting&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;new&lt;/span&gt;

&lt;span class=&quot;n&quot;&gt;greeting&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;message&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;Jane&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;c1&quot;&gt;# Prints &quot;Generating a greeting for Jane&quot; and returns &quot;Hello, Jane!&quot;&lt;/span&gt;

&lt;span class=&quot;n&quot;&gt;greeting&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;message&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;John&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;c1&quot;&gt;# Prints &quot;Generating a greeting for John&quot; and returns &quot;Hello, John!&quot;&lt;/span&gt;

&lt;span class=&quot;n&quot;&gt;greeting&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;message&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;Jane&quot;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;c1&quot;&gt;# Returns the cached &quot;Hello, Jane!&quot; without printing anything&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;The cache belongs to the object instance, with a separate entry for each argument combination. The shard also supports methods without arguments, &lt;code class=&quot;highlighter-rouge&quot;&gt;nil&lt;/code&gt; results, default and named arguments, private methods, and method names ending in &lt;code class=&quot;highlighter-rouge&quot;&gt;?&lt;/code&gt; or &lt;code class=&quot;highlighter-rouge&quot;&gt;!&lt;/code&gt;.&lt;/p&gt;

&lt;h2 id=&quot;where-i-use-it&quot;&gt;Where I use it&lt;/h2&gt;

&lt;p&gt;I use the shard throughout the Crystal tools in my dotfiles. For example, one method calculates a SHA-256 digest for a file or directory. Calls for the same path reuse that work while other paths are cached independently.&lt;/p&gt;

&lt;p&gt;I also use it in &lt;a href=&quot;https://github.com/davidrunger/skedjewel&quot;&gt;Skedjewel&lt;/a&gt;, a scheduler for Sidekiq jobs. Skedjewel memoizes parsed schedule values and its lock manager. I use Skedjewel to run scheduled jobs for the &lt;a href=&quot;https://github.com/davidrunger/david_runger&quot;&gt;david_runger&lt;/a&gt; Rails app, so this is “production” use in the sense that it runs my hobby app, not evidence from a large commercial deployment.&lt;/p&gt;

&lt;h2 id=&quot;limitations&quot;&gt;Limitations&lt;/h2&gt;

&lt;p&gt;This is a small library that solves the problem I have, not a comprehensive caching system. In particular:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Cached entries have no expiration, eviction, or reset mechanism.&lt;/li&gt;
  &lt;li&gt;A method called with an unbounded number of distinct argument combinations will grow its cache without bound.&lt;/li&gt;
  &lt;li&gt;The shard does not promise thread safety.&lt;/li&gt;
  &lt;li&gt;It memoizes instance methods, with caches scoped to individual object instances.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;A more sophisticated implementation might address these limitations. I haven’t needed those features, so I have kept the shard simple.&lt;/p&gt;

&lt;p&gt;If that tradeoff fits your program, you can find the source, installation instructions, and complete examples at &lt;a href=&quot;https://github.com/davidrunger/memoization&quot;&gt;github.com/davidrunger/memoization&lt;/a&gt;.&lt;/p&gt;</content><author><name>David Runger</name></author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://david-runger-public-uploads.s3.us-east-1.amazonaws.com/memoization-fox.jpg" /><media:content medium="image" url="https://david-runger-public-uploads.s3.us-east-1.amazonaws.com/memoization-fox.jpg" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Easy PostgreSQL major version upgrades for a Docker Compose hobby app</title><link href="https://davidrunger.com/blog/easy-postgresql-major-version-upgrades-for-a-docker-compose-hobby-app" rel="alternate" type="text/html" title="Easy PostgreSQL major version upgrades for a Docker Compose hobby app" /><published>2026-01-22T00:00:00-06:00</published><updated>2026-01-22T00:00:00-06:00</updated><id>repo://posts.collection/_posts/2026-01-22-easy-postgresql-major-version-upgrades-for-a-docker-compose-hobby-app.md</id><content type="html" xml:base="https://davidrunger.com/blog/easy-postgresql-major-version-upgrades-for-a-docker-compose-hobby-app">&lt;p&gt;For a small hobby app, a logical dump and restore is a simple, low-risk way to upgrade PostgreSQL. The old database remains untouched until you delete its Docker volume, making rollback straightforward.&lt;/p&gt;

&lt;p&gt;This is the runbook that I use for my Docker Compose app. I paste each command into an SSH session individually and check its output before continuing.&lt;/p&gt;

&lt;h2 id=&quot;caveats&quot;&gt;Caveats&lt;/h2&gt;

&lt;p&gt;This approach requires downtime and assumes that the database is small enough to dump and restore quickly (mine is about 122 MB). For a large database or strict uptime requirements, consider &lt;code class=&quot;highlighter-rouge&quot;&gt;pg_upgrade&lt;/code&gt; or a replication-based cutover instead.&lt;/p&gt;

&lt;p&gt;The examples also assume that:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;PostgreSQL runs through Docker Compose.&lt;/li&gt;
  &lt;li&gt;Its data lives in a named Docker volume.&lt;/li&gt;
  &lt;li&gt;You can SSH into the server and run commands interactively.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Adapt the service, database, user, and volume names to your app.&lt;/p&gt;

&lt;h2 id=&quot;before-the-upgrade&quot;&gt;Before the upgrade&lt;/h2&gt;

&lt;p&gt;Confirm that your application supports the new PostgreSQL version. Review its release notes for relevant breaking changes and run your test suite against it.&lt;/p&gt;

&lt;p&gt;Prepare a pull request that updates the PostgreSQL image, volume name, and, if necessary, volume mount path. During the upgrade, you will apply those changes temporarily, revert them, and then merge the pull request to make them permanent.&lt;/p&gt;

&lt;h2 id=&quot;upgrade-from-postgresql-17-to-18&quot;&gt;Upgrade from PostgreSQL 17 to 18&lt;/h2&gt;

&lt;h3 id=&quot;1-record-the-current-state&quot;&gt;1. Record the current state&lt;/h3&gt;

&lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;git status
git show
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;docker compose &lt;span class=&quot;nb&quot;&gt;exec &lt;/span&gt;postgres psql &lt;span class=&quot;nt&quot;&gt;-U&lt;/span&gt; app_user app_production &lt;span class=&quot;nt&quot;&gt;-c&lt;/span&gt; &lt;span class=&quot;s1&quot;&gt;&apos;SELECT VERSION();&apos;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;docker compose &lt;span class=&quot;nb&quot;&gt;exec &lt;/span&gt;postgres psql &lt;span class=&quot;nt&quot;&gt;-U&lt;/span&gt; app_user app_production &lt;span class=&quot;nt&quot;&gt;-c&lt;/span&gt; &lt;span class=&quot;se&quot;&gt;\&lt;/span&gt;
  &lt;span class=&quot;s1&quot;&gt;&apos;SELECT COUNT(*) FROM users; SELECT * FROM orders ORDER BY created_at DESC LIMIT 1;&apos;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;The query results provide a baseline for verification after the restore.&lt;/p&gt;

&lt;h3 id=&quot;2-create-an-off-server-backup&quot;&gt;2. Create an off-server backup&lt;/h3&gt;

&lt;p&gt;If you have a separate backup pipeline, run it now:&lt;/p&gt;

&lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;bin/backup-to-s3.sh
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;This backup provides another recovery path if the server itself fails during the upgrade.&lt;/p&gt;

&lt;h3 id=&quot;3-update-the-docker-configuration&quot;&gt;3. Update the Docker configuration&lt;/h3&gt;

&lt;p&gt;Update &lt;code class=&quot;highlighter-rouge&quot;&gt;docker-compose.yml&lt;/code&gt; so that:&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;The image changes from &lt;code class=&quot;highlighter-rouge&quot;&gt;postgres:17.x&lt;/code&gt; to &lt;code class=&quot;highlighter-rouge&quot;&gt;postgres:18.x&lt;/code&gt;.&lt;/li&gt;
  &lt;li&gt;The volume name changes from &lt;code class=&quot;highlighter-rouge&quot;&gt;postgres-data-v17&lt;/code&gt; to &lt;code class=&quot;highlighter-rouge&quot;&gt;postgres-data-v18&lt;/code&gt;.&lt;/li&gt;
  &lt;li&gt;The volume mount path changes if required by the new image.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;A new volume name gives the new PostgreSQL version an empty data directory and preserves the old database for rollback.&lt;/p&gt;

&lt;p&gt;For example:&lt;/p&gt;

&lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;nb&quot;&gt;sed&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;-i&lt;/span&gt;&lt;span class=&quot;s1&quot;&gt;&apos;&apos;&lt;/span&gt; &lt;span class=&quot;s1&quot;&gt;&apos;s/postgres:17.6-alpine/postgres:18.0-alpine/g&apos;&lt;/span&gt; docker-compose.yml
&lt;span class=&quot;nb&quot;&gt;sed&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;-i&lt;/span&gt;&lt;span class=&quot;s1&quot;&gt;&apos;&apos;&lt;/span&gt; &lt;span class=&quot;s1&quot;&gt;&apos;s/postgres-data-v17:/postgres-data-v18:/g&apos;&lt;/span&gt; docker-compose.yml
&lt;span class=&quot;nb&quot;&gt;sed&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;-i&lt;/span&gt;&lt;span class=&quot;s1&quot;&gt;&apos;&apos;&lt;/span&gt; &lt;span class=&quot;s1&quot;&gt;&apos;s|/var/lib/postgresql/data|/var/lib/postgresql|g&apos;&lt;/span&gt; docker-compose.yml
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Then review the changes:&lt;/p&gt;

&lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;git status
git diff
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Only the image tag, volume name, and mount path should have changed.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;PostgreSQL 18 image note (updated July 2026):&lt;/strong&gt; The official image now sets &lt;code class=&quot;highlighter-rouge&quot;&gt;PGDATA&lt;/code&gt; to &lt;code class=&quot;highlighter-rouge&quot;&gt;/var/lib/postgresql/18/docker&lt;/code&gt; and expects the volume to be mounted at &lt;code class=&quot;highlighter-rouge&quot;&gt;/var/lib/postgresql&lt;/code&gt;. Earlier PostgreSQL images used &lt;code class=&quot;highlighter-rouge&quot;&gt;/var/lib/postgresql/data&lt;/code&gt;. This change reached the PostgreSQL 18 image through several revisions, so my upgrades from 17 to 18.0 and 18.0 to 18.1 still worked with the old mount path, while 18.1 to 18.4 required another dump and restore. Check the &lt;a href=&quot;https://hub.docker.com/_/postgres&quot;&gt;official image documentation&lt;/a&gt; before any image update, including a minor-version update.&lt;/p&gt;

&lt;h3 id=&quot;4-pull-the-new-image&quot;&gt;4. Pull the new image&lt;/h3&gt;

&lt;p&gt;Pull it before the downtime begins:&lt;/p&gt;

&lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;docker compose pull postgres
docker images postgres
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h3 id=&quot;5-stop-application-services&quot;&gt;5. Stop application services&lt;/h3&gt;

&lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;docker compose stop web worker nginx
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Nothing should write to PostgreSQL after this point.&lt;/p&gt;

&lt;h3 id=&quot;6-dump-the-old-database-cluster&quot;&gt;6. Dump the old database cluster&lt;/h3&gt;

&lt;p&gt;&lt;code class=&quot;highlighter-rouge&quot;&gt;pg_dumpall&lt;/code&gt; creates a portable SQL script containing all databases and global objects, including roles and permissions. A complete dump and restore generally require a database superuser. Check your app user:&lt;/p&gt;

&lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;docker compose &lt;span class=&quot;nb&quot;&gt;exec &lt;/span&gt;postgres psql &lt;span class=&quot;nt&quot;&gt;-U&lt;/span&gt; app_user &lt;span class=&quot;nt&quot;&gt;-c&lt;/span&gt; &lt;span class=&quot;se&quot;&gt;\&lt;/span&gt;
  &lt;span class=&quot;s1&quot;&gt;&apos;SELECT rolsuper FROM pg_roles WHERE rolname = current_user;&apos;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;If the command returns &lt;code class=&quot;highlighter-rouge&quot;&gt;true&lt;/code&gt; or &lt;code class=&quot;highlighter-rouge&quot;&gt;t&lt;/code&gt;, create and inspect the dump:&lt;/p&gt;

&lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;docker compose &lt;span class=&quot;nb&quot;&gt;exec &lt;/span&gt;postgres pg_dumpall &lt;span class=&quot;nt&quot;&gt;-U&lt;/span&gt; app_user &lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt; backup.sql
&lt;span class=&quot;nb&quot;&gt;head&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;-5&lt;/span&gt; backup.sql | &lt;span class=&quot;nb&quot;&gt;grep&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;-q&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;PostgreSQL&quot;&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class=&quot;nb&quot;&gt;echo&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;Backup format looks correct&quot;&lt;/span&gt;
&lt;span class=&quot;nb&quot;&gt;ls&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;-lh&lt;/span&gt; backup.sql
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Confirm that the file size is plausible before continuing.&lt;/p&gt;

&lt;h3 id=&quot;7-replace-postgresql&quot;&gt;7. Replace PostgreSQL&lt;/h3&gt;

&lt;p&gt;Stop and remove the old PostgreSQL container, then start the service with its new image and volume:&lt;/p&gt;

&lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;docker compose down postgres
docker compose up &lt;span class=&quot;nt&quot;&gt;--detach&lt;/span&gt; postgres
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Verify the new version:&lt;/p&gt;

&lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;docker compose &lt;span class=&quot;nb&quot;&gt;exec &lt;/span&gt;postgres psql &lt;span class=&quot;nt&quot;&gt;-U&lt;/span&gt; app_user &lt;span class=&quot;nt&quot;&gt;-c&lt;/span&gt; &lt;span class=&quot;s1&quot;&gt;&apos;SELECT VERSION();&apos;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h3 id=&quot;8-restore-and-verify-the-data&quot;&gt;8. Restore and verify the data&lt;/h3&gt;

&lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;docker compose &lt;span class=&quot;nb&quot;&gt;exec&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;--no-TTY&lt;/span&gt; postgres psql &lt;span class=&quot;nt&quot;&gt;-U&lt;/span&gt; app_user &amp;lt; backup.sql
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Because the new container’s initialization may already have created the app role and database, the restore can report that they already exist. Review the complete output for any other errors.&lt;/p&gt;

&lt;p&gt;Run the same verification query as before:&lt;/p&gt;

&lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;docker compose &lt;span class=&quot;nb&quot;&gt;exec &lt;/span&gt;postgres psql &lt;span class=&quot;nt&quot;&gt;-U&lt;/span&gt; app_user app_production &lt;span class=&quot;nt&quot;&gt;-c&lt;/span&gt; &lt;span class=&quot;se&quot;&gt;\&lt;/span&gt;
  &lt;span class=&quot;s1&quot;&gt;&apos;SELECT COUNT(*) FROM users; SELECT * FROM orders ORDER BY created_at DESC LIMIT 1;&apos;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;The results should match the baseline.&lt;/p&gt;

&lt;h3 id=&quot;9-restart-and-test-the-application&quot;&gt;9. Restart and test the application&lt;/h3&gt;

&lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;docker compose up &lt;span class=&quot;nt&quot;&gt;-d&lt;/span&gt; web worker nginx
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Confirm that the app loads and can write to the database, such as by submitting a form that creates a test record.&lt;/p&gt;

&lt;h3 id=&quot;10-deploy-the-permanent-configuration&quot;&gt;10. Deploy the permanent configuration&lt;/h3&gt;

&lt;p&gt;Revert the temporary edit:&lt;/p&gt;

&lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;git restore docker-compose.yml
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Merge and deploy the prepared pull request. Then confirm that the app still uses the new PostgreSQL version and that its data remains intact:&lt;/p&gt;

&lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;docker compose &lt;span class=&quot;nb&quot;&gt;exec &lt;/span&gt;postgres psql &lt;span class=&quot;nt&quot;&gt;-U&lt;/span&gt; app_user app_production &lt;span class=&quot;nt&quot;&gt;-c&lt;/span&gt; &lt;span class=&quot;s1&quot;&gt;&apos;SELECT VERSION();&apos;&lt;/span&gt;
docker compose &lt;span class=&quot;nb&quot;&gt;exec &lt;/span&gt;postgres psql &lt;span class=&quot;nt&quot;&gt;-U&lt;/span&gt; app_user app_production &lt;span class=&quot;nt&quot;&gt;-c&lt;/span&gt; &lt;span class=&quot;se&quot;&gt;\&lt;/span&gt;
  &lt;span class=&quot;s1&quot;&gt;&apos;SELECT COUNT(*) FROM users; SELECT * FROM orders ORDER BY created_at DESC LIMIT 1;&apos;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h3 id=&quot;11-remove-the-rollback-data&quot;&gt;11. Remove the rollback data&lt;/h3&gt;

&lt;p&gt;After you are confident that the deployment succeeded:&lt;/p&gt;

&lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;nb&quot;&gt;rm &lt;/span&gt;backup.sql
docker volume &lt;span class=&quot;nb&quot;&gt;rm &lt;/span&gt;app_postgres-data-v17
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h2 id=&quot;rolling-back&quot;&gt;Rolling back&lt;/h2&gt;

&lt;p&gt;Until you remove the old volume, you can return to PostgreSQL 17:&lt;/p&gt;

&lt;div class=&quot;language-bash highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;docker compose down
git restore docker-compose.yml
docker compose up &lt;span class=&quot;nt&quot;&gt;-d&lt;/span&gt; postgres
docker compose up &lt;span class=&quot;nt&quot;&gt;-d&lt;/span&gt; web worker nginx
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h2 id=&quot;complete-example&quot;&gt;Complete example&lt;/h2&gt;

&lt;p&gt;I use this process for the &lt;a href=&quot;https://davidrunger.com/&quot;&gt;davidrunger.com&lt;/a&gt; database. See the &lt;a href=&quot;https://github.com/davidrunger/david_runger/blob/main/docs/postgres-17-to-18-upgrade.sh&quot;&gt;complete playbook&lt;/a&gt; and its &lt;a href=&quot;https://github.com/davidrunger/david_runger/blob/main/docker-compose.yml&quot;&gt;&lt;code class=&quot;highlighter-rouge&quot;&gt;docker-compose.yml&lt;/code&gt;&lt;/a&gt; for a concrete example with my service names, deployment flow, verification queries, and volume naming scheme.&lt;/p&gt;</content><author><name>David Runger</name></author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://davidrunger.com/blog/images/davidrunger.jpg" /><media:content medium="image" url="https://davidrunger.com/blog/images/davidrunger.jpg" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Automatically compiling Crystal for development tooling</title><link href="https://davidrunger.com/blog/automatically-compiling-crystal-for-development-tooling" rel="alternate" type="text/html" title="Automatically compiling Crystal for development tooling" /><published>2024-08-08T00:00:00-05:00</published><updated>2024-08-08T00:00:00-05:00</updated><id>repo://posts.collection/_posts/2024-08-08-automatically-compiling-crystal-for-development-tooling.md</id><content type="html" xml:base="https://davidrunger.com/blog/automatically-compiling-crystal-for-development-tooling">&lt;h2 id=&quot;why-crystal&quot;&gt;Why Crystal?&lt;/h2&gt;

&lt;p&gt;I write many small command-line tools for my own development workflow. For example, my &lt;code class=&quot;highlighter-rouge&quot;&gt;gform&lt;/code&gt; command fetches a Git repository and rebases the current branch onto the updated &lt;code class=&quot;highlighter-rouge&quot;&gt;main&lt;/code&gt; branch. It also runs &lt;code class=&quot;highlighter-rouge&quot;&gt;install-packages-in-background&lt;/code&gt;, another tool that checks dependency lock files and installs any changed dependencies in the background.&lt;/p&gt;

&lt;p&gt;Most of my small tools are bash scripts. Bash starts quickly:&lt;/p&gt;

&lt;div class=&quot;language-console highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;go&quot;&gt;❯ time bash -c &apos;echo &quot;Hi!&quot;&apos;
Hi!
bash -c &apos;echo &quot;Hi!&quot;&apos;  0.00s user 0.00s system 89% cpu 0.006 total
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Bash becomes unpleasant for more complicated logic, though, and it lacks a package ecosystem comparable to those of many general-purpose languages. Ruby and Node are more expressive, but their startup time is noticeable for frequently invoked tools. On my machine, even minimal programs took about 130 milliseconds with Ruby and 200 milliseconds with Node.&lt;/p&gt;

&lt;p&gt;I also tried Lua, which starts as quickly as bash and has a package manager, but its limited standard library left me implementing basic functionality that I expected the language to provide.&lt;/p&gt;

&lt;p&gt;A compiled language offered fast programs and richer language features, but introduced a different inconvenience: I would need to recompile each tool after changing its source and make the resulting binary available on my &lt;code class=&quot;highlighter-rouge&quot;&gt;PATH&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;ChatGPT and I devised a setup that handles both tasks automatically. Although I use it with Crystal, the same approach should work with other compiled languages such as Rust or Go.&lt;/p&gt;

&lt;p&gt;The code lives in my &lt;a href=&quot;https://github.com/davidrunger/dotfiles&quot;&gt;dotfiles repository&lt;/a&gt;. Its &lt;a href=&quot;https://github.com/davidrunger/dotfiles/blob/main/bin/run-crystal-program&quot;&gt;current runner&lt;/a&gt; and &lt;a href=&quot;https://github.com/davidrunger/dotfiles/blob/main/bin/symlink-crystal-programs&quot;&gt;symlink setup script&lt;/a&gt; are the source of truth, so readers can see how the setup has evolved since this post was published.&lt;/p&gt;

&lt;h2 id=&quot;why-i-chose-crystal&quot;&gt;Why I chose Crystal&lt;/h2&gt;

&lt;p&gt;&lt;img src=&quot;https://david-runger-public-uploads.s3.amazonaws.com/crystal.png&quot; alt=&quot;Crystal logo&quot; /&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;https://crystal-lang.org/&quot;&gt;Crystal&lt;/a&gt;&lt;/strong&gt; is a compiled, statically typed language with Ruby-inspired syntax. Its type inference means that explicit type annotations are often unnecessary, while its compiler produces efficient native programs.&lt;/p&gt;

&lt;p&gt;I also appreciate Crystal’s clear compiler errors and approachable &lt;a href=&quot;https://crystal-lang.org/reference/latest/&quot;&gt;documentation&lt;/a&gt;. It gives me much of what I enjoy about writing Ruby while producing fast, memory-efficient executables.&lt;/p&gt;

&lt;h2 id=&quot;example-my-unique-union-program&quot;&gt;Example: my &lt;code class=&quot;highlighter-rouge&quot;&gt;unique-union&lt;/code&gt; program&lt;/h2&gt;

&lt;p&gt;As an example, my &lt;code class=&quot;highlighter-rouge&quot;&gt;unique-union&lt;/code&gt; program combines the words in two arguments, removes duplicates, and sorts the result:&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;❯ unique-union &quot;wave hello bye&quot; &quot;hello ocean wave&quot;
bye
hello
ocean
wave
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;I use this small program as a building block in another tool.&lt;/p&gt;

&lt;p&gt;Here is its Crystal source:&lt;/p&gt;

&lt;div class=&quot;language-cr highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;c1&quot;&gt;#!/usr/bin/env crystal&lt;/span&gt;

&lt;span class=&quot;c1&quot;&gt;# file location: ~/code/dotfiles/crystal-programs/unique-union.cr&lt;/span&gt;

&lt;span class=&quot;k&quot;&gt;def&lt;/span&gt; &lt;span class=&quot;nf&quot;&gt;unique_union&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;set1&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;no&quot;&gt;String&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;set2&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;no&quot;&gt;String&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;no&quot;&gt;Array&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;no&quot;&gt;String&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
  &lt;span class=&quot;n&quot;&gt;set1_words&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;set1&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;split&lt;/span&gt;
  &lt;span class=&quot;n&quot;&gt;set2_words&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;set2&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;split&lt;/span&gt;

  &lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;set1_words&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;|&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;set2_words&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;).&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;sort&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;end&lt;/span&gt;

&lt;span class=&quot;k&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;no&quot;&gt;ARGV&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;size&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;==&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;2&lt;/span&gt;
  &lt;span class=&quot;n&quot;&gt;unique_union&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;no&quot;&gt;ARGV&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;0&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;],&lt;/span&gt; &lt;span class=&quot;no&quot;&gt;ARGV&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;]).&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;each&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;|&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;item&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;|&lt;/span&gt; &lt;span class=&quot;nb&quot;&gt;puts&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;item&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;else&lt;/span&gt;
  &lt;span class=&quot;nb&quot;&gt;puts&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;Usage: unique-union &lt;/span&gt;&lt;span class=&quot;se&quot;&gt;\&quot;&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;first list of words&lt;/span&gt;&lt;span class=&quot;se&quot;&gt;\&quot;&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt; &lt;/span&gt;&lt;span class=&quot;se&quot;&gt;\&quot;&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;second list of words&lt;/span&gt;&lt;span class=&quot;se&quot;&gt;\&quot;&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;
  &lt;span class=&quot;nb&quot;&gt;exit&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;end&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h2 id=&quot;setting-up-automatic-crystal-compilation&quot;&gt;Setting up automatic Crystal compilation&lt;/h2&gt;

&lt;p&gt;Two shell scripts make the source available as a command and compile it when necessary.&lt;/p&gt;

&lt;h3 id=&quot;part-1-symlink-crystal-programs&quot;&gt;Part 1: &lt;code class=&quot;highlighter-rouge&quot;&gt;symlink-crystal-programs&lt;/code&gt;&lt;/h3&gt;

&lt;p&gt;My &lt;code class=&quot;highlighter-rouge&quot;&gt;~/.zshrc&lt;/code&gt; runs &lt;code class=&quot;highlighter-rouge&quot;&gt;symlink-crystal-programs&lt;/code&gt; in the background whenever I open a terminal:&lt;/p&gt;

&lt;div class=&quot;language-sh highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;c&quot;&gt;# Set up (in the background) symlinks for programs written in Crystal&lt;/span&gt;
&lt;span class=&quot;o&quot;&gt;{&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;(&lt;/span&gt; symlink-crystal-programs &lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt;&amp;amp;3 &amp;amp; &lt;span class=&quot;o&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;}&lt;/span&gt; 3&amp;gt;&amp;amp;1
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;The surrounding shell syntax preserves standard output while letting the script run asynchronously, so it does not delay shell startup.&lt;/p&gt;

&lt;p&gt;The script recreates a directory of extensionless symlinks, one for each &lt;code class=&quot;highlighter-rouge&quot;&gt;.cr&lt;/code&gt; source file. That directory is on my &lt;code class=&quot;highlighter-rouge&quot;&gt;PATH&lt;/code&gt;, and every symlink points to the same &lt;code class=&quot;highlighter-rouge&quot;&gt;run-crystal-program&lt;/code&gt; script:&lt;/p&gt;

&lt;div class=&quot;language-console highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;go&quot;&gt;❯ tree ~/bin/crystal-symlinks
/home/david/bin/crystal-symlinks
├── ...
&lt;/span&gt;&lt;span class=&quot;gp&quot;&gt;├── install-packages-in-background -&amp;gt;&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;/home/david/code/dotfiles/bin/run-crystal-program
&lt;span class=&quot;go&quot;&gt;├── ...
&lt;/span&gt;&lt;span class=&quot;gp&quot;&gt;├── open-pr-in-browser -&amp;gt;&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;/home/david/code/dotfiles/bin/run-crystal-program
&lt;span class=&quot;gp&quot;&gt;├── runger-config -&amp;gt;&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;/home/david/code/dotfiles/bin/run-crystal-program
&lt;span class=&quot;gp&quot;&gt;└── unique-union -&amp;gt;&lt;/span&gt;&lt;span class=&quot;w&quot;&gt; &lt;/span&gt;/home/david/code/dotfiles/bin/run-crystal-program
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Here is &lt;code class=&quot;highlighter-rouge&quot;&gt;symlink-crystal-programs&lt;/code&gt;:&lt;/p&gt;

&lt;div class=&quot;language-sh highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;c&quot;&gt;#!/usr/bin/env bash&lt;/span&gt;
&lt;span class=&quot;c&quot;&gt;# file name (available on PATH): symlink-crystal-programs&lt;/span&gt;

&lt;span class=&quot;nb&quot;&gt;set&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;-euo&lt;/span&gt; pipefail &lt;span class=&quot;c&quot;&gt;# exit on any error, don&apos;t allow undefined variables, pipes don&apos;t swallow errors&lt;/span&gt;

&lt;span class=&quot;nv&quot;&gt;crystal_programs_source_code_directory&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$HOME&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;/code/dotfiles/crystal-programs&quot;&lt;/span&gt;
&lt;span class=&quot;nv&quot;&gt;crystal_executable_symlinks_directory&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$HOME&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;/bin/crystal-symlinks&quot;&lt;/span&gt;

&lt;span class=&quot;c&quot;&gt;# Delete the symlinks directory, to ensure that there aren&apos;t any dangling programs left there.&lt;/span&gt;
&lt;span class=&quot;nb&quot;&gt;rm&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;-rf&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$crystal_executable_symlinks_directory&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;

&lt;span class=&quot;c&quot;&gt;# Recreate the symlinks directory.&lt;/span&gt;
&lt;span class=&quot;nb&quot;&gt;mkdir&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;-p&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$crystal_executable_symlinks_directory&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;

&lt;span class=&quot;k&quot;&gt;for &lt;/span&gt;crystal_program_source_file &lt;span class=&quot;k&quot;&gt;in&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$crystal_programs_source_code_directory&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;/&lt;span class=&quot;k&quot;&gt;*&lt;/span&gt;.cr &lt;span class=&quot;p&quot;&gt;;&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;do
  &lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;symlink_name&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;$(&lt;/span&gt;&lt;span class=&quot;nb&quot;&gt;basename&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$crystal_program_source_file&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt; .cr&lt;span class=&quot;si&quot;&gt;)&lt;/span&gt;
  &lt;span class=&quot;nb&quot;&gt;ln&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;-sf&lt;/span&gt; ~/code/dotfiles/bin/run-crystal-program &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$crystal_executable_symlinks_directory&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;/&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$symlink_name&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;done&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Dropping the &lt;code class=&quot;highlighter-rouge&quot;&gt;.cr&lt;/code&gt; extension lets me invoke &lt;code class=&quot;highlighter-rouge&quot;&gt;unique-union.cr&lt;/code&gt; as simply &lt;code class=&quot;highlighter-rouge&quot;&gt;unique-union&lt;/code&gt;.&lt;/p&gt;

&lt;h3 id=&quot;part-2-run-crystal-program&quot;&gt;Part 2: &lt;code class=&quot;highlighter-rouge&quot;&gt;run-crystal-program&lt;/code&gt;&lt;/h3&gt;

&lt;p&gt;The shared &lt;code class=&quot;highlighter-rouge&quot;&gt;run-crystal-program&lt;/code&gt; script:&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;Ensures that the Crystal source corresponding to the invoked command exists.&lt;/li&gt;
  &lt;li&gt;Compiles the source if the binary does not exist or is older than the source, the runner, or the dependency lockfile.&lt;/li&gt;
  &lt;li&gt;Explains the reason for each rebuild, including the relevant timestamps when it compares files.&lt;/li&gt;
  &lt;li&gt;Executes the binary with the supplied arguments.&lt;/li&gt;
&lt;/ol&gt;

&lt;div class=&quot;language-sh highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;c&quot;&gt;#!/usr/bin/env bash&lt;/span&gt;
&lt;span class=&quot;c&quot;&gt;# file location: ~/code/dotfiles/bin/run-crystal-program&lt;/span&gt;

&lt;span class=&quot;c&quot;&gt;# This program is intended to be symlinked as an executable file to a path with&lt;/span&gt;
&lt;span class=&quot;c&quot;&gt;# the same file name -- minus the `.cr` file extension -- as a Crystal program&lt;/span&gt;
&lt;span class=&quot;c&quot;&gt;# that is defined in `~/code/dotfiles/crystal-programs/`. When invoked, this&lt;/span&gt;
&lt;span class=&quot;c&quot;&gt;# script will compile that Crystal program, if necessary (i.e. if it has never&lt;/span&gt;
&lt;span class=&quot;c&quot;&gt;# been compiled before, if the Crystal source code or `shard.lock` has been&lt;/span&gt;
&lt;span class=&quot;c&quot;&gt;# changed subsequent to the most recent compilation, or if this compilation&lt;/span&gt;
&lt;span class=&quot;c&quot;&gt;# script has changed), and pass along any arguments to that compiled program.&lt;/span&gt;
&lt;span class=&quot;c&quot;&gt;# The compiled programs will live in `~/bin/crystal-binaries/`.&lt;/span&gt;

&lt;span class=&quot;nb&quot;&gt;set&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;-euo&lt;/span&gt; pipefail &lt;span class=&quot;c&quot;&gt;# exit on any error, don&apos;t allow undefined variables, pipes don&apos;t swallow errors&lt;/span&gt;

&lt;span class=&quot;nv&quot;&gt;crystal_compiled_binaries_directory&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$HOME&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;/bin/crystal-binaries&quot;&lt;/span&gt;

&lt;span class=&quot;nv&quot;&gt;script_name&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;$(&lt;/span&gt;&lt;span class=&quot;nb&quot;&gt;basename&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$0&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;nv&quot;&gt;source_file&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$HOME&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;/code/dotfiles/crystal-programs/&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$script_name&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;.cr&quot;&lt;/span&gt;
&lt;span class=&quot;nv&quot;&gt;binary_file&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$crystal_compiled_binaries_directory&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;/&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$script_name&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;
&lt;span class=&quot;nv&quot;&gt;compilation_script_file&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;$(&lt;/span&gt;&lt;span class=&quot;nb&quot;&gt;realpath&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$0&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;nv&quot;&gt;shard_lock_file&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$HOME&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;/code/dotfiles/shard.lock&quot;&lt;/span&gt;

print_file_timestamp&lt;span class=&quot;o&quot;&gt;()&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;{&lt;/span&gt;
  &lt;span class=&quot;nb&quot;&gt;local &lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;file&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$1&lt;/span&gt;

  &lt;span class=&quot;nb&quot;&gt;printf&lt;/span&gt; &lt;span class=&quot;s1&quot;&gt;&apos;  %s: %s\n&apos;&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$file&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;$(&lt;/span&gt;&lt;span class=&quot;nb&quot;&gt;stat&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;-c&lt;/span&gt; &lt;span class=&quot;s1&quot;&gt;&apos;%y&apos;&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$file&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt;&amp;amp;2
&lt;span class=&quot;o&quot;&gt;}&lt;/span&gt;

&lt;span class=&quot;c&quot;&gt;# Check if the source file exists.&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;[&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;!&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;-f&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$source_file&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;]&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;;&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;then
  &lt;/span&gt;&lt;span class=&quot;nb&quot;&gt;echo&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;Error: Source file &lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$source_file&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt; does not exist.&quot;&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt;&amp;amp;2
  &lt;span class=&quot;nb&quot;&gt;exit &lt;/span&gt;1
&lt;span class=&quot;k&quot;&gt;fi&lt;/span&gt;

&lt;span class=&quot;c&quot;&gt;# Check if the binary doesn&apos;t exist, the source file has changed, the&lt;/span&gt;
&lt;span class=&quot;c&quot;&gt;# compilation script has changed, or the shard lockfile has changed.&lt;/span&gt;
&lt;span class=&quot;nv&quot;&gt;compilation_reason&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;s1&quot;&gt;&apos;&apos;&lt;/span&gt;
&lt;span class=&quot;nv&quot;&gt;timestamp_comparison_files&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=()&lt;/span&gt;

&lt;span class=&quot;k&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;[&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;!&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;-f&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$binary_file&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;]&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;;&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;then
  &lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;compilation_reason&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;the compiled binary &lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$binary_file&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt; does not exist&quot;&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;elif&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;[&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$source_file&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;-nt&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$binary_file&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;]&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;;&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;then
  &lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;compilation_reason&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;the source file is newer than the compiled binary&quot;&lt;/span&gt;
  &lt;span class=&quot;nv&quot;&gt;timestamp_comparison_files&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=(&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$source_file&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$binary_file&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;elif&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;[&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$compilation_script_file&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;-nt&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$binary_file&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;]&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;;&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;then
  &lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;compilation_reason&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;the compilation script is newer than the compiled binary&quot;&lt;/span&gt;
  &lt;span class=&quot;nv&quot;&gt;timestamp_comparison_files&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=(&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$compilation_script_file&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$binary_file&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;elif&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;[&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$shard_lock_file&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;-nt&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$binary_file&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;]&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;;&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;then
  &lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;compilation_reason&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;the shard.lock file is newer than the compiled binary&quot;&lt;/span&gt;
  &lt;span class=&quot;nv&quot;&gt;timestamp_comparison_files&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=(&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$shard_lock_file&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$binary_file&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;fi

if&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;[&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;-n&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$compilation_reason&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;]&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;;&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;then&lt;/span&gt;
  &lt;span class=&quot;c&quot;&gt;# Create the compiled binaries directory.&lt;/span&gt;
  &lt;span class=&quot;nb&quot;&gt;mkdir&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;-p&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$crystal_compiled_binaries_directory&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;

  &lt;span class=&quot;c&quot;&gt;# Add shards directory for dotfiles to the CRYSTAL_PATH.&lt;/span&gt;
  &lt;span class=&quot;c&quot;&gt;# More details: https://github.com/davidrunger/dotfiles/commit/d73a9df .&lt;/span&gt;
  &lt;span class=&quot;nb&quot;&gt;export &lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;CRYSTAL_PATH&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$HOME&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;/.shards/dotfiles:/usr/share/crystal/src&quot;&lt;/span&gt;

  &lt;span class=&quot;c&quot;&gt;# Compile the binary.&lt;/span&gt;
  &lt;span class=&quot;nb&quot;&gt;echo&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;Compiling &lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$source_file&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt; because &lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$compilation_reason&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;:&quot;&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt;&amp;amp;2
  &lt;span class=&quot;k&quot;&gt;for &lt;/span&gt;file &lt;span class=&quot;k&quot;&gt;in&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;k&quot;&gt;${&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;timestamp_comparison_files&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[@]&lt;/span&gt;&lt;span class=&quot;k&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;;&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;do
    &lt;/span&gt;print_file_timestamp &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$file&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;
  &lt;span class=&quot;k&quot;&gt;done

  if&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;!&lt;/span&gt; crystal build &lt;span class=&quot;nt&quot;&gt;--warnings&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;none &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$source_file&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;-o&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$binary_file&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;;&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;then
    &lt;/span&gt;&lt;span class=&quot;nb&quot;&gt;echo&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;There was an error compiling &lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$source_file&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt; .&quot;&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt;&amp;amp;2
    &lt;span class=&quot;nb&quot;&gt;exit &lt;/span&gt;1
  &lt;span class=&quot;k&quot;&gt;fi

  &lt;/span&gt;&lt;span class=&quot;nb&quot;&gt;echo&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;Done compiling &lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$binary_file&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;.&quot;&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;&amp;gt;&lt;/span&gt;&amp;amp;2
&lt;span class=&quot;k&quot;&gt;fi&lt;/span&gt;

&lt;span class=&quot;c&quot;&gt;# Execute the compiled binary, passing along any provided arguments.&lt;/span&gt;
&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$binary_file&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$@&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;h2 id=&quot;how-the-pieces-fit-together&quot;&gt;How the pieces fit together&lt;/h2&gt;

&lt;p&gt;The setup uses three directories:&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;&lt;code class=&quot;highlighter-rouge&quot;&gt;~/code/dotfiles/crystal-programs/&lt;/code&gt; stores the Crystal source.&lt;/li&gt;
  &lt;li&gt;&lt;code class=&quot;highlighter-rouge&quot;&gt;~/bin/crystal-symlinks/&lt;/code&gt;, which is on my &lt;code class=&quot;highlighter-rouge&quot;&gt;PATH&lt;/code&gt;, stores a symlink for each program.&lt;/li&gt;
  &lt;li&gt;&lt;code class=&quot;highlighter-rouge&quot;&gt;~/bin/crystal-binaries/&lt;/code&gt; stores the compiled executables.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Invoking &lt;code class=&quot;highlighter-rouge&quot;&gt;unique-union&lt;/code&gt; resolves to &lt;code class=&quot;highlighter-rouge&quot;&gt;~/bin/crystal-symlinks/unique-union&lt;/code&gt;, which runs &lt;code class=&quot;highlighter-rouge&quot;&gt;run-crystal-program&lt;/code&gt;. The runner derives the source and binary paths from the invoked name, recompiles when any of its tracked inputs is newer, and executes the resulting binary.&lt;/p&gt;

&lt;h2 id=&quot;downsides&quot;&gt;Downsides&lt;/h2&gt;

&lt;p&gt;The first invocation after a source or dependency change pauses for a few seconds while &lt;code class=&quot;highlighter-rouge&quot;&gt;run-crystal-program&lt;/code&gt; compiles the executable. Later invocations reuse that binary and are fast.&lt;/p&gt;

&lt;p&gt;Even with a current binary, the runner adds roughly eight milliseconds of shell overhead. Pointing each command directly to its binary would avoid that cost, but would also sacrifice automatic compilation.&lt;/p&gt;

&lt;h2 id=&quot;conclusion&quot;&gt;Conclusion&lt;/h2&gt;

&lt;p&gt;For me, those costs are worthwhile: I can write development tools in Crystal and run current, compiled versions without a manual build step.&lt;/p&gt;</content><author><name>David Runger</name></author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://david-runger-public-uploads.s3.amazonaws.com/crystal-logo-stacked.png" /><media:content medium="image" url="https://david-runger-public-uploads.s3.amazonaws.com/crystal-logo-stacked.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Flaky specs due to ActionCable leakage</title><link href="https://davidrunger.com/blog/flaky-parallel-specs-caused-by-actioncable" rel="alternate" type="text/html" title="Flaky specs due to ActionCable leakage" /><published>2024-07-25T00:00:00-05:00</published><updated>2024-07-25T00:00:00-05:00</updated><id>repo://posts.collection/_posts/2024-07-25-flaky-parallel-specs-caused-by-actioncable.md</id><content type="html" xml:base="https://davidrunger.com/blog/flaky-parallel-specs-caused-by-actioncable">&lt;h2 id=&quot;when-this-can-happen&quot;&gt;When this can happen&lt;/h2&gt;

&lt;p&gt;Your specs can fail intermittently when all of the following are true:&lt;/p&gt;

&lt;ol&gt;
  &lt;li&gt;You run tests in parallel (locally and/or in CI).&lt;/li&gt;
  &lt;li&gt;Your application uses ActionCable with the Redis adapter.&lt;/li&gt;
  &lt;li&gt;The parallel test processes share a Redis instance, even if they use different Redis databases.&lt;/li&gt;
  &lt;li&gt;The &lt;code class=&quot;highlighter-rouge&quot;&gt;test&lt;/code&gt; &lt;code class=&quot;highlighter-rouge&quot;&gt;channel_prefix&lt;/code&gt; in &lt;code class=&quot;highlighter-rouge&quot;&gt;config/cable.yml&lt;/code&gt; is not unique to each process.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2 id=&quot;why-this-setup-causes-flakiness&quot;&gt;Why this setup causes flakiness&lt;/h2&gt;

&lt;p&gt;Redis Pub/Sub channels are shared by every database in an instance. If &lt;strong&gt;Spec A&lt;/strong&gt; broadcasts an ActionCable update, a simultaneously running &lt;strong&gt;Spec B&lt;/strong&gt; can receive it. In a system or feature spec, that unexpected broadcast can change the browser state and fail the spec.&lt;/p&gt;

&lt;p&gt;The failing spec offers no obvious clue because the broadcast originated in another process. Which specs overlap varies between runs, so the failure can also be difficult to reproduce.&lt;/p&gt;

&lt;h2 id=&quot;use-a-unique-channel_prefix-for-each-test-process&quot;&gt;Use a unique &lt;code class=&quot;highlighter-rouge&quot;&gt;channel_prefix&lt;/code&gt; for each test process&lt;/h2&gt;

&lt;p&gt;Set &lt;code class=&quot;highlighter-rouge&quot;&gt;channel_prefix&lt;/code&gt; to a value that differs for every test process. ActionCable then uses distinct Redis Pub/Sub channels, keeping each process’s broadcasts isolated.&lt;/p&gt;

&lt;p&gt;For example, I &lt;a href=&quot;https://github.com/davidrunger/david_runger/pull/4586&quot;&gt;changed&lt;/a&gt; the &lt;code class=&quot;highlighter-rouge&quot;&gt;test&lt;/code&gt; section of &lt;code class=&quot;highlighter-rouge&quot;&gt;config/cable.yml&lt;/code&gt; from:&lt;/p&gt;

&lt;div class=&quot;language-yaml highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;na&quot;&gt;test&lt;/span&gt;&lt;span class=&quot;pi&quot;&gt;:&lt;/span&gt;
  &lt;span class=&quot;na&quot;&gt;&amp;lt;&amp;lt;&lt;/span&gt;&lt;span class=&quot;pi&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;nv&quot;&gt;*default&lt;/span&gt;
  &lt;span class=&quot;na&quot;&gt;channel_prefix&lt;/span&gt;&lt;span class=&quot;pi&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;david_runger_test&lt;/span&gt;
  &lt;span class=&quot;na&quot;&gt;url&lt;/span&gt;&lt;span class=&quot;pi&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;redis://localhost:6379&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;to:&lt;/p&gt;

&lt;div class=&quot;language-yaml highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;na&quot;&gt;test&lt;/span&gt;&lt;span class=&quot;pi&quot;&gt;:&lt;/span&gt;
  &lt;span class=&quot;na&quot;&gt;&amp;lt;&amp;lt;&lt;/span&gt;&lt;span class=&quot;pi&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;nv&quot;&gt;*default&lt;/span&gt;
  &lt;span class=&quot;na&quot;&gt;channel_prefix&lt;/span&gt;&lt;span class=&quot;pi&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;david_runger_test&amp;lt;%= ENV[&apos;DB_SUFFIX&apos;] %&amp;gt;&lt;/span&gt;
  &lt;span class=&quot;na&quot;&gt;url&lt;/span&gt;&lt;span class=&quot;pi&quot;&gt;:&lt;/span&gt; &lt;span class=&quot;s&quot;&gt;redis://localhost:6379&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;The &lt;code class=&quot;highlighter-rouge&quot;&gt;&amp;lt;%= ENV[&apos;DB_SUFFIX&apos;] %&amp;gt;&lt;/code&gt; ERB interpolation makes the prefix process-specific. My CI assigns every RSpec process a distinct &lt;code class=&quot;highlighter-rouge&quot;&gt;DB_SUFFIX&lt;/code&gt;, such as &lt;code class=&quot;highlighter-rouge&quot;&gt;_unit&lt;/code&gt;, &lt;code class=&quot;highlighter-rouge&quot;&gt;_api&lt;/code&gt;, or &lt;code class=&quot;highlighter-rouge&quot;&gt;_feature&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Rails documents &lt;code class=&quot;highlighter-rouge&quot;&gt;channel_prefix&lt;/code&gt; for avoiding channel-name collisions when applications share a Redis server: &lt;a href=&quot;https://guides.rubyonrails.org/action_cable_overview.html#redis-adapter&quot;&gt;Redis Adapter&lt;/a&gt;.&lt;/p&gt;

&lt;h2 id=&quot;separate-redis-databases-do-not-isolate-pubsub&quot;&gt;Separate Redis databases do not isolate Pub/Sub&lt;/h2&gt;

&lt;p&gt;Different Redis database numbers do not solve this problem: Redis Pub/Sub is not scoped to databases. Redis recommends prefixing channel names when they need to be scoped: &lt;a href=&quot;https://redis.io/docs/latest/develop/pubsub/#database--scoping&quot;&gt;Database &amp;amp; Scoping&lt;/a&gt;.&lt;/p&gt;</content><author><name>David Runger</name></author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://davidrunger.com/blog/images/davidrunger.jpg" /><media:content medium="image" url="https://davidrunger.com/blog/images/davidrunger.jpg" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Enforce Git hooks in a Rails initializer</title><link href="https://davidrunger.com/blog/enforce-git-hooks-in-a-rails-initializer" rel="alternate" type="text/html" title="Enforce Git hooks in a Rails initializer" /><published>2024-07-02T00:00:00-05:00</published><updated>2024-07-02T00:00:00-05:00</updated><id>repo://posts.collection/_posts/2024-07-02-enforce-git-hooks-in-a-rails-initializer.md</id><content type="html" xml:base="https://davidrunger.com/blog/enforce-git-hooks-in-a-rails-initializer">&lt;h2 id=&quot;git-hooks&quot;&gt;Git hooks&lt;/h2&gt;

&lt;p&gt;Git hooks are scripts that Git runs automatically at points such as before a commit, push, or merge. For example, a &lt;a href=&quot;https://github.com/davidrunger/david_runger/blob/8e20a1c49595bf48a66682c9e48ce5cc3f892c16/bin/githooks/pre-push&quot;&gt;pre-push hook&lt;/a&gt; in this site’s repository uses &lt;a href=&quot;https://github.com/betterleaks/betterleaks&quot;&gt;Betterleaks&lt;/a&gt; to scan the outgoing diff for secrets and runs linters. See the &lt;a href=&quot;https://git-scm.com/docs/githooks&quot;&gt;Git hook documentation&lt;/a&gt; for the full list.&lt;/p&gt;

&lt;p&gt;That feedback is useful only when each contributor’s clone is configured to use the hooks stored in the repository.&lt;/p&gt;

&lt;h2 id=&quot;the-configuration-gap&quot;&gt;The configuration gap&lt;/h2&gt;

&lt;p&gt;Git does not automatically use hook scripts committed to a repository; each clone must opt in by setting &lt;code class=&quot;highlighter-rouge&quot;&gt;core.hooksPath&lt;/code&gt;. Setup documentation can ask contributors to do that, but instructions are easy to miss.&lt;/p&gt;

&lt;h2 id=&quot;check-during-rails-boot&quot;&gt;Check during Rails boot&lt;/h2&gt;

&lt;p&gt;For a Rails application, an initializer can verify the setting and refuse to boot until it is fixed. That makes a missing one-time setup step visible when a developer first uses the app.&lt;/p&gt;

&lt;p&gt;This is a usability guardrail, not a security or policy boundary. A developer can bypass Git hooks (for example, with &lt;code class=&quot;highlighter-rouge&quot;&gt;git push --no-verify&lt;/code&gt;) or change the initializer without booting the app. Checks that must not be bypassed belong in CI or protected server-side controls.&lt;/p&gt;

&lt;p&gt;The idea is not Rails-specific: an application built with any framework could run an equivalent check during local development startup.&lt;/p&gt;

&lt;h2 id=&quot;the-initializer&quot;&gt;The initializer&lt;/h2&gt;

&lt;p&gt;Place this in &lt;code class=&quot;highlighter-rouge&quot;&gt;config/initializers/githooks_check.rb&lt;/code&gt;:&lt;/p&gt;

&lt;div class=&quot;language-rb highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;k&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;
  &lt;span class=&quot;no&quot;&gt;Rails&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;env&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;local?&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;&amp;amp;&amp;amp;&lt;/span&gt;
    &lt;span class=&quot;no&quot;&gt;ENV&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;s1&quot;&gt;&apos;SKIP_GITHOOKS_CHECK&apos;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;].&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;blank?&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;&amp;amp;&amp;amp;&lt;/span&gt;
    &lt;span class=&quot;no&quot;&gt;ENV&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;[&lt;/span&gt;&lt;span class=&quot;s1&quot;&gt;&apos;CI&apos;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;].&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;blank?&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;&amp;amp;&amp;amp;&lt;/span&gt;
    &lt;span class=&quot;sb&quot;&gt;`git config core.hooksPath`&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;strip&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;!=&lt;/span&gt; &lt;span class=&quot;s1&quot;&gt;&apos;bin/githooks&apos;&lt;/span&gt;
&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
  &lt;span class=&quot;vg&quot;&gt;$stderr&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;puts&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;&amp;lt;&amp;lt;~&lt;/span&gt;&lt;span class=&quot;no&quot;&gt;ERROR&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;sh&quot;&gt;
    You have not configured the git hooks for this repo! To do so, run:
        git config core.hooksPath bin/githooks
    Or, if you must, you can put SKIP_GITHOOKS_CHECK=1 in your .env file.
&lt;/span&gt;&lt;span class=&quot;no&quot;&gt;  ERROR&lt;/span&gt;

  &lt;span class=&quot;nb&quot;&gt;exit&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;mi&quot;&gt;1&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;end&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;(You can see the actual source code &lt;a href=&quot;https://github.com/davidrunger/david_runger/blob/8e20a1c49595bf48a66682c9e48ce5cc3f892c16/config/initializers/githooks_check.rb&quot;&gt;here&lt;/a&gt;.)&lt;/p&gt;

&lt;p&gt;Without the setting, &lt;code class=&quot;highlighter-rouge&quot;&gt;bin/rails server&lt;/code&gt; prints the following error to standard error and aborts:&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;❯ bin/rails server
=&amp;gt; Booting Puma
=&amp;gt; Rails 7.1.3.4 application starting in development
=&amp;gt; Run `bin/rails server --help` for more startup options
You have not configured the git hooks for this repo! To do so, run:
    git config core.hooksPath bin/githooks
Or, if you must, you can put SKIP_GITHOOKS_CHECK=1 in your .env file.
Exiting
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;The conditions limit the check to local, non-CI boots; &lt;code class=&quot;highlighter-rouge&quot;&gt;SKIP_GITHOOKS_CHECK&lt;/code&gt; provides an escape hatch. The remaining condition verifies that Git uses &lt;code class=&quot;highlighter-rouge&quot;&gt;bin/githooks&lt;/code&gt;. If it does not, the initializer explains the fix and exits unsuccessfully.&lt;/p&gt;

&lt;h2 id=&quot;configuring-the-git-hooks&quot;&gt;Configuring the Git hooks&lt;/h2&gt;

&lt;p&gt;At the project root, run:&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;❯ git config core.hooksPath bin/githooks
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;This tells Git to find hooks in the repository’s &lt;a href=&quot;https://github.com/davidrunger/david_runger/tree/main/bin/githooks&quot;&gt;&lt;code class=&quot;highlighter-rouge&quot;&gt;bin/githooks/&lt;/code&gt;&lt;/a&gt; directory. Git runs &lt;code class=&quot;highlighter-rouge&quot;&gt;pre-push&lt;/code&gt; before pushing and &lt;code class=&quot;highlighter-rouge&quot;&gt;pre-commit&lt;/code&gt; before committing, when those executable scripts exist.&lt;/p&gt;

&lt;p&gt;Rails can now boot:&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;❯ bin/rails server
=&amp;gt; Booting Puma
=&amp;gt; Rails 7.1.3.4 application starting in development
   [ ... ]
* Listening on http://127.0.0.1:3000
* Listening on http://[::1]:3000
Use Ctrl-C to stop
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Future pushes will also run the repository’s pre-push checks locally.&lt;/p&gt;</content><author><name>David Runger</name></author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://davidrunger.com/blog/images/davidrunger.jpg" /><media:content medium="image" url="https://davidrunger.com/blog/images/davidrunger.jpg" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Using VS Code as a Rails app:update merge tool</title><link href="https://davidrunger.com/blog/using-vs-code-as-a-rails-app-update-merge-tool" rel="alternate" type="text/html" title="Using VS Code as a Rails app:update merge tool" /><published>2024-06-26T00:00:00-05:00</published><updated>2024-06-26T00:00:00-05:00</updated><id>repo://posts.collection/_posts/2024-06-26-using-vs-code-as-a-rails-app-update-merge-tool.md</id><content type="html" xml:base="https://davidrunger.com/blog/using-vs-code-as-a-rails-app-update-merge-tool">&lt;p&gt;When upgrading Rails, you should run &lt;a href=&quot;https://guides.rubyonrails.org/upgrading_ruby_on_rails.html#the-update-task&quot;&gt;&lt;code class=&quot;highlighter-rouge&quot;&gt;bin/rails app:update&lt;/code&gt;&lt;/a&gt;. The command compares your application’s configuration files with those that the new Rails version would generate, then asks how to resolve each difference:&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;❯ bin/rails app:update
    conflict  config/boot.rb
Overwrite /home/david/code/david_runger/config/boot.rb? (enter &quot;h&quot; for help) [Ynaqdhm]
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Entering &lt;code class=&quot;highlighter-rouge&quot;&gt;h&lt;/code&gt; explains the available options:&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;Y - yes, overwrite
n - no, do not overwrite
a - all, overwrite this and all others
q - quit, abort
h - help, show this help
d - diff, show the differences between the old and the new
m - merge, run merge tool
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Overwriting (&lt;code class=&quot;highlighter-rouge&quot;&gt;Y&lt;/code&gt;) can discard application-specific configuration, while declining (&lt;code class=&quot;highlighter-rouge&quot;&gt;n&lt;/code&gt;) can miss relevant changes from the new Rails version. A three-way merge (&lt;code class=&quot;highlighter-rouge&quot;&gt;m&lt;/code&gt;) makes it easier to preserve your configuration while incorporating new defaults.&lt;/p&gt;

&lt;p&gt;Without a configured merge tool, however, choosing &lt;code class=&quot;highlighter-rouge&quot;&gt;m&lt;/code&gt; produces an error:&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;Overwrite /home/david/code/david_runger/config/boot.rb? (enter &quot;h&quot; for help) [Ynaqdhm] m
Please specify merge tool to `THOR_MERGE` env.
Overwrite /home/david/code/david_runger/config/boot.rb? (enter &quot;h&quot; for help) [Ynaqdhm]
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Here is how to use VS Code as that merge tool.&lt;/p&gt;

&lt;h2 id=&quot;create-a-vs-code-adapter&quot;&gt;Create a VS Code adapter&lt;/h2&gt;

&lt;p&gt;For each conflict, Rails’s underlying Thor library passes &lt;code class=&quot;highlighter-rouge&quot;&gt;THOR_MERGE&lt;/code&gt; two paths: a temporary file containing the new Rails default and the application’s existing file.&lt;/p&gt;

&lt;p&gt;VS Code’s &lt;a href=&quot;https://code.visualstudio.com/docs/configure/command-line#_core-cli-options&quot;&gt;&lt;code class=&quot;highlighter-rouge&quot;&gt;--merge&lt;/code&gt; option&lt;/a&gt; instead expects four paths:&lt;/p&gt;

&lt;div class=&quot;highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;code --merge &amp;lt;path1&amp;gt; &amp;lt;path2&amp;gt; &amp;lt;base&amp;gt; &amp;lt;result&amp;gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;An adapter script can supply the two additional paths. Open a new script in VS Code, then make it executable after you save and close it:&lt;/p&gt;

&lt;div class=&quot;language-sh highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;code &lt;span class=&quot;nt&quot;&gt;--wait&lt;/span&gt; bin/app_update_thor_merge_tool
&lt;span class=&quot;nb&quot;&gt;chmod&lt;/span&gt; +x bin/app_update_thor_merge_tool
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Paste this into the script:&lt;/p&gt;

&lt;div class=&quot;language-sh highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;c&quot;&gt;#!/usr/bin/env bash&lt;/span&gt;

&lt;span class=&quot;nv&quot;&gt;rails_default_config&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$1&lt;/span&gt;
&lt;span class=&quot;nv&quot;&gt;app_current_config&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$2&lt;/span&gt;

&lt;span class=&quot;nv&quot;&gt;temporary_directory&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;$(&lt;/span&gt;&lt;span class=&quot;nb&quot;&gt;mktemp&lt;/span&gt; &lt;span class=&quot;nt&quot;&gt;-d&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;)&lt;/span&gt;
&lt;span class=&quot;nb&quot;&gt;trap&lt;/span&gt; &lt;span class=&quot;s1&quot;&gt;&apos;rm -rf &quot;$temporary_directory&quot;&apos;&lt;/span&gt; EXIT

&lt;span class=&quot;nv&quot;&gt;current_copy&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$temporary_directory&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;/current&quot;&lt;/span&gt;
&lt;span class=&quot;nv&quot;&gt;base_copy&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$temporary_directory&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;/base&quot;&lt;/span&gt;

&lt;span class=&quot;nb&quot;&gt;cp&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$app_current_config&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$current_copy&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;
&lt;span class=&quot;nb&quot;&gt;cp&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$app_current_config&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$base_copy&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;

code &lt;span class=&quot;nt&quot;&gt;--merge&lt;/span&gt; &lt;span class=&quot;se&quot;&gt;\&lt;/span&gt;
  &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$rails_default_config&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt; &lt;span class=&quot;se&quot;&gt;\&lt;/span&gt;
  &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$current_copy&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt; &lt;span class=&quot;se&quot;&gt;\&lt;/span&gt;
  &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$base_copy&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt; &lt;span class=&quot;se&quot;&gt;\&lt;/span&gt;
  &lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;nv&quot;&gt;$app_current_config&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt; &lt;span class=&quot;se&quot;&gt;\&lt;/span&gt;
  &lt;span class=&quot;nt&quot;&gt;--wait&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Then run &lt;code class=&quot;highlighter-rouge&quot;&gt;app:update&lt;/code&gt; with the script’s absolute path in &lt;code class=&quot;highlighter-rouge&quot;&gt;THOR_MERGE&lt;/code&gt;:&lt;/p&gt;

&lt;div class=&quot;language-sh highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;nv&quot;&gt;THOR_MERGE&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;=&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;&quot;&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;$(&lt;/span&gt;&lt;span class=&quot;nb&quot;&gt;pwd&lt;/span&gt;&lt;span class=&quot;si&quot;&gt;)&lt;/span&gt;&lt;span class=&quot;s2&quot;&gt;/bin/app_update_thor_merge_tool&quot;&lt;/span&gt; bin/rails app:update
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;When you choose &lt;code class=&quot;highlighter-rouge&quot;&gt;m&lt;/code&gt; for a conflict, VS Code opens its three-way merge editor:&lt;/p&gt;

&lt;p&gt;&lt;img src=&quot;https://david-runger-public-uploads.s3.amazonaws.com/app-upate-vs-code-three-way-merge.png&quot; alt=&quot;app:update VS Code three-way merge&quot; /&gt;&lt;/p&gt;

&lt;p&gt;The left pane contains the new Rails default, the right pane contains your application’s current configuration, and the center pane contains the result. Review the relevant Rails documentation, resolve the differences, save the result, and close the tab. Because the adapter passes &lt;code class=&quot;highlighter-rouge&quot;&gt;--wait&lt;/code&gt;, &lt;code class=&quot;highlighter-rouge&quot;&gt;app:update&lt;/code&gt; will then advance to the next file.&lt;/p&gt;

&lt;p&gt;Repeat until you have reviewed every conflict.&lt;/p&gt;</content><author><name>David Runger</name></author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://david-runger-public-uploads.s3.amazonaws.com/app-upate-vs-code-three-way-merge.png" /><media:content medium="image" url="https://david-runger-public-uploads.s3.amazonaws.com/app-upate-vs-code-three-way-merge.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Quick Tip: Use gem.wtf to open a Ruby gem’s repository</title><link href="https://davidrunger.com/blog/quick-tip-use-gem.wtf-to-go-to-the-github-repo-of-a-gem" rel="alternate" type="text/html" title="Quick Tip: Use gem.wtf to open a Ruby gem&apos;s repository" /><published>2023-02-13T00:00:00-06:00</published><updated>2023-02-13T00:00:00-06:00</updated><id>repo://posts.collection/_posts/2023-02-13-quick-tip-use-gem.wtf-to-go-to-the-github-repo-of-a-gem.md</id><content type="html" xml:base="https://davidrunger.com/blog/quick-tip-use-gem.wtf-to-go-to-the-github-repo-of-a-gem">&lt;p&gt;&lt;a href=&quot;https://gem.wtf/&quot;&gt;gem.wtf&lt;/a&gt; takes you straight to a Ruby gem’s source repository (usually a GitHub repo). Enter &lt;code class=&quot;highlighter-rouge&quot;&gt;gem.wtf/&amp;lt;gem-name&amp;gt;&lt;/code&gt; in your browser’s address bar; for example, &lt;a href=&quot;https://gem.wtf/rails&quot;&gt;gem.wtf/rails&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;It uses the gem’s RubyGems metadata, so it works with repositories outside GitHub too. If a gem does not declare a source repository, it falls back to its documentation or RubyGems page.&lt;/p&gt;

&lt;p&gt;It’s handy for checking a gem’s README, issues, and history before using the gem or while debugging a problem.&lt;/p&gt;

&lt;p&gt;It was created by &lt;a href=&quot;https://github.com/zeke&quot;&gt;Zeke Sikelianos&lt;/a&gt;; its source is available on &lt;a href=&quot;https://github.com/zeke/gem.wtf&quot;&gt;GitHub&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Bonus tip:&lt;/strong&gt; &lt;a href=&quot;https://ghub.io/&quot;&gt;ghub.io&lt;/a&gt; does the same for npm packages.&lt;/p&gt;</content><author><name>David Runger</name></author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://davidrunger.com/blog/images/davidrunger.jpg" /><media:content medium="image" url="https://davidrunger.com/blog/images/davidrunger.jpg" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Mocking external requests in Rails feature tests</title><link href="https://davidrunger.com/blog/mocking-external-requests-in-rails-feature-tests" rel="alternate" type="text/html" title="Mocking external requests in Rails feature tests" /><published>2023-01-30T00:00:00-06:00</published><updated>2023-01-30T00:00:00-06:00</updated><id>repo://posts.collection/_posts/2023-01-30-mocking-external-requests-in-rails-feature-tests.md</id><content type="html" xml:base="https://davidrunger.com/blog/mocking-external-requests-in-rails-feature-tests">&lt;p&gt;Need to test a redirect to an external service without contacting it? Intercept the request in the browser that Capybara drives.&lt;/p&gt;

&lt;p&gt;Here is the relevant part of a Cuprite feature spec that verifies a Google OAuth redirect:&lt;/p&gt;

&lt;div class=&quot;language-rb highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;n&quot;&gt;context&lt;/span&gt; &lt;span class=&quot;s1&quot;&gt;&apos;when OmniAuth test mode is disabled&apos;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;ss&quot;&gt;:permit_all_external_requests&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;do&lt;/span&gt;
  &lt;span class=&quot;n&quot;&gt;around&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;do&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;|&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;spec&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;|&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;original_omni_auth_test_mode&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;no&quot;&gt;OmniAuth&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;config&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;test_mode&lt;/span&gt;
    &lt;span class=&quot;no&quot;&gt;OmniAuth&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;config&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;test_mode&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;kp&quot;&gt;false&lt;/span&gt;

    &lt;span class=&quot;n&quot;&gt;spec&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;run&lt;/span&gt;
  &lt;span class=&quot;k&quot;&gt;ensure&lt;/span&gt;
    &lt;span class=&quot;no&quot;&gt;OmniAuth&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;config&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;test_mode&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;original_omni_auth_test_mode&lt;/span&gt;
  &lt;span class=&quot;k&quot;&gt;end&lt;/span&gt;

  &lt;span class=&quot;n&quot;&gt;context&lt;/span&gt; &lt;span class=&quot;s1&quot;&gt;&apos;when Google responds with &quot;This is Google OAuth.&quot;&apos;&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;do&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;let&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;ss&quot;&gt;:google_response_content&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;{&lt;/span&gt; &lt;span class=&quot;s1&quot;&gt;&apos;This is Google OAuth.&apos;&lt;/span&gt; &lt;span class=&quot;p&quot;&gt;}&lt;/span&gt;

    &lt;span class=&quot;n&quot;&gt;before&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;do&lt;/span&gt;
      &lt;span class=&quot;n&quot;&gt;browser&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;page&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;driver&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;browser&lt;/span&gt;
      &lt;span class=&quot;n&quot;&gt;browser&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;network&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;intercept&lt;/span&gt;
      &lt;span class=&quot;n&quot;&gt;browser&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;on&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;ss&quot;&gt;:request&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;do&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;|&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;request&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;|&lt;/span&gt;
        &lt;span class=&quot;k&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;request&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;match?&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;sr&quot;&gt;%r{&lt;/span&gt;&lt;span class=&quot;se&quot;&gt;\A&lt;/span&gt;&lt;span class=&quot;sr&quot;&gt;https://accounts.google.com/o/oauth2/auth&lt;/span&gt;&lt;span class=&quot;se&quot;&gt;\?&lt;/span&gt;&lt;span class=&quot;sr&quot;&gt;}&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
          &lt;span class=&quot;n&quot;&gt;request&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;respond&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;ss&quot;&gt;body: &lt;/span&gt;&lt;span class=&quot;n&quot;&gt;google_response_content&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
        &lt;span class=&quot;k&quot;&gt;else&lt;/span&gt;
          &lt;span class=&quot;n&quot;&gt;request&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;continue&lt;/span&gt;
        &lt;span class=&quot;k&quot;&gt;end&lt;/span&gt;
      &lt;span class=&quot;k&quot;&gt;end&lt;/span&gt;
    &lt;span class=&quot;k&quot;&gt;end&lt;/span&gt;

    &lt;span class=&quot;n&quot;&gt;it&lt;/span&gt; &lt;span class=&quot;s2&quot;&gt;&quot;renders Google&apos;s response&quot;&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;do&lt;/span&gt;
      &lt;span class=&quot;n&quot;&gt;visit&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;new_user_session_path&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
      &lt;span class=&quot;n&quot;&gt;expect&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;page&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;).&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;to&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;have_css&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s1&quot;&gt;&apos;google-sign-in-button&apos;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;

      &lt;span class=&quot;n&quot;&gt;click_sign_in_with_google&lt;/span&gt;

      &lt;span class=&quot;n&quot;&gt;expect&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;page&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;).&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;to&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;have_text&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;google_response_content&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
    &lt;span class=&quot;k&quot;&gt;end&lt;/span&gt;
  &lt;span class=&quot;k&quot;&gt;end&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;end&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;&lt;code class=&quot;highlighter-rouge&quot;&gt;browser.network.intercept&lt;/code&gt; pauses requests, and the &lt;code class=&quot;highlighter-rouge&quot;&gt;:request&lt;/code&gt; handler either supplies a fake response or lets the request proceed. The URL matcher is deliberately narrow: the assertion only proves the browser reached Google’s OAuth authorization URL when it displays the supplied content. No request reaches Google.&lt;/p&gt;

&lt;p&gt;The &lt;code class=&quot;highlighter-rouge&quot;&gt;:permit_all_external_requests&lt;/code&gt; metadata is specific to this test suite’s request policy. Keep or replace it with the equivalent configuration in your application. OmniAuth test mode must be disabled because it otherwise skips the external redirect; the &lt;code class=&quot;highlighter-rouge&quot;&gt;ensure&lt;/code&gt; restores the original setting even when the example fails.&lt;/p&gt;

&lt;h2 id=&quot;why-browser-interception&quot;&gt;Why browser interception?&lt;/h2&gt;

&lt;p&gt;&lt;a href=&quot;https://github.com/bblimke/webmock&quot;&gt;WebMock&lt;/a&gt; intercepts HTTP requests made by the Ruby process. In a JavaScript-capable feature spec, the browser is a separate process, so WebMock cannot intercept its navigation to an external site. Cuprite’s browser-level network API fills that gap.&lt;/p&gt;

&lt;p&gt;This is useful for integration boundaries such as OAuth. My regression test covers a Content Security Policy (CSP) failure: the login form submits to my application, which redirects the browser to Google. A restrictive &lt;code class=&quot;highlighter-rouge&quot;&gt;form-action&lt;/code&gt; directive can block that redirect unless the OAuth domain is allowed. The &lt;a href=&quot;https://github.com/w3c/webappsec-csp/issues/8&quot;&gt;CSP discussion&lt;/a&gt; describes the interaction.&lt;/p&gt;

&lt;p&gt;Mocking the destination keeps the test focused on whether the browser follows the redirect. It also avoids a dependency on Google’s availability, content, credentials, and an Internet connection, making the test faster and more reliable.&lt;/p&gt;

&lt;h2 id=&quot;selenium-alternative&quot;&gt;Selenium alternative&lt;/h2&gt;

&lt;p&gt;This site previously used &lt;code class=&quot;highlighter-rouge&quot;&gt;selenium-webdriver&lt;/code&gt;. Selenium can intercept the same request through its DevTools integration:&lt;/p&gt;

&lt;div class=&quot;language-rb highlighter-rouge&quot;&gt;&lt;div class=&quot;highlight&quot;&gt;&lt;pre class=&quot;highlight&quot;&gt;&lt;code&gt;&lt;span class=&quot;n&quot;&gt;around&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;do&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;|&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;spec&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;|&lt;/span&gt;
  &lt;span class=&quot;n&quot;&gt;page&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;driver&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;browser&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;intercept&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;do&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;|&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;request&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;,&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;&amp;amp;&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;continue&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;|&lt;/span&gt;
    &lt;span class=&quot;n&quot;&gt;continue&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;call&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;request&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt; &lt;span class=&quot;k&quot;&gt;do&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;|&lt;/span&gt;&lt;span class=&quot;n&quot;&gt;response&lt;/span&gt;&lt;span class=&quot;o&quot;&gt;|&lt;/span&gt;
      &lt;span class=&quot;k&quot;&gt;if&lt;/span&gt; &lt;span class=&quot;n&quot;&gt;request&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;url&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;start_with?&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;(&lt;/span&gt;&lt;span class=&quot;s1&quot;&gt;&apos;https://accounts.google.com/o/oauth2/auth?&apos;&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;)&lt;/span&gt;
        &lt;span class=&quot;n&quot;&gt;response&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;code&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;mi&quot;&gt;200&lt;/span&gt;
        &lt;span class=&quot;n&quot;&gt;response&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;body&lt;/span&gt; &lt;span class=&quot;o&quot;&gt;=&lt;/span&gt; &lt;span class=&quot;s1&quot;&gt;&apos;This is Google OAuth.&apos;&lt;/span&gt;
      &lt;span class=&quot;k&quot;&gt;end&lt;/span&gt;
    &lt;span class=&quot;k&quot;&gt;end&lt;/span&gt;
  &lt;span class=&quot;k&quot;&gt;end&lt;/span&gt;

  &lt;span class=&quot;n&quot;&gt;spec&lt;/span&gt;&lt;span class=&quot;p&quot;&gt;.&lt;/span&gt;&lt;span class=&quot;nf&quot;&gt;run&lt;/span&gt;
&lt;span class=&quot;k&quot;&gt;end&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;&lt;/div&gt;

&lt;p&gt;Add &lt;code class=&quot;highlighter-rouge&quot;&gt;selenium-devtools&lt;/code&gt; to the &lt;code class=&quot;highlighter-rouge&quot;&gt;test&lt;/code&gt; group for this approach. The surrounding OmniAuth setup and assertion are the same. The former Selenium version of this test is available in &lt;a href=&quot;https://github.com/davidrunger/david_runger/commit/6aab7bf&quot;&gt;the original implementation&lt;/a&gt;; the current Cuprite version is in &lt;a href=&quot;https://github.com/davidrunger/david_runger/blob/main/spec/features/user_google_login_spec.rb&quot;&gt;the site’s source&lt;/a&gt;.&lt;/p&gt;</content><author><name>David Runger</name></author><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://davidrunger.com/blog/images/davidrunger.jpg" /><media:content medium="image" url="https://davidrunger.com/blog/images/davidrunger.jpg" xmlns:media="http://search.yahoo.com/mrss/" /></entry></feed>