Hotwire: Reactive Rails with no JavaScript?

Cover for Hotwire: Reactive Rails with no JavaScript?

It’s time to actually conjure up the long-teased new magic by DHH & Co. and learn to use Hotwire beyond 5-minute tutorials. This umbrella term, behind libraries for building modern web interfaces seemingly without any effort or JavaScript, is on everyone’s tongue since the big unveil this year. The HTML-over-the-wire approach is making ripples through the Rails universe: countless blogs, Reddit posts, screencasts, and five RailsConf talks this year, including the one by yours truly, are dedicated to it. I want to take more time to explain Hotwire thoroughly—with code examples and testing strategies. As my favorite rock band would say, let’s get “Hotwired” to self-destruct …learn new tricks!

Life is short

To see the Hotwire-ing of a Rails 6 app at a glance, without any further fluff, feel free to study this PR.

This article explains the code above in great detail. It’s an adaptation (and an extension of) my RailsConf 2021 talk: “Frontendless Rails frontend”, which is already available online for all RailsConf attendees. If you don’t have conference tickets, don’t worry: you can find the slide deck here, and the same page will be updated with the video once all the talks have become public.

“This is the way”

For the last five years, I’ve been doing mostly pure backend development: REST and GraphQL APIs, WebSocket, gRPC, databases, caches—pretty much all there is to do “behind-the-veil”.

The entire frontend-revolution has passed me over like a giant wave: I still don’t understand why we need to stuff every web app with reacts and webpacks. The classic HTML-first Rails way was my way (or the highway 😉). Remember the days when the JavaScript in your application didn’t require its own MVC (or MVVM) to function? I miss those days. And these days are quietly making a hell of a comeback.

Today, we’re witnessing the rise of HTML-over-the-wire (yes, it’s an actual term now). Sparked by Phoenix LiveView, this approach, based on pushing backend-rendered templates over WebSocket to all connected clients, has gotten a lot of traction in the Rails community, thanks to the StimulusReflex family of gems. It was finally blessed by DHH himself when he introduced Hotwire to the world early this year.

Are we standing on the verge of yet another global paradigm shift in web development? Will we go back to the simple mental model of server-rendered templates, just this time with all the bells and whistles of reactive interfaces at little-to-no effort? As much as I’d love that, I realize it’s wishful thinking: Big Tech is too invested in client-rendered apps ever to go back. Frontend development in the 2020s is a separate qualification and a separate industry with its own entry requirements; there’s no way we’ll see “full-stack” again.

However, HOTWire (see that acronym right there? Smart on Basecamp’s part, huh?) provides a much-needed alternative to the, shall we say “complicated”, rocket science of modern client-side programming for the browser.

For the Rails developer who’s tired of doing API-only apps with no control over the presentation and who misses creating a user experiences as an escape from wrangling SQL and JSONs forty hours a week, Hotwire is a long-sought breath of fresh air that makes web development fun again.

In this post, I’d like to demonstrate how to apply the HTML-over-the-wire philosophy to an existing Rails application through Hotwire. As with most of my recent articles, I will use my AnyCable demo application as a guinea pig.

And this app fits the task well: it’s interactive and reactive, driven by Turbolinks and a handful of custom (Java)scripts, and it also boasts decent system tests coverage (meaning we can refactor safely). Our hotwire-ification will be executed with some easy-to-follow steps:

Turbolinks has been known in the Rails world for a long time; the first major release was published as early as 2013. However, in my early days, Rails developers followed a rule of thumb: If your frontend is acting weird, try disabling Turbolinks. Making third-party JS code compatible with Turbolink’s fake (read: pushState + AJAX) navigation was not a walk in the park.

I stopped avoiding Turbolinks when StimulusJS came out. It radically solved the problem of connecting and disconnecting JavaScript sprinkles by relying on the modern DOM mutation APIs. Turbolinks combined with Stimulus for code organization, and DOM manipulation yields an effortless “SPA” experience at a fraction of React/Angular development costs.

The same good old Turbolinks has now been re-branded as Turbo Drive, as it literally drives Turbo—the heart of Hotwire package deal.

If your app already uses Turbolinks (and mine did), switching to Turbo Drive is dead simple. It’s just about renaming some things.

All you need is to replace turbolinks with @hotwired/turbo-rails in your package.json file, and in the Gemfile—replace turbolinks with turbo-rails.

The initialization code looks a little bit different and is now even more concise:

- import Turbolinks from 'turbolinks';

- Turbolinks.start();
+ import "@hotwired/turbo"

Note that we don’t need to start Turbo Drive manually (and we can’t stop it).

Some “Find & Replace” is required to update all the HTML data attributes from data-turbolinks to data-turbo.

The only change that took me some time to figure out was dealing with forms and redirects. Previously, with Turbolinks, I used remote forms (remote: true) and the Redirection concern to respond with JavaScript templates. Turbo Drive has its own built-in support for hijacking forms, so remote: true is no longer required. However, it turned out that the redirection code must be updated. Or, more precisely, the redirection status code:

- redirect_to workspace
+ redirect_to workspace, status: :see_other

Using the somewhat obscure See Other HTTP response code (303) is a smart choice: it allows Turbo to rely on the native Fetch API redirect: "follow" option, so you don’t have to explicitly initiate another request to fetch the new content after form submission. According to the specification, “if status is 303 and request’s method is not GET or HEAD, a GET request must be performed automatically. Compare this with “if status is 301 or 302 and request’s method is POST—see the difference?

Other 3xx statuses are only meant for POST requests, while with Rails we usually use POST, PATCH, PUT, and DELETE.

Framing with Turbo Frames

Time to move on to something truly new: Turbo Frames.

Turbo Frames brings seamless updates to parts of the page (not to the whole page as Turbo Drive does). We can say it’s very similar to what <iframe> does but without creating separate windows, DOM trees, and security nightmares that come with it.

Let’s take a look at an example in use.

The AnyCable demo application (called AnyWork) allows you to create dashboards with multiple ToDo lists and a chat. A user can interact with items in different lists: add them, delete them, and mark them as completed.

Turbo Frames in action: each item is its own frame

Originally, completing and deleting items were backed by AJAX requests and a custom Stimulus controller. I decided to rewrite this functionality using Turbo Frames to go HTML all-in.

How can we decompose our ToDo lists to handle individual item updates? Let’s turn each item into a frame!

<!-- _item.html.rb -->
<%= turbo_frame_tag dom_id(item) do %>
  <div class="any-list--item<%= item.completed? ? " checked" : ""%>">
    <%= form_for item do |f| %>
      <!-- ... -->
    <% end %>
    <%= button_to item_path(item), method: :delete %>
      <!-- ... -->
    <% end %>
  </div>
<% end %>

We did three important things here:

  • wrapped our item container into a <turbo-frame> tag via the helper and passed a unique identifier (check out the handy dom_id method from ActionView);
  • added an HTML form to make Turbo intercept submissions and update the contents of the frame; and
  • used the button_to helper with method: :delete, which also creates an HTML form under the hood.

Now, whenever a form is submitted within the frame, Turbo intercepts the submissions, performs an AJAX request, extracts a frame with the same ID from the response HTML, and replaces the contents of this frame.

All of the above works with exactly zero hand-written JavaScript!

Let’s take a look at our updated controller code:

class ItemsController < ApplicationController
  def update
    item.update!(item_params)

    render partial: "item", locals: { item }
  end

  def destroy
    item.destroy!

    render partial: "item", locals: { item }
  end
end

Note that we respond with the same partial when we delete an item. However, we need to remove the item’s HTML node, not update it. How can we do that? We can respond with an empty frame! Let’s update our partial to do that:

<!-- _item.html.rb -->
<%= turbo_frame_tag dom_id(item) do %>
  <% unless item.destroyed? %>
    <div class="any-list--item<%= item.completed? ? " checked" : ""%>">
      <!-- ... -->
    </div>
  <% end %>
<% end %>

You’ve probably asked yourself the question: “How do we trigger form submission when marking an item as completed?” In other words, how can we make the checkbox state change trigger submit a form? We can do that by defining an inline event listener:

<%= f.check_box :completed, onchange: "this.form.requestSubmit();" %>

NB: Using requestSubmit() and not submit() is important: the former triggers a “submit” event that could be intercepted by Turbo, and the latter does not.

To sum it up, we can get rid of all the custom JS for this particular functionality just by altering our HTML template a bit and simplifying the controller’s code. I’m excited, aren’t you?

We can go further and convert our lists into frames, too. This would allow us to switch from Turbo Drive page updates to specific node updates when adding a new item. You can try this at home!

Imagine you also want to display a flash notification to a user whenever an item has been completed or deleted (“Item has been successfully removed”). Can we do this with Turbo Frames? Sounds like we’d need to wrap our flash messages container into a frame and push the updated HTML along with the markup for the item. That was my initial idea, and it didn’t work out: frame updates are scoped to the initiator frame. Thus, we can not update anything outside of it.

After some research, I found out that Turbo Streams could help with this.

Streaming with Turbo Streams

Compared to Drive and Frames, Turbo Streams is a completely new technology. Unlike those other two, Streams are explicit. Nothing happens automagically and you are in charge of what and when should be updated on a page. To do that, you’ll need to make use of some special <turbo-stream> elements.

Let’s take a look at this element from the example stream:

<turbo-stream action="replace" target="flash-alerts">
  <template>
    <div class="flash-alerts--container" id="flash-alerts">
      <!--  -->
    </div>
  </template>
</turbo-stream>

This element is responsible for replacing the node under DOM ID flash-alerts with the new HTML contents passed inside the <template> tag (note action="replace"). Whenever you drop a <turbo-stream> element on the page, it immediately performs the action and destroys itself. Under the hood, it uses HTML Custom Elements API—yet another example of using modern web APIs for the sake of developer happiness (i.e., less JavaScript 🙂).

I would say that Turbo Streams are a declarative alternative to the JavaScript templates of old. In the 2010s we wrote it like this:

// destroy.js.erb
$("#<%= dom_id(item) %>").remove();

And now we do the following:

<!--  destroy.html.erb -->
<%= turbo_stream.remove dom_id(item) %>

Currently, there are only five actions available: append, prepend, replace, remove, and update (which replaces only the text content of the node). We’ll talk about this limitation and how to overcome it below.

Let’s return to our initial problem: showing the flash notification in response to the ToDo item’s completion or deletion.

We want to respond with both <turbo-frame> and <turbo-stream> updates at once. How can we do that? Let’s add a new partial template:

<!-- _item_update.html.erb -->
<%= render item %>

<%= turbo_stream.replace "flash-alerts" do %>
  <%= render "shared/alerts" %>
<% end %>

Add a little change to the ItemsController:

+    flash.now[:notice] = "Item has been updated"

-    render partial: "item", locals: { item }
+    render partial: "item_update", locals: { item }

Unfortunately, the code above doesn’t work as expected: we don’t see any flash alerts.
After digging through the documentation, I found that Turbo expects an HTTP response to have the text/vnd.turbo-stream.html content type in order to activate stream elements. Okay, let’s do that:

-    render partial: "item_update", locals: { item }
+    render partial: "item_update", locals: { item }, content_type: "text/vnd.turbo-stream.html"

We now have the opposite situation: flash messages work, but the item’s content is not being updated 😞. Am I asking too much from Hotwire? Reading through the Turbo source code, I found that mixing streams and frames like this is not possible.

It turns out, there are two ways to implement this functionality:

  • Use streams for everything.
  • Put <turbo-stream> inside the <turbo-frame>.

The second option, in my opinion, runs counter to the idea of reusing HTML partials for regular page loads and Turbo updates. So, I went with the first one:

<!-- _item_update.html.erb -->
<%= turbo_stream.replace dom_id(item) do %>
  <%= render item %>
<% end %>

<%= turbo_stream.replace "flash-alerts" do %>
  <%= render "shared/alerts" %>
<% end %>

Mission complete. But at what cost? We had to add a new template for this use case. And I’m afraid that in a real-world application, the number of such ad hoc partials would grow as the app evolves.

UPDATE (2021-04-13): Alex Takitani suggested a more elegant solution: using a layout to update flash contents. We can define the application layout for Turbo Stream responses as follows:

<!-- layouts/application.turbo_stream.erb -->
<%= turbo_stream.replace "flash-alerts" do %>
  <%= render "shared/alerts" %>
<% end %>

<%= yield %>

Then, we need to remove the explicit rendering from the controller (because otherwise layout wouldn’t be used):

   def update
     item.update!(item_params)

     flash.now[:notice] = "Item has been updated"
-
-    render partial: "item_update", locals: { item }, content_type: "text/vnd.turbo-stream.html"
   end

NB: Don’t forget to add format: :turbo_stream to the corresponding requests in your controller/request specs to make implicit rendering work.

And let’s convert our _item_update partial to the update Turbo Stream template:

<!-- update.turbo_stream.erb -->
<%= turbo_stream.replace dom_id(item) do %>
  <%= render item %>
<% end %>

Cool, right? This is the Rails way!

Now let’s move to some real(-time) streaming.

Turbo Streams are often mentioned in the context of real-time updates (and are usually compared to StimulusReflex).

Let’s see how we can build lists synchronization on top of Turbo Streams:

Turbo Streams in action, synchronizing page updates for all connected clients

Before Turbo, I had to add a custom Action Cable channel and a Stimulus controller to handle broadcasts. I also needed to take care of the message format as I had to distinguish between deletion and completion of the items. In other words, a lot of code to maintain.

Turbo Streams takes care of most of this: the turbo-rails gem comes with a general Turbo::StreamChannel and a helper (#turbo_stream_from) to create a subscription right from HTML:

<!-- worspaces/show.html.erb -->
<div>
  <%= turbo_stream_from workspace %>
  <!-- ... -->
</div>

In the controller, we already had the #broadcast_new_item and #broadcast_changes “after action” hooks responsible for broadcasting updates. All we need now is to switch to Turbo::StreamChannel:

 def broadcast_changes
   return if item.errors.any?
   if item.destroyed?
-    ListChannel.broadcast_to list, type: "deleted", id: item.id
+    Turbo::StreamsChannel.broadcast_remove_to workspace, target: item
   else
-    ListChannel.broadcast_to list, type: "updated", id: item.id, desc: item.desc, completed: item.completed
+    Turbo::StreamsChannel.broadcast_replace_to workspace, target: item, partial: "items/item", locals: { item }
   end
 end

This migration went smoothly. …Almost. All the controller unit tests that were verifying broadcasts (#have_broadcasted_to) have failed.

Unfortunately, Turbo Rails doesn’t provide any testing tools (yet?), so I have to write my own, an all-too-familiar tale:

module Turbo::HaveBroadcastedToTurboMatcher
  include Turbo::Streams::StreamName

  def have_broadcasted_turbo_stream_to(*streamables, action:, target:) # rubocop:disable Naming/PredicateName
    target = target.respond_to?(:to_key) ? ActionView::RecordIdentifier.dom_id(target) : target
    have_broadcasted_to(stream_name_from(streamables))
      .with(a_string_matching(%(turbo-stream action="#{action}" target="#{target}")))
  end
end

RSpec.configure do |config|
  config.include Turbo::HaveBroadcastedToTurboMatcher
end

This is how I use my new matcher in tests:

 it "broadcasts a deleted message" do
-  expect { subject }.to have_broadcasted_to(ListChannel.broadcasting_for(list))
-    .with(type: "deleted", id: item.id)
+  expect { subject }.to have_broadcasted_turbo_stream_to(
+    workspace, action: :remove, target: item
+  )
 end

Going real-time with Turbo has gone well so far, and a bunch of code has been removed!

And we still haven’t written a single line of JavaScript code. Is this the real life? Is this just fantasy?

When am I going to wake up? Well, right now.

Beyond Turbo, or using Stimulus and custom elements

During the Turbo migration, I stumbled upon a couple of use-cases where using the existing API was not enough, so I finally had to write some JavaScript code!

Use-case number one: adding new lists to the dashboard in real-time. How is it different from the list of items example from the previous chapter? The markup. Let’s take a look at the dashboard layout:

<div id="workspace_1">
  <div id="list_1">...</div>
  <div id="list_2">...</div>
  <div id="new_list">
    <form>...</form>
  </div>
</div>

The last element is always the new list form container. Whenever we add a new list, it’s inserted right before the #new_list node. Do you recall that Turbo Streams supports only five actions? Do you see where the problem is? Here’s the code I used originally:

handleUpdate(data) {
  this.formTarget.insertAdjacentHTML("beforebegin", data.html);
}

To implement similar behavior using Turbo Streams, we need to add a hack to move the list to the right place immediately after being added via the stream. So, let’s add our own JavaScript sprinkle.

Let’s first give our task a formal definition: “When a new list item is appended to the workspace container, it should be positioned right before the new form element.” The word “when” here means that we need to observe the DOM and react to changes. Doesn’t it sound familiar? Yes, we already mentioned the MutationObserver API in relation to Stimulus! Let’s use it.

Luckily, we don’t have to write advanced JavaScript to use this feature; we can use stimulus-use (forgive me for this unavoidable tautology). Stimulus Use is a collection of useful behaviors for Stimulus controllers, simple snippets to solve complex problems. In our case, we need the useMutation behavior.

The code for the controller is pretty concise and self-explanatory:

import { Controller } from "stimulus";
import { useMutation } from "stimulus-use";

export default class extends Controller {
  static targets = ["lists", "newForm"];

  connect() {
    [this.observeLists, this.unobserveLists] = useMutation(this, {
      element: this.listsTarget,
      childList: true,
    });
  }

  mutate(entries) {
    // There should only be one entry in entry.addedNodes when adding a new list via streams
    const entry = entries[0];

    if (!entry.addedNodes.length) return;

    // Disable observer while we modify the childList
    this.unobserveLists();
    // Move newForm to the end of the childList
    this.listsTarget.append(this.newFormTarget);
    this.observeLists();
  }
}

The problem is solved.

Let’s talk about the second edge case: implementing the chat functionality.

We had a very simple chat attached to each dashboard: users can send ephemeral (not stored anywhere) messages and receive them in real-time. Messages have a different look depending on the context: my messages have green borders are located to the left; other messages are grey and located to the right. But we broadcast the same HTML to everyone connected to the chat. How can we let users feel the difference? That is a very common problem for chat-like applications, and, in general, it could be solved either by sending personalized HTML to per-user channels or by enhancing incoming HTML client-side. I prefer the second option, so let’s implement it.

To pass the information about the current user to JavaScript, I use meta tags:

<!-- layouts/application.html.erb -->
<head>
  <% if logged_in? %>
    <meta name="current-user-name" content="<%= current_user.name %>" data-turbo-track="reload">
    <meta name="current-user-id" content="<%= current_user.id %>" data-turbo-track="reload">
  <% end %>
  <!-- ... -->
</head>

And a small JS helper to access these values:

let user;

export const currentUser = () => {
  if (user) return user;

  const id = getMeta("id");
  const name = getMeta("name");

  user = { id, name };
  return user;
};

function getMeta(name) {
  const element = document.head.querySelector(
    `meta[name='current-user-${name}']`
  );
  if (element) {
    return element.getAttribute("content");
  }
}

To broadcast chat messages, we’ll use a Turbo::StreamChannel:

def create
  Turbo::StreamsChannel.broadcast_append_to(
    workspace,
    target: ActionView::RecordIdentifier.dom_id(workspace, :chat_messages),
    partial: "chats/message",
    locals: { message: params[:message], name: current_user.name, user_id: current_user.id }
  )
  # ...
end

Here is the original chat/message template:

<div class="chat--msg">
  <%= message %>
  <span data-role="author" class="chat--msg--author"><%= name %></span>
</div>

And the pre-existing JS code that applies different styling depending on the current user (that we will take out very soon):

// Don't get attached to this
appendMessage(html, mine) {
  this.messagesTarget.insertAdjacentHTML("beforeend", html);
  const el = this.messagesTarget.lastElementChild;
  el.classList.add(mine ? "mine" : "theirs");

  if (mine) {
    const authorElement = el.querySelector('[data-role="author"]');
    if (authorElement) authorElement.innerText = "You";
  }
}

Now, when Turbo is responsible for updating HTML, we need to do things differently. Of course, the useMutation trick could be used here as well. And that’s probably what I would do in the real project. However, my goal today is to demonstrate different ways to solve problems.

Remember, we’ve been talking about Custom Elements? (Yeah, that was several pages ago, sorry, this has turned out to be a v-e-e-ry long read.) It’s the Web API that powers Turbo. Why don’t we use that!

Let me first share an updated HTML template:

<any-chat-message class="chat--msg" data-author-id="<%= user_id %>>
  <%= message %>
  <span data-role="author" class="chat--msg--author"><%= name %></span>
</any-chat-message>

We only added the data-author-id attribute and replaced a <div> with our custom tag—<any-chat-message>.

Now let’s register the custom element:

import { currentUser } from "../utils/current_user";

// This is how you create custom HTML elements with a modern API
export class ChatMessageElement extends HTMLElement {
  connectedCallback() {
    const mine = currentUser().id == this.dataset.authorId;

    this.classList.add(mine ? "mine" : "theirs");

    const authorElement = this.querySelector('[data-role="author"]');

    if (authorElement && mine) authorElement.innerText = "You";
  }
}

customElements.define("any-chat-message", ChatMessageElement);

That’s it! Now when a new <any-chat-message> element is added on a page, it automatically updates itself if the message came from the current user. And we don’t even need Stimulus for that!

You can find the full source code from this article in this PR.

So, does Reactive Rails With Zero JavaScript exist after all? Not really. We removed a lot of JS code but eventually had to replace it with something new. This new code is different from what we had before: it’s more, I’d say, utilitarian. It’s also more advanced and requires a good knowledge of both JavaScript and the latest browser APIs, which is definitely a trade-off to consider.

P.S. I also have a similar PR for CableReady and StimulusReflex. You can compare it with the Hotwire one and share your opinions with us on Twitter.

Drop us a line if you want to enlist Martian backend (or frontend) experts to enhance your digital product.

Join our email newsletter

Get all the new posts delivered directly to your inbox. Unsubscribe anytime.