Backend

Hotwire: Reactive Rails with no JavaScript?

Translations

If you’re interested in translating or adapting this post, please.

It is time to cast the long-teased new magic by DHH & Co. and learn to use Hotwire beyond 5-minute tutorials. The umbrella name behind libraries for building modern web interfaces seemingly without any effort or JavaScript is on everyone’s lips since the big unveil this year. The HTML-over-the-wire approach is making ripples through the Rails universe: countless blog posts, reddits, screencasts, and five RailsConf talks this year, including the one by yours truly, are dedicated to it. Here, I want to take more space to explain Hotwire thoroughly—with code examples and testing strategies. As my favorite rock band would say, let’s get Hotwired to… self-destruct learn new tricks!

Life is short

To see the Hotwire-ing of a Rails 6 app at a glance, without further ado, feel free to study this PR.

The following article explains the code above in great detail. It is an adaptation and an extension of my RailsConf 2021 talk: “Frontendless Rails frontend”, which is already available online for all the RailsConf attendees. If you don’t have the conference tickets, don’t worry: you can find the slide deck here, and the same page will be updated with the video once all talks become public.

“This is the way”

For the last five years, I’ve been doing mostly pure backend development: REST and GraphQL APIs, WebSocket, gRPC, databases, caches, all there is behind-the-screen.

The whole frontend evolution has passed me over like a giant wave: I still don’t understand why we need to stuff every web app with reacts and webpacks. The classic HTML-first Rails way was my way (or the highway 😉). Remember the days when JavaScript in your application did not require its own MVC (or MVVM) to function? I miss those days. And these days are quietly making a hell of a comeback.

Today we are witnessing the rise of HTML-over-the-wire (yes, it’s an actual term now). Sparked by Phoenix LiveView, the approach based on pushing backend-rendered templates over WebSocket to all connected clients got a lot of traction in the Rails community, thanks to the StimulusReflex family of gems. It was finally blessed by DHH himself when he introduced Hotwire to the world early this year.

Are we standing on the verge of yet another global paradigm shift in web development? Going back to the simple mental model of server-rendered templates, this time with all the bells and whistles of reactive interfaces at little-to-none effort? As much as I’d love that, I realize it’s wishful thinking: the Big Tech is too invested in client-rendered apps ever to go back. Frontend development in the 2020s is a separate qualification and a separate industry with its own entry requirements; there’s no way we will become “full-stack” again.

However, HOTWire (see that acronym right there? Smart on Basecamp’s part, huh?) provides a much-needed alternative to the complex, or should we say “complicated”, rocket science of modern client-side programming for the browser.

For a Rails developer who is tired of doing API-only apps with no control over the presentation and who misses creating user experiences as an escape from massaging SQL and JSONs forty hours a week, Hotwire is a long-sought breath of fresh air that makes web development fun again.

In this post, I’d like to demonstrate how to apply the HTML-over-the-wire philosophy to the existing Rails application through Hotwire. As with most of my recent articles, I will use my AnyCable demo application as a guinea pig.

This app fits the task well: interactive and reactive, driven by Turbolinks and a handful of custom (Java)scripts, it also boasts a decent system tests coverage (meaning we can refactor safely). Our hotwire-ification will be done in steps that are easy to follow along:

Turbolinks has been known in the Rails world for a long time; the first major release was published as early as 2013. However, in my early days, Rails developers had a rule of thumb: If your frontend is acting weird, try disabling Turbolinks. Making third-party JS code compatible with Turbolink’s fake (read: pushState + AJAX) navigation was not a walk in the park.

I stopped avoiding Turbolinks when StimulusJS came out. It radically solved the problem of connecting and disconnecting JavaScript sprinkles by relying on the modern DOM mutation APIs. Turbolinks combined with Stimulus for code organization, and DOM manipulation yields an effortless “SPA” experience at a fraction of React-Angular development costs.

The same good old Turbolinks is now rebranded as Turbo Drive as it literally drives Turbo—the heart of Hotwire package deal.

If your app already uses Turbolinks (and mine did), switching to Turbo Drive is dead simple. It’s about renaming things.

All you need is to replace turbolinks with @hotwired/turbo-rails in your package.json and in the Gemfile—replace turbolinks with turbo-rails.

The initialization code looks a little bit different and is now even more concise:

- import Turbolinks from 'turbolinks';

- Turbolinks.start();
+ import "@hotwired/turbo"

Note that we don’t need to start Turbo Drive manually (and we can not stop it).

Some “Find & Replace” is required to update all the HTML data attributes from data-turbolinks to data-turbo.

The only change that took me some time to figure out was dealing with forms and redirects. Previously, with Turbolinks, I used remote forms (remote: true) and the Redirection concern to respond with JavaScript templates. Turbo Drive has its own built-in support for hijacking forms, so remote: true is no longer required. However, it turned out that the redirection code must be updated. Or, more precisely, redirection status code:

- redirect_to workspace
+ redirect_to workspace, status: :see_other

Using the somewhat obscure See Other HTTP response code (303) is a smart choice: it allows Turbo to rely on the native Fetch API redirect: "follow" option, so you don’t have to explicitly initiate another request to fetch the new content after form submission. According to the specification, “if status is 303 and request’s method is not GET or HEAD, a GET request must be performed automatically. Compare this with “if status is 301 or 302 and request’s method is POST—see the difference?

Other 3xx statuses are only meant for POST requests, while with Rails we usually use POST, PATCH, PUT, and DELETE.

Framing with Turbo Frames

Time to move on to something truly new: Turbo Frames.

Turbo Frames bring seamless updates to parts of the page (not to the whole page as Turbo Drive does). We can say it’s very similar to what <iframe> does but without creating separate windows, DOM trees, and security nightmares that come with it.

Let’s take a look at the example usage.

AnyCable demo application (called AnyWork) allows you to create dashboards with multiple ToDo lists and a chat. A user can interact with items in different lists: add them, delete them, and mark them as completed.

Turbo Frames in action: each item is its own frame

Originally, completing and deleting items were backed by AJAX requests and a custom Stimulus controller. I decided to rewrite this functionality using Turbo Frames to go HTML all-in.

How can we decompose our ToDo lists to handle individual item updates? Let’s turn each item into a frame!

<!-- _item.html.rb -->
<%= turbo_frame_tag dom_id(item) do %>
  <div class="any-list--item<%= item.completed? ? " checked" : ""%>">
    <%= form_for item do |f| %>
      <!-- ... -->
    <% end %>
    <%= button_to item_path(item), method: :delete %>
      <!-- ... -->
    <% end %>
  </div>
<% end %>

We did three important things here:

  • wrapped our item container into a <turbo-frame> tag via the helper and passed a unique identifier (check out the handy dom_id method from ActionView);
  • added an HTML form to make Turbo intercept submissions and update the contents of the frame; and
  • used the button_to helper with method: :delete, which also creates an HTML form under the hood.

Now, whenever a form is submitted within the frame, Turbo intercepts the submissions, performs AJAX request, extracts a frame with the same ID from the response HTML, and replaces the contents of this frame.

All of the above works with exactly zero hand-written JavaScript!

Let’s take a look at our updated controller code:

class ItemsController < ApplicationController
  def update
    item.update!(item_params)

    render partial: "item", locals: { item }
  end

  def destroy
    item.destroy!

    render partial: "item", locals: { item }
  end
end

Note that we respond with the same partial when we delete an item. But we need to remove the item’s HTML node, not update it. How can we do that? We can respond with an empty frame! Let’s update our partial to do that:

<!-- _item.html.rb -->
<%= turbo_frame_tag dom_id(item) do %>
  <% unless item.destroyed? %>
    <div class="any-list--item<%= item.completed? ? " checked" : ""%>">
      <!-- ... -->
    </div>
  <% end %>
<% end %>

You’ve probably asked yourself a question: “How do we trigger form submission when marking an item as completed?” In other words, how to make checkbox state change trigger submit a form? We can do that by defining an inline event listener:

<%= f.check_box :completed, onchange: "this.form.requestSubmit();" %>

The requestSubmit() is not supported by all modern browsers yet. Consider using the polyfill.

NB: Using requestSubmit() and not submit() is important: the former triggers the “submit” event that could be intercepted by Turbo, and the latter does not.

To sum up, we could get rid of all the custom JS for this particular functionality just by altering our HTML template a bit and simplifying the controller’s code. I’m excited, aren’t you?

We can go further and convert our lists into frames too. That would allow us to switch from Turbo Drive page updates to specific node updates when adding a new item. You can try this at home!

It’s also possible to trigger the whole page or another frame update from within a frame (see the docs), but there is currently no way to update two independent frames.

Imagine you also want to display a flash notification to a user whenever an item is completed or deleted (“Item has been successfully removed”). Can we do this with Turbo Frames? Sounds like we need to wrap our flash messages container into a frame and push the updated HTML along with the markup for the item. That was my initial idea, and it didn’t work out: frame updates are scoped to the initiator frame. Thus, we can not update anything outside of it.

After some research, I found out that Turbo Streams could help with that.

Streaming with Turbo Streams

Compared to Drive and Frames, Turbo Streams is a completely new technology. Unlike those two, Streams are explicit. Nothing happens automagically and you are in charge of what and when should be updated on a page. And to do that, you need to use special <turbo-stream> elements.

Let’s take a look at the example stream element:

<turbo-stream action="replace" target="flash-alerts">
  <template>
    <div class="flash-alerts--container" id="flash-alerts">
      <!--  -->
    </div>
  </template>
</turbo-stream>

If want to learn more, check out the source code of the <turbo-stream> element here.

This element is responsible for replacing (action="replace") the node under DOM ID flash-alerts with the new HTML contents passed inside the <template> tag. Whenever you drop such <turbo-stream> element on the page, it immediately performs the action and destroys itself. Under the hood, it uses HTML Custom Elements API—yet another example of using modern web APIs for the sake of developer happiness (i.e., less JavaScript 🙂).

I would say that Turbo Streams are a declarative alternative to the JavaScript templates of old. In 2010s we wrote it like this:

// destroy.js.erb
$("#<%= dom_id(item) %>").remove();

And now we do the following:

<!--  destroy.html.erb -->
<%= turbo_stream.remove dom_id(item) %>

Currently, there are only five actions available: append, prepend, replace, remove, and update (replaces only the text content of the node). We’ll talk about this limitation and how to overcome it below.

Let’s return to our initial problem: showing the flash notification in response to the ToDo item’s completion or deletion.

We want to respond with both <turbo-frame> and <turbo-stream> updates at once. How can we do that? Let’s add a new partial template for that:

<!-- _item_update.html.erb -->
<%= render item %>

<%= turbo_stream.replace "flash-alerts" do %>
  <%= render "shared/alerts" %>
<% end %>

Add a little change to the ItemsController:

+    flash.now[:notice] = "Item has been updated"

-    render partial: "item", locals: { item }
+    render partial: "item_update", locals: { item }

Unfortunately, the code above doesn’t work as expected: we don’t see any flash alerts.
After digging through the documentation, I found that Turbo expects HTTP response to have the text/vnd.turbo-stream.html content type in order to activate stream elements. Okay, let’s do that:

-    render partial: "item_update", locals: { item }
+    render partial: "item_update", locals: { item }, content_type: "text/vnd.turbo-stream.html"

We now have the opposite situation: flash messages work, but the items’ content is not updated 😞. Am I asking too much from Hotwire? Reading through the Turbo source code, I found that mixing streams and frames like this is not possible.

It turned out, there are two ways to implement this functionality:

  • Use streams for everything.
  • Put <turbo-stream> inside the <turbo-frame>.

The second option, in my opinion, runs counter the idea of reusing HTML partials for regular page loads and Turbo updates. So, I went with the first one:

<!-- _item_update.html.erb -->
<%= turbo_stream.replace dom_id(item) do %>
  <%= render item %>
<% end %>

<%= turbo_stream.replace "flash-alerts" do %>
  <%= render "shared/alerts" %>
<% end %>

Mission complete. But at what cost? We had to add a new template for this use case. And I’m afraid that in a real-world application, the number of such ad hoc partials would grow as the app evolves.

UPDATE (2021-04-13): Alex Takitani suggested a more elegant solution: using a layout to update flash contents. We can define the application layout for Turbo Stream responses as follows:

<!-- layouts/application.turbo_stream.erb -->
<%= turbo_stream.replace "flash-alerts" do %>
  <%= render "shared/alerts" %>
<% end %>

<%= yield %>

Then, we need to remove the explicit rendering from the controller (because otherwise layout wouldn’t be used):

   def update
     item.update!(item_params)

     flash.now[:notice] = "Item has been updated"
-
-    render partial: "item_update", locals: { item }, content_type: "text/vnd.turbo-stream.html"
   end

NB: Don’t forget to add format: :turbo_stream to the corresponding requests in your controller/request specs to make implicit rendering work.

And convert our _item_update partial to update Turbo Stream template:

<!-- update.turbo_stream.erb -->
<%= turbo_stream.replace dom_id(item) do %>
  <%= render item %>
<% end %>

Cool, right? This is the Rails way!

Now let’s move to some real(-time) streaming.

Turbo Streams are often mentioned in the context of real-time updates (and are usually compared to StimulusReflex).

Let’s see how we can build lists synchronization on top of Turbo Streams:

Turbo Streams in action, synchronizing page updates for all connected clients

Before Turbo, I had to add a custom Action Cable channel and a Stimulus controller to handle broadcasts. I also needed to take care of the message format as I had to distinguish between deletion and completion of the items. In other words, a lot of code to maintain.

Turbo Streams takes care of most of this: the turbo-rails gem comes with a general Turbo::StreamChannel and a helper (#turbo_stream_from) to create a subscription right from HTML:

<!-- worspaces/show.html.erb -->
<div>
  <%= turbo_stream_from workspace %>
  <!-- ... -->
</div>

Although we’re using Action Cable here, Turbo Streams are transport-agnostic. You can push updates using any cable you want, just write an adapter for it.

In the controller, we already had the #broadcast_new_item and #broadcast_changes “after action” hooks responsible for broadcasting updates. All we need now is to switch to Turbo::StreamChannel:

 def broadcast_changes
   return if item.errors.any?
   if item.destroyed?
-    ListChannel.broadcast_to list, type: "deleted", id: item.id
+    Turbo::StreamsChannel.broadcast_remove_to workspace, target: item
   else
-    ListChannel.broadcast_to list, type: "updated", id: item.id, desc: item.desc, completed: item.completed
+    Turbo::StreamsChannel.broadcast_replace_to workspace, target: item, partial: "items/item", locals: { item }
   end
 end

This migration went smoothly. Almost. All the controller unit tests that were verifying broadcasts (#have_broadcasted_to) have failed.

Unfortunately, Turbo Rails doesn’t provide any testing tools (yet?), so I have to write my own, an all-too-familiar tale:

module Turbo::HaveBroadcastedToTurboMatcher
  include Turbo::Streams::StreamName

  def have_broadcasted_turbo_stream_to(*streamables, action:, target:) # rubocop:disable Naming/PredicateName
    target = target.respond_to?(:to_key) ? ActionView::RecordIdentifier.dom_id(target) : target
    have_broadcasted_to(stream_name_from(streamables))
      .with(a_string_matching(%(turbo-stream action="#{action}" target="#{target}")))
  end
end

RSpec.configure do |config|
  config.include Turbo::HaveBroadcastedToTurboMatcher
end

This is how I use my new matcher in tests:

 it "broadcasts a deleted message" do
-  expect { subject }.to have_broadcasted_to(ListChannel.broadcasting_for(list))
-    .with(type: "deleted", id: item.id)
+  expect { subject }.to have_broadcasted_turbo_stream_to(
+    workspace, action: :remove, target: item
+  )
 end

Going real-time with Turbo goes well so far! A bunch of code has been removed.

And we still haven’t written a single line of JavaScript code. Is this the real life?

Is this just fantasy? When am I going to wake up? Well, right now.

Beyond Turbo, or using Stimulus and custom elements

During the Turbo migration, I stumbled upon a couple of use-cases where using the existing API was not enough, so I finally had to write some JavaScript code!

Use-case number one: adding new lists to the dashboard in real-time. How is it different to from a list of items example from the previous chapter? The markup. Let’s take a look at the dashboard layout:

<div id="workspace_1">
  <div id="list_1">...</div>
  <div id="list_2">...</div>
  <div id="new_list">
    <form>...</form>
  </div>
</div>

That is where CableReady from the StimulusReflex clan beats the opponent: it supports more than 30 actions, including insert_adjacent_html.

The last element is always the new list form container. Whenever we add a new list, it’s inserted right before the #new_list node. Do you remember that Turbo Streams support only five actions? Do you see where the problem is? Here is the code I used originally:

handleUpdate(data) {
  this.formTarget.insertAdjacentHTML("beforebegin", data.html);
}

To implement a similar behavior using Turbo Streams, we need to add a hack to move the list to the right place immediately after being added via the stream. So, let’s add our own JavaScript sprinkle.

Let’s first give our task a formal definition: “When a new list item is appended to the workspace container, it should be positioned right before the new form element.” The word “when” here means that we need to observe the DOM and react to changes. Doesn’t it sound familiar? Yes, we already mentioned the MutationObserver API in relation to Stimulus! Let’s use it.

Luckily, we don’t have to write advanced JavaScript to use this feature; we can use stimulus-use (forgive me for this unavoidable tautology). Stimulus Use is a collection of useful behaviors for Stimulus controllers, simple snippets to solve complex problems. In our case, we need the useMutation behavior.

The code for the controller is pretty concise and self-explanatory:

import { Controller } from "stimulus";
import { useMutation } from "stimulus-use";

export default class extends Controller {
  static targets = ["lists", "newForm"];

  connect() {
    [this.observeLists, this.unobserveLists] = useMutation(this, {
      element: this.listsTarget,
      childList: true,
    });
  }

  mutate(entries) {
    // There should be only one entry in case of adding a new list via streams
    const entry = entries[0];

    if (!entry.addedNodes.length) return;

    // Disable observer while we modify the childList
    this.unobserveLists();
    // Move newForm to the end of the childList
    this.listsTarget.append(this.newFormTarget);
    this.observeLists();
  }
}

The problem is solved.

Let’s talk about the second edge case: implementing the chat functionality.

We had a very simple chat attached to each dashboard: users can send ephemeral (not stored anywhere) messages and receive them in real-time. Messages have a different look depending on the context: my messages have green borders are located to the left; other messages are grey and located to the right. But we broadcast the same HTML to everyone connected to the chat. How can we let users feel the difference? That is a very common problem for chat-like applications, and, in general, it could be solved either by sending personalized HTML to per-user channels or by enhancing incoming HTML client-side. I prefer the second option, so let’s implement it.

To pass the information about the current user to JavaScript, I use meta tags:

<!-- layouts/application.html.erb -->
<head>
  <% if logged_in? %>
    <meta name="current-user-name" content="<%= current_user.name %>" data-turbo-track="reload">
    <meta name="current-user-id" content="<%= current_user.id %>" data-turbo-track="reload">
  <% end %>
  <!-- ... -->
</head>

And a small JS helper to access these values:

let user;

export const currentUser = () => {
  if (user) return user;

  const id = getMeta("id");
  const name = getMeta("name");

  user = { id, name };
  return user;
};

function getMeta(name) {
  const element = document.head.querySelector(
    `meta[name='current-user-${name}']`
  );
  if (element) {
    return element.getAttribute("content");
  }
}

To broadcast chat messages, we will use a Turbo::StreamChannel:

def create
  Turbo::StreamsChannel.broadcast_append_to(
    workspace,
    target: ActionView::RecordIdentifier.dom_id(workspace, :chat_messages),
    partial: "chats/message",
    locals: { message: params[:message], name: current_user.name, user_id: current_user.id }
  )
  # ...
end

Here is the original chat/message template:

<div class="chat--msg">
  <%= message %>
  <span data-role="author" class="chat--msg--author"><%= name %></span>
</div>

And the pre-existing JS code that applies different styling depending on the current user that we will take out very soon:

// Don't get attached to it
appendMessage(html, mine) {
  this.messagesTarget.insertAdjacentHTML("beforeend", html);
  const el = this.messagesTarget.lastElementChild;
  el.classList.add(mine ? "mine" : "theirs");

  if (mine) {
    const authorElement = el.querySelector('[data-role="author"]');
    if (authorElement) authorElement.innerText = "You";
  }
}

Now, when Turbo is responsible for updating HTML, we need to do things differently. Of course, the useMutation trick could be used here as well. And that’s probably what I would do in the real project. However, my goal today is to demonstrate different ways to solve problems.

Remember, we’ve been talking about Custom Elements (yeah, that was several pages ago, sorry, it turned out to be a v-e-e-ry long read)? It is the Web API that powers Turbo. Why don’t we use that!

Let me first share an updated HTML template:

<any-chat-message class="chat--msg" data-author-id="<%= user_id %>>
  <%= message %>
  <span data-role="author" class="chat--msg--author"><%= name %></span>
</any-chat-message>

We only added data-author-id attribute and replaced <div> with our custom tag—<any-chat-message>.

Now let’s register the custom element:

import { currentUser } from "../utils/current_user";

// This is how you create custom HTML elements with a modern API
export class ChatMessageElement extends HTMLElement {
  connectedCallback() {
    const mine = currentUser().id == this.dataset.authorId;

    this.classList.add(mine ? "mine" : "theirs");

    const authorElement = this.querySelector('[data-role="author"]');

    if (authorElement && mine) authorElement.innerText = "You";
  }
}

customElements.define("any-chat-message", ChatMessageElement);

That’s it! Now when a new <any-chat-message> element is added on a page, it automatically updates itself if the message came from the current user. And we don’t even need Stimulus for that!

You can find the full source code from this article in this PR.

So, does Reactive Rails With Zero JavaScript exist after all? Not really. We removed a lot of JS code but eventually had to replace it with something new. This new code is different from what we had before: it’s more, I’d say, utilitarian. It is also more advanced and requires a good knowledge of JavaScript and the latest browser APIs, which is definitely a trade-off to consider.

P.S. I also have a similar PR for CableReady and StimulusReflex. You can compare it with the Hotwire one and share your opinions with us on Twitter.


Drop us a line if you want to enlist Martian backend (or frontend) experts to enhance your digital product.

Humans! We come in peace and bring cookies. We also care about your privacy: if you want to know more or withdraw your consent, please see the Privacy Policy.