The future of full-stack Rails: Turbo Morph Drive
Topics
Translations
The “getting-back-into-full-stack” trend in web development communities is gaining more traction. Frontend frameworks are trying to embrace server components, htmx is the new black, and LiveView and LiveWire are conquering Elixir and Laravel applications, respectively. And, of course, Ruby on Rails has its newer offspring, Hotwire. Let’s explore how far you can go with the full-stack approach in Rails, and what the future might hold!
Other parts:
- The future of full-stack Rails: Turbo Morph Drive
- The future of full-stack Rails II: Turbo View Transitions
Oh, and Ruby on Rails should have probably been first in that list of full-stack technologies. Why? Because it is always trusted by small and productive teams.
As David Heinemeier Hansson, creator of the framework (Rails), said at the very recent Rails World conference, “Ruby on Rails is a one-person framework”. This “person”, by design, could only be a full-stack engineer (not a “part-stack” one).
Speaking of that Rails World conference, in terms of news and announcements, it was a very fruitful event. One thing of particular interest related to this post is Turbo 8, the new version of Turbo, a library for building modern HTML-driven web applications, which has been a default frontend component for Rails applications since Rails 7.
We still don’t know the complete list of changes and new features in Turbo 8, but a couple have already been revealed: DOM morphing and page transitions. Both features are still being properly formulated, and we’ve yet to see their final look, but that doesn’t mean we shouldn’t attempt to explore those ideas and try to apply them to today’s Hotwire applications.
In this two-part series, I’d like to explore these aforementioned frontend technologies and demonstrate how we can use them today with Turbo 7 (so you can still enjoy TypeScript 😁).
Our demo application: Turbo Music Drive
I’ve created a demo application to showcase how Turbo can be used to drive interactive web applications. I call it Turbo Music Drive, and it’s a music library and a player with some basic browsing capabilities. It’s a Rails 7.1 application with a few models and controllers to serve the data, with sprinkles of Stimulus on the client side.
Go to turbo-music-drive.fly.dev to see it in action 🎧.
Final version of the demo app
I first built all the functionality with plain Turbo (v7), and even that version looked pretty neat, given I didn’t have to write a single line of JavaScript. Just take a look at the baseline version of the application:
The plain, Turbo version of the demo app
Isn’t it cool? I think it is. However, if you give it a more thorough look, you can spot some imperfections. No worries, we’re going to fix them all.
To morph or not to morph?
Before doing a deep dive into the topic of morphing, let’s recall how Turbo performs page updates related to navigation.
Turbo Drive intercepts navigation events (clicks on links, form submissions, and so on), performs an AJAX request in the background, then it updates both the page contents with the new HTML and the browser history. We’re going to focus on the “updates the page contents” part.
Currently, Turbo Drive updates the page contents by replacing the whole <body> element with the new HTML. So, it could be simplified to the following code:
render(newHTML) {
document.body.innerHTML = newHTML;
}The problem with this approach is that we lose the local state of the browser page. We can look at our demo application to see a couple of examples of this problem:
Scroll and CSS transition glitches
First, we can see that the scroll position of the albums container is reset whenever we perform the page reload while filtering tracks. Second, the sound wave animation of the player is reset whenever we navigate between pages (although the player is a permanent element, its HTML is preserved).
These are just some examples of potential UX problems caused by fully swapping a page’s HTML. Another example is losing focus or input field state (e.g. when implementing an autosave feature with Turbo). But if you’re aiming for a perfect user experience, you should try to avoid problems like this. And if we switch to incremental DOM updates or morphing, we can.
Turbo meets Idiomorph
Morphing is not a new thing. This technique has been used for years, primarily by other full-stack frameworks, like Phoenix LiveView and StimulusReflex, and it has proven to be a very effective technique. So, it’s not surprising that Turbo plans to adopt it as well. Luckily, we don’t have to wait for Turbo 8 to try it out—Turbo 7 is flexible enough to allow us to implement a custom rendering strategy ourselves right now.
First, we need to pick a morphing library. There are a few, but I’ve chosen Idiomorph because it’s what we’ll end up with in Turbo 8 (it also has some benefits over other libraries, like morphdom, which is what I previously used).
Then, we can integrate it into Turbo Drive using the following snippet from the handbook:
document.addEventListener("turbo:before-render", (event) => {
event.detail.render = async (prevEl, newEl) => {
await new Promise((resolve) => setTimeout(() => resolve(), 0));
Idiomorph.morph(prevEl, newEl);
};
});Note that we had to add a hack to perform asynchronous morphing; without going into too much detail, this is required to make sure Turbo caching works fine. See this issue for more information.
Let’s see if those few lines of code were enough to fix our previously-encountered problems:
Straightforward morphing fixed only the horizontal scroll issue
Unfortunately, it was not. The horizontal scroll position is preserved, but the vertical one is still being reset. Why is that?
It turns out that Turbo restores the scroll position to zero on each navigation event. This logic can be explained as follows: if a URL has changed, we assume we’re on a new page, so we must have multi-page-navigation-type behavior. In our case, we’re only updating the query string, so it’s safe to assume that this is the same page being refreshed; no need to scroll back. We can work around this by adding the following code to our callback function:
let prevPath = window.location.pathname;
document.addEventListener("turbo:before-render", (event) => {
Turbo.navigator.currentVisit.scrolled = prevPath === window.location.pathname;
prevPath = window.location.pathname;
event.detail.render = async (prevEl, newEl) => {
await new Promise((resolve) => setTimeout(() => resolve(), 0));
Idiomorph.morph(prevEl, newEl);
};
});Great! Both scrolling problems have been fixed. But what about the player animation? For some reason, it’s still being reset on every navigation event, even though we shouldn’t be touching this element at all. How strange!
Let’s add some puts-like debugging to our rendering process using CSS. We can add a default animation to every node which will highlight the element once it’s been added to the DOM tree:
@keyframes highlight {
0% {
outline: 1px solid red;
}
100% {
outline: 0px solid red;
}
}
* {
animation: highlight 0.2s ease;
}Now, we can see which parts of the page are being re-rendered. You might be surprised, but the player is being highlighted on every navigation event:
Debug DOM modifications via CSS highlighting
Investigating this took some time, but in the end, I discovered the following: data-turbo-permanent elements are removed before every rendering operation and then added back to the DOM tree instead of matching new elements (if any). This means the element is still the same, but it’s being unmounted and re-mounted, thus causing CSS animation to restart.
To fix this, we can port the data-turbo-permanent functionality to our custom morphing-based rendering. This makes a lot of sense: instead of having two different rendering engines that don’t know about each other, it’s better to keep all the logic in a single place.
Let’s change the player attribute to data-morph-permanent and add the beforeNodeMorphed callback to the Idiomorph.morph call:
event.detail.render = async (prevEl, newEl) => {
await new Promise((resolve) => setTimeout(() => resolve(), 0));
Idiomorph.morph(prevEl, newEl, {
callbacks: {
beforeNodeMorphed: (fromEl, toEl) => {
if (typeof fromEl !== "object" || !fromEl.hasAttribute) return true;
if (fromEl.isEqualNode(toEl)) return false;
if (
fromEl.hasAttribute("data-morph-permanent") &&
toEl.hasAttribute("data-morph-permanent")
) {
return false;
}
return true;
},
},
});
});Let’s see what has changed:
The player CSS animation works now
Our player stays in the DOM tree during navigations (we don’t see any highlighting), and the animation is smooth (as well as the overall user experience). Awesome!
If you think that we’re done with adopting morphing as a primary rendering strategy for Turbo, you’re wrong. We still need to take care of some other Hotwire stuff.
Morphing vs. other Hotwire components
Turbo Drive is just one part of the Hotwire family; we also have Turbo Frames and Turbo Streams, which also perform page updates. Further, Stimulus controllers are also affected by how we modify HTML since they’re connected to the DOM tree. Let’s see what we need to do to make them play nicely with morphing.
Dealing with frames
Turbo Frames can be seen as sub-pages. They’re rendered independently of the page, and hence, we must set up morphing for them separately. Luckily, it’s straightforward to do:
document.addEventListener("turbo:before-frame-render", (event) => {
event.detail.render = (prevEl, newEl) => {
Idiomorph.morph(prevEl, newEl.children, { morphStyle: "innerHTML" });
};
});We define a listener for the before-frame-render event and override the render function. This is similar to what we did for the before-render event, and the only difference is that we use the innerHTML morph style instead of the default outerHTML. This is because when a frame is updated, we only update its children.
The only frame we have in our application is the one showing listener stats on the artist page. Let’s see if there is any difference between morphing and replacing:
The listeners counter frame is broken
Oops, it looks like we broke our listeners counter; it’s not being animated or even updated anymore. Why? Well, a Stimulus controller powers the animation, and it seems it isn’t correctly handling incremental DOM updates. Let’s dig deeper into this issue.
Writing morphing-aware Stimulus controllers
Stimulus controllers are coupled with HTML elements. It’s a common practice to define some setup and teardown logic for controllers in the connect() and disconnect() callbacks respectively. That’s precisely how the animated-number controller works. That’s also why it stopped working after we migrated to morphing: the DOM element the controller attached to stays the same; we only update the data-animated-number-end-value attribute. So, how can we fix this?
In general, there are two ways to make Stimulus controllers morphing-aware: we either restart them after morphing (triggering disconnect() and connect(), thus emulating HTML replacement), or we adjust them to react on attribute updates. Turbo 8 (as of now) has decided to go with the first option, and they’ve added a callback function to restart Stimulus controllers after morphing. Sure, we could do the same, but I we’d lose the benefits of morphing for Stimulus; we already have lifecycle callbacks in Stimulus to track attribute changes (values) or updates on dependent elements (targets). Let’s use them!
All we need is to extend the animated-number controller and add a callback for the data-animated-number-end-value attribute change:
import AnimatedNumber from "stimulus-animated-number";
export default class extends AnimatedNumber {
endValueChanged(_newValue, oldValue) {
this.startValue = oldValue;
this.animate();
}
}Note that we also set the startValue to the previous endValue. This way, we can improve our animation and make it start from the last value instead of zero. That’s how we leverage the benefits of morphing for Stimulus controllers. Let’s see it in action:
The listeners counter is working again, better than ever
P.S. Turbo Morph Drive vs. Turbo 8 Page Refresh
I must confess: our Turbo Morph Drive is not the same as the upcoming Page Refresh feature of Turbo 8. Turbo is not going to switch to morphing for all page updates, only for those refreshing the current page (via the corresponding Turbo Streams action). However, I think having two different update modes within the same application may become a source of confusion for developers, and it’s a limiting factor for unveiling the power of morphing (like we saw in the Stimulus example). I recommend going with morphing all the way, but it’s up to you to decide.
What about the refresh Turbo Stream action? You can implement it yourself by following this guide by Marco Roth. My version is as follows:
import { StreamActions } from "@hotwired/turbo";
const sessionID = Math.random().toString(36).slice(4);
StreamActions.refresh = function () {
// We don't want to trigger page refreshes for the current user.
// (We assume that the current user receives the required HTML updates in response to this request)
if (this.getAttribute("session-id") !== sessionID) {
window.Turbo.cache.exemptPageFromPreview();
window.Turbo.visit(window.location.href, { action: "replace" });
}
};
document.addEventListener("turbo:before-fetch-request", (event) => {
event.detail.fetchOptions.headers["X-Turbo-Session-ID"] = sessionID;
});On the Rails side, we can store the current Turbo session ID in the Current object and use it when broadcasting the refresh stream:
# application_controller.rb
class ApplicationController < ActionController::Base
around_action :set_turbo_session_id
private
def set_turbo_session_id(&block)
Current.set(turbo_session_id: request.headers["X-Turbo-Session-ID"], &block)
end
end
# application_record.rb
class ApplicationRecord < ActiveRecord::Base
primary_abstract_class
class << self
def broadcasts_refreshes
after_commit do
Turbo::StreamsChannel.broadcast_stream_to(
self,
content: Turbo::StreamsChannel.turbo_stream_action_tag(
:refresh,
:"session-id" => Current.turbo_session_id
)
)
end
end
end
end
# in some model
class Artist < ApplicationRecord
# ...
broadcasts_refreshes
endCheck out this commit to see the complete implementation.
Our Turbo Morph Drive implementation is almost complete. Adding morphing to Turbo Streams updates also makes sense, but we’ll leave that as homework for you, dear reader.
Next time, we’ll enhance our application with beautiful page transitions. Stay tuned!
The source code for both parts can be found on GitHub.
At Evil Martians, we transform growth-stage startups into unicorns, build developer tools, and create open source products. If you’re ready to engage warp drive, give us a shout!