Ruby Bytes, or generating standalone generators

Ruby on Rails follows Ruby’s “developer happiness” philosophy of being “optimized for happiness”. And for me, as a developer, one of the key elements of happiness is productivity. And Rails absolutely nailed it: convention over configuration, balanced abstractions (aka “complexity compression”), and, of course, the full-fledged omakase menu—all of these help you ship features, without the need to struggle with tooling (well, at least in the beginning, but that’s a different story…)

Let’s take a step backwards and unwrap a particular productivity tool from the Rails’ tool belt—generators. More precisely, I want to discuss one specific kind of generator—application templates.

Ruby Bytes was born once I realized that a Rails-agnostic way of creating and distributing application templates is useful in its own right. Let’s get to it.

From custom scripts to Thor to RailsBytes

Application templates were introduced in Rails 2.3—14 years ago! These templates can be used to create new Rails apps with the predefined configuration (rails new -m <location>), or to enhance an existing app (rails app:template LOCATION=<location>).

Technically, templates are similar to generators. The major difference is that you cannot define options (which are passed via CLI arguments).

In the early days, the implementation of generators was a part of the Rails codebase and was a thing in and of itself. However, for Rails 3.0, generators were rewritten on top of the Thor gem—a toolkit to build command-line utilities. Well, the word “rewritten” can be applied to both Rails and Thor. The Rails team (more precisely, José Valim) invested a lot into Thor development and upgrading its design, with the aim of making it a core piece of Rails generators. Since then, Thor has been adopted by many Ruby tools, such as Bundler, Vagrant, and, most recently, MRSK.

The fact that Rails generators (and, thus, templates) rely on Thor is important for this post. And soon, you’ll see why.

Back to Rails application templates, we should note one important feature: the location of the template to apply can either be a file path or a URL. Thus, it’s possible to use templates hosted somewhere on the Internet! And that leads us to the next milestone in the history of Rails templates.

Although this feature has been in Rails since 2009, it was only in 2020 that it turned into something even bigger: Rails Bytes, a community-driven library of application templates, was announced by Chris Oliver. (It’s worth noting that another similar project, Rubidium by Dave Kimura, was created around the same time.)

Rails Bytes made it easier (and, thus, it became more popular) to share Rails application templates. The most common use case for templates is to configure a group of gems/tools to work together.

For a single library, you can create a generator as a part of the gem and register it within a Rails app (for example, AnyCable’s rails g anycable:install works this way). But when you need to stitch many pieces together, this isn’t an option.

Application templates as installation wizards, or I how ended up here

Our first steps with view_component-contrib

When I started working on view_component-contrib, a collection of extensions for view_component, the most significant part of the library was not in the code itself but in the proposed code organization for Rails projects using view components. This code organization involves configuring Rails itself, View Component, frontend tools, and so on. I came to the understanding that writing documentation is not enough and is too error-prone. So, I started looking for ways to automate the configuration process.

I had some prior experience with interactive Rails generators, so building a Rails application template seemed like a viable option. But view_component-contrib is a gem; why not build a regular Rails generator? Well, first of all, I expected the template to change more frequently than the gem’s code. Releasing a new version every time the template is modified looked like a task with massive overhead. Secondly, I wanted to let users configure everything using a single command (yep, not even two). And finally, I just wanted to play with Rails Bytes 🙂

So, I started writing a template and quickly realized that the code was barely readable. A Rails template must be a single file; thus, if you want, for example, to copy some source files, you must embed them into the template’s code.

I decided to overcome this problem by splitting the template into components or partials, and programmatically gluing or compiling them together.

With a bit of ERB and a couple of Rake tasks, I created a template generation and publishing pipeline for view_component-contrib. The experiment went very well, so I decided to use this technique where appropriate.

Automatic Rails apps dockerization with Ruby on Whales

The next time I referred to Rails templates was when I was working on a major upgrade for my Ruby on Whales tutorial. Even though we had a detailed article and configuration examples on GitHub, getting started with using Docker for Rails application development required a lot of manual work.

For this upgrade, I set out to bring building Rails templates to the next level. Not only did I re-use the compilation code, but I also introduced testing utilities. This application template became a full-featured project.

It became clear to me that the common code (compilation, testing, publishing) must be abstracted away, so I wouldn’t need to copy-paste it the next time I decided to create a template.

Also, with Ruby on Whales, there was a paradoxical situation. The template aims to move development from the host machine to Docker to avoid installing all the system dependencies but to apply this template, you need to at least provision the Rails app (since you need to execute the rails app:template task).

To solve both problems, reusability and dependency on Rails, I decided to build something new.

Introducing Ruby Bytes

My first goal was to make the Ruby on Whales template work without Rails. Then, when I worked on yet another generator, I realized that having a Rails-agnostic mechanism to create and distribute application templates is valuable on its own. So, my goal transformed into bringing rails app:template to the whole Ruby world (and beyond!)

First, I dug through the Rails source code to find out how exactly Rails templates work. The app:template command entry-point is the Rails::Generators::AppGenerator#apply method:

task template: :environment do
  template = ENV["LOCATION"]
  # ...
  generator = Rails::Generators::AppGenerator.new [Rails.root], {}, { destination_root: Rails.root }
  generator.apply template, verbose: false
end

The good news was that the #apply method comes from Thor! Thus, we can try to use a Rails template with a custom Thor command.

Thor comes with a self-titled CLI, thor, which can be used to install and execute custom commands. So, I created a minimal Thor command:

# rbytes-thor.rb
class Rbytes < Thor
  desc "template", "Load and run generator from RailsBytes"
  def template(url)
    puts "Run template from: #{url}"
    Base.new.apply url
  end

  class Base < Thor::Group
    # ...
  end
end

Then, I installed it globally and tried to apply the “hello world” template from Rails Bytes:

$ thor install rbytes-thor.rb

$ thor rbytes:template https://railsbytes.com/script/x7msKX

Runtemplate from: https://railsbytes.com/script/x7msKX
       apply  https://railsbytes.com/script/x7msKX
hello world from https://railsbytes.com 👋

And it worked!

After some experimenting, I found that many actions are implemented in Thor (e.g., #file and #inject_into_file), but there are some that come with Rails via the Rails::Generators::Actions. So, I copied the entire module to my custom Thor command, spiced it up with some String class extensions, and thus, made the Ruby on Whales template work without Rails!

This is how the Ruby Bytes project was born. Why Ruby Bytes? Its primary purpose: build standalone Ruby application templates to be used with or without Rails via Rails Bytes.

Ruby Bytes is a toolkit to build, deploy and install Rails-like application templates for Ruby applications.

The bit of story that concerns extracting the template compilation and testing logic is not very interesting, so let’s skip it. If you’re curious, you can take a look at an example project, such as ruby-on-whales.

The cherry on top was adding a reusable GitHub Action to compile and publish templates to Rails Bytes. All you need is to provide your credentials:

jobs:
  publish:
    uses: palkan/rbytes/.github/workflows/railsbytes.yml@master
    with:
      template: template/ruby-on-whales.rb
    secrets:
      RAILS_BYTES_ACCOUNT_ID: "$"
      RAILS_BYTES_TOKEN: "$"
      RAILS_BYTES_TEMPLATE_ID: z5OsoB

Ruby Bytes comes with a CLI to compile, publish and install templates. We also ship a custom template to generate new Ruby Bytes projects. So, if you want to build a complex Rails (or not Rails) application template, Ruby Bytes is here to help you!

And Evil Martians are here to help you, too! We can assist with your Rails project, a web or mobile application, or even if you’re in need of expert problem solving with product design, frontend, backend, or software reliability, we’re good to go! Reach out now!