System of a test II: Robust Rails browser testing with SitePrism

In the first part of our Rails system tests series, we only covered the topic of setting up the environment for browser tests in Rails. We didn’t even show a piece of an actual test, or scenario. Why so? First, the article couldn’t fit more than it could fit. Secondly, I didn’t want to go over the basics of writing tests with Capybara—there are plenty of great resources from the community if you’re just getting started with end-to-end testing in Rails world.

To see how we setup our system tests with Ferrum, Cuprite and Docker—check out the first part of this series: Proper browser testing in Ruby on Rails.

So, this time we are going to skip the basics and talk about some advanced techniques in writing system tests. These are the topics this post will cover:

• Introducing SitePrism
• Capybara: configuring test identifiers
• Bonus: non-leaking test IDs with Slim

Introducing SitePrism

Browser tests are getting more and more popular since they’ve been bundled into Rails as system tests. However, many developers tend to avoid them because of slow execution times and high maintainability costs. There is not much that can be done about the speed—browser tests will never be as fast as unit tests. This is a compromise we’re willing to take, as unit test don’t allow us to test the end user experience at all.

Maintainability, on the other hand, is something that can be improved. An answer to the question “Why don’t you write end-to-end tests?” that I often hear from fellow developers goes like this: “HTML markup changes break builds, thus, we end up refactoring tests too often.” That is also true for most of the test suites I’ve seen: even if HTML/CSS/JS change doesn’t affect user experience, it could make our tests red.

Let’s consider an example. Assume that we have a movie listing page, and we want to test the searching functionality:

before_all do
  create(:movie, title: "Hong Faan Kui")
  create(:movie, title: "Rocky")
  create(:movie, title: "Red Heat")
end

scenario "I can search for a movie" do
  visit "/movies"

  # first, make sure the page is fully loaded;
  # we use an async matcher here (have_text)
  within ".totals" do
    expect(page).to have_text "Total movies: "
  end

  # perform action—search for movies with "R" in their title
  within ".topbar" do
    fill_in :q, with: "R", fill_options: { clear: :backspace }
    click_on "Search"
  end

  # check expectations—we should see two movies, sorted alphabetically
  within "table" do
    # first, check the total number of rows in the table
    expect(page).to have_css "tr", count: 2

    # then, check the contents
    within "tr:first-child" do
      # NOTE: We don't need to wait here, searching has been already performed
      # Setting `wait` to zero allows us to fail faster in case of a mismatch
      expect(page).to have_text "Red Heat", wait: 0
    end

    within "tr:nth-child(2)" do
      expect(page).to have_text "Rocky", wait: 0
    end
  end
end

Although the test above does its job, it has a number of problems:

• Readability leaves much to be desired. All these CSS selectors and methods arguments ({clear: :backspace}) make it harder to extract the actual user story from this test.
• Having N selectors means having N potential points of failure caused by frontend changes. The more tests you have for this page (or other pages using similar UI components), the higher the maintenance cost.

Both problems are results of the leaking implementation details: we rely on the internal “API” (our markup). Hence, our test verifies not only the user story but our frontend code, too.

Well, it’s hardly possible to write automated tests without using some technical details; we can’t tell a browser: “Click this blue button, please, sir”. What we can do is keep the number of references to the internal implementation as small as possible and take them away from the test scenarios. Yes, we need an abstraction! And here comes the site_prism gem.

Let me first show you the rewritten scenario:

# See below for `prism` definition
subject { prism.movies }

scenario "I can search for a movie" do
  subject.load

  # perform action: search for movies with "R" in their title
  subject.search "R"

  # check expectations: we should see two movies, sorted alphabetically
  expect(subject.table).to have_rows(count: 2)

  expect(subject.table.first_row).to have_text "Red Heat", wait: 0
  expect(subject.table.second_row).to have_text "Rocky", wait: 0
end

Looks much better to me: every line of code represents either an action or an expectation, no HTML/CSS at all. Let me show you what hides under the hood.

SitePrism, as stated in their Readme, “gives you a simple, clean and semantic DSL for describing your site using the Page Object Model pattern.” Technically speaking, that means that you describe pages of your application using Ruby classes and a bit of DSL. Here’s a definition for our MoviesPage class:

class MoviesPage < SitePrism::Page
  # Element defines a method to return the matching node
  # (it's similar to `find(".totals")`)
  element :totals, ".totals"

  # Section is a similar to a page, but describes only a part of it.
  section :table, TableSection, "table"
  section :topbar, SitePrism::Section, ".topbar"

  # Load validation allows to ensure that the page is loaded
  # (after we call `page.load`).
  # So, we don't need to do that in our tests.
  load_validation { has_totals? && totals.has_text?("Total movies") }

  # We can add custom helper methods
  def search(query)
    # Calling a section with block is equal to `within(topbar) { ... }`
    topbar do
      # Here we can use all the Capybara methods
      fill_in :q, with: query, fill_options: { clear: :backspace }
      click_on "Search"
    end
  end
end

As you can see, page class extracts all the implementation details from our test:

• we describe our page structure using selectors;
• we define assertions to make sure the page is loaded—our tests shouldn’t be responsible for that;
• we declare custom methods to simplify common tasks (e.g., #search).

Now, in case our markup changes, we only need to change the page class, no need to change the tests! Another important benefit is that our pages can be modular, built from multiple sections. And these sections could be re-used!

Here is a TableSection class for our example:

class TableSection < SitePrism::Section
  element :first_row, "tr:first-child"
  element :second_row, "tr:nth-child(2)"
  element :last_row, "tr:last-child"

  elements :rows, "tr"
end

And we can use all Ruby powers to organize pages and sections: composition and inheritance, metaprogramming, etc.

NOTE: SitePrism page object is stateless (usually); it only provides a convenient interface to communicate with the current page contents (all underlying Capybara calls are delegated to Capybara.current_session).

That was a quick overview of how SitePrism works. Let’s move on to some practical tips on organizing prisms in a Rails project.

Setting up a project to work with SitePrism

First, let’s update our Gemfile:

group :test do
  gem 'site_prism', '~> 3.5.0'
end

Note that we are using a specific version of SitePrism: this is due to compatibility issues with the recent Capybara releases. Hopefully, that will be resolved in the future.

Where should we store pages? SitePrism extensive documentation doesn’t answer this question. It’s up to us to organize page objects. Let me share my setup.

Since SitePrism classes belong to the test environment, I prefer to keep them under test/ or spec/ path. More precisely, I put them inside the test/system/pages (or spec/system/pages) folder.

Then, I configure Rails autoloader to load classes from this folder, so I shouldn’t care about require-ing everything manually and resolving dependencies:

# config/application.rb

# Autoload SitePrism Page objects
# (It's not possible to add autoload_paths from env-specific configs)
config.autoload_paths << Rails.root.join("spec/system/pages") if Rails.env.test?

I am following the advice from SitePrism maintainers on using a single entry point for all the pages in the application—ApplicationPrism:

# spec/system/pages/application_prism.rb

class ApplicationPrism
  def movies
    MoviesPage.new
  end

  # ...
end

Finally, I add a prism method to access an instance of the application prism in tests:

# For RSpec
RSpec.shared_context "site_prism" do
  let(:prism) { ApplicationPrism.new }
end

RSpec.configure do |config|
  config.include_context "site_prism", type: :system
end

# For Rails Minitest
class ApplicationSystemTestCase < ActionDispatch::SystemTestCase
  # ...
  def prism
    @prism ||= ApplicationPrism.new
  end
end

Pages could be organized in namespaces, reflecting the application structure. For example:

spec/
  system/
    pages/
      library/
        resource_page.rb
        document_page.rb
      application_prism.rb
      home_page.rb
      library_prism.rb
      movies_page.rb

To access namespaced page objects, we have sub-prism classes:

class LibraryPrism
  def resource
    Library::ResourcePage.new
  end

  def document
    Library::DocumentPage.new
  end
end

class ApplicationPrism
  def library
    @library ||= LibraryPrism.new
  end
end

# in tests
scenario "I can see a library resource" do
  resource_page = prism.library.resource

  resource_page.load(id: resources(:ruby_under_microscope))

  expect(page).to have_text "Rubyのしくみ"
end

As you can see, our prism classes simply describe the file structure. To me, it seems like a duplication. We can reduce the boilerplate by adding a bit of metaprogramming to our ApplicationPrism:

class ApplicationPrism
  def initialize(root_namespace = "")
    @root_namespace = root_namespace
  end

  def respond_to_missing?(_mid, _include_private = false)
    true
  end

  def method_missing(mid, *args, &block)
    mid.to_s.camelize.then do |class_prefix|
      page = "#{root_namespace}#{class_prefix}Page".safe_constantize
      next page.new(*args, &block) if page

      ApplicationPrism.new("#{root_namespace}#{class_prefix}::")
    end
  end

  private

  attr_reader :root_namespace
end

The code above automatically resolves page and sub-prism classes if they follow the naming convention: prism.some_namespace.any => SomeNamespace::AnyPage.new.

Finally, shared components (or sections) could go into, for example, page/sections or page/components folder:

spec/
  system/
    pages/
      library/...
      sections/
        table.rb #=> Sections::Table
        form.rb  #=> Sections::Form
      application_prism.rb
      ...

Now, as we handled the file organization, let’s see some practical examples.

Dealing with forms

There is hardly an application which does not have any HTML forms. Filling and submitting forms in browser tests often becomes challenging: complex frontend components don’t play well with the click_on and fill_in helpers, locating inputs could be complicated, etc.

Let’s consider an example:

= form_tag @login_form, method: :post, url: login_path do
  .field
    label.label Email
    .control
      = f.email_field :email, class: "input", required: true
  .field
    label.label Password
    .control
      = f.password_field :password, class: "input", required: true
  .field
    .control
      = f.button type: :submit, class: "button" do
        span.icon.is-small
          i.fas.fa-check

This example uses Slim template language (we’re going to talk about it a bit later).

The corresponding test using plain Capybara can look like this:

visit "/login"

fill_in "Email", with: "L369G42@russmail.com"
fill_in "Password", with: "broen"

find("button .fa-check").click

The problem is that it doesn’t work: it would raise Capybara::ElementNotFound: Unable to find field "E-mail". That’s because our label is not associated with the input field (via for attribute). We should have been used f.label for that but we didn’t.

Let’s try to fix it:

fill_in :email, with: "L369G42@russmail.com"
fill_in :password, with: "broen"

No luck! Still the same Capybara::ElementNotFound, because Rails form helpers wrap input names based on the model name: login_form[email] and login_form[password].

Here’s the working scenario:

visit "/login"

fill_in "login_form[email]", with: "L369G42@russmail.com"
fill_in "login_form[password]", with: "broen"

find("button .fa-check").click

And now we have internal implementation details leaking into our integration tests 😢.

How to rewrite this test using page objects?

Let’s do it first in a straightforward way:

class LoginPage < SitePrism::Page
  set_url "/login"

  element :email_field, %(input[name="login_form[email]"])
  element :password_field, %(input[name="login_form[password]"])
  element :submit_btn, "button .fa-check"
end

Our test would look like this:

login_page.load

login_page.email_field.fill_in with: "L369G42@russmail.com"
login_page.password_field.fill_in with: "broen"

login_page.submit_btn.click

We removed all the internals from the tests itself, now they live in our page class. Does this approach improve maintainability? Not really. We just moved the problematic code from one place to another, we still need to explicitly declare all the fields and corresponding selectors.

This is an example of overspecified page objects: we explicitly declare elements which could be located in a more human-friendly (not machine-friendly) way. Remember: we do care about readability, too.

This example also demonstrates how we can lose some user experience information by overusing page objects: login_page.submit_btn doesn’t tell us anything about the nature of this button: “What text on it? Oh, it’s an icon! Which one?” That makes this test less valuable from the specification point of view.

This is how I would like to see this scenario written in the ideal world:

login_page.load

login_page.form do |form|
  form.fill_in "Email", with: "L369G42@russmail.com"
  form.fill_in "Password", with: "broen"

  find(:icon, "check").click
end

The form filling part is equal to our very first attempt. Except from the fact that now it works. Here’s how we can achieve this:

class LoginPage < SitePrism::Page
  set_url "/login"

  # It makes sense to define section classes inside the page class
  # unless you want to re-use them between pages
  class Form < SitePrism::Section
    # Override the locator to automatically wrap the name of the field
    def fill_in(name, **options)
      super "login_form[#{name.underscore}]", **options
    end
  end

  section :form, Form, "form"
end

We can generalize this approach in a shared form class:

module Sections
  class Form < SitePrism::Section
    class << self
      attr_accessor :prefix
    end

    # Section extends Forwardable
    def_delegators "self.class", :prefix

    def fill_in(name, **options)
      super field_name(name), **options
    end

    private

    def field_name(name)
      return name if prefix.nil?

      name = name.to_s

      # "Fully-qualified" name has been passed
      return name if name.include?("[")

      name = name.underscore

      # Manually replacing spaces with underscores required
      # for compound names, such as "Last Name"
      name.tr!(" ", "_")

      "#{prefix}[#{name}]"
    end
  end
end

And now we can define our form section as follows:

class LoginPage < SitePrism::Page
  set_url "/login"

  section :form, Sections::Form, "form" do
    self.prefix = "login_form"
  end
end

Usually, shared form classes contain helpers to fill unusual fields: rich text editors, JS date pickers, etc. Here is, for example, the helper we use to fill Froala text inputs:

# It's a simplified version which assumes there is only one editor in the form
def fill_editor(with:)
  find(:css, ".fr-element").click.set(with)
end

Let’s skip the find(:icon, "check").click explanation for a moment and talk about other use-cases for page objects.

“Headless” pages and default URL parameters

So far, we defined page classes for specific pages (i.e., with specific URLs). What if our application consists of many structurally similar pages like, for example, in admin UIs? Should we define multiple page objects just to load different pages (to specify different set_url values)?

There is a trick I once used in a project. I named it headless pages. We were dealing with an admin UI that had three main types of pages: index, form and card. The resulting entities looked mostly the same, so we decided to use only 3 page objects instead of 3N (where N is the number of entities).

Here is the simplified version of the IndexPage class:

class IndexPage < BasePage
  set_url "/{path}"

  section :table, Components::Table, ".table"
  section :filter, Components::FilterList, ".filter-list-tabs"

  element :counter, ".total-counter"

  def search(query)
    # ...
  end
end

Note that the URL is fully parameterized (via the path variable). Thus, we can use this page object the following way:

subject { prism.index }

scenario do
  subject.load(path: "/users")

  # or
  expect(subject).to be_displayed(path: "/users")
end

We decided to go further and made it possible to specify URL parameters during the page object initialization:

subject { prism.index(path: "/users") }

scenario do
  subject.load

  # or
  expect(subject).to be_displayed
end

This trick is useful for “non-headless” pages as well (for example, for show pages with identifiers). Here is how we implemented it in our BasePage class:

class BasePage < SitePrism::Page
  # You can create a new page either by passing a params hash
  # or by providing a record to be used as an /:id param (should respond to #to_param)
  def initialize(record_or_params = {})
    record_or_params = { id: record_or_params.to_param } if record_or_params.respond_to?(:to_param)
    @default_params = record_or_params
    super()
  end

  def load(expansion_or_html = {}, &block)
    expansion_or_html.reverse_merge(default_params) if expansion_or_html.is_a?(Hash)
    super
  end

  def displayed?(*args)
    expected_mappings = args.last.is_a?(::Hash) ? args.pop : {}
    expected_mappings.reverse_merge(default_params)
    super(*args, expected_mappings)
  end

  private

  attr_reader :default_params
end

More tips

In the end of this part, I would like to share a few more tips on writing page object classes.

To reduce the number of selectors to maintain even further, we can specify the default selectors for shared sections.
For example:

class Sections::Form < SitePrism::Section
  set_default_search_arguments "form"
end

And now we can omit the selector when using this section:

class LoginPage < SitePrism::Page
  # ...

  section :form, Sections::Form
end

Of course, we can always override it for a particular page.

Let’s take a look at what else we have in our BasePage class:

class BasePage < SitePrism::Page
  class << self
    # Override .section to create anonymous section easier
    def section(name, *args, &block)
      return super if args.first.is_a?(Class) || block_given?

      super(name, BaseSection, *args)
    end
  end
end

I often use anonymous sections, i.e., sections not backed by specific classes. Sections are more powerful than elements. For example, we can use them for scoping:

some_page.some_section do
  # now all the matcher and methods are executed `within(some_section)`
end

The patch above allows us to define anonymous section a bit shorter:

# before
section :my_section, Section::SitePrism, ".selector"

# after
section :my_section, ".selector"

Finally, if you ever wanted to sleep to wait for some asynchronous event, consider using waiting instead:

def wait(*args)
  SitePrism::Waiter.wait_until_true(*args) { yield }
end

This helper could be used inside other page methods (where you cannot use expectations). We used it, for example, to select values from autocomplete inputs:

def select_autocomplete(value, query:, from:)
  # enter query
  find(:field, from).set(query)
  # wait for autocomplete to return the list of matching items
  wait { has_text? value }
  # click on the matching item
  find(".dropdown-item", text: value).click
end

Capybara: configuring test identifiers

So far, we’ve been using many different ways of locating elements and sections: CSS classes, HTML attributes and tags, and the mix of all of them. Although we minimized the number of HTML/CSS occurrences in our testing codebase, we still need to keep them in sync with our frontend code. How to make our identification logic more stable and consistent?

In the old days of jQuery, we had the following idea to separate JS-related CSS from the actual styles—using specific prefixes (usually, .js-*) for classes used by JS code. What about borrowing this idea for testing? Let’s come up with a specific HTML attribute to use for elements identification in tests, say, test_id.

Did you know that Capybara has a built-in mechanism to use a custom attribute to identify interactive elements (inputs, buttons, links)? You can enable it like this:

Capybara.test_id = "test_id"

Now if you have an element, say, button, with an HTML attribute test_id="my_button" you can click it via the following code:

click_on "my_button"

# or
find_button("my_button").click

Unfortunately, that doesn’t work with plain #find. Let’s see how we can fix this.

Capybara has an extensible Selectors API, which allows to define named rules to locate elements. Let’s add a custom :test_id selector:

Capybara.add_selector(:test_id) do
  xpath do |locator|
    XPath.descendant[XPath.attr(Capybara.test_id) == locator]
  end
end

It uses XML Path DSL, which we don’t have time to cover today.

Now we can use this selector with #find to locate elements:

find(:test_id, "my_button").click

Why do we care about #find? Because that’s the method used by SitePrism to find elements and sections.
Here is how we use test IDs in page objects:

class MoviesPage < BasePage
  element :totals, :test_id, "totals"

  # NOTE: Here we use :table selector; it is aware of `test_id` as well
  section :table, TableSection, :table, "main-table"
  section :topbar, :test_id, "topbar"
end

Using dedicated test identifiers makes browser tests more resilient to frontend changes. But at what cost? Should we force developers to refactor tests and frontend code right away? That could be a decent amount of work. I recommend using the following rule of thumb:

• If it’s possible to locate elements via existing CSS properties—use them; otherwise—use test_id.
• Whenever a test fails due to CSS changes, add test_id and refactor the affected page objects.

We can call this approach a gradual test_id adoption.

Let’s rewind a bit and talk about find(:icon, "check") from the previous section. You’ve probably already guessed that under the hood it uses a custom selector. Here is the code:

Capybara.add_selector(:icon) do
  xpath do |locator|
    XPath.descendant(:i).where(XPath.attr(:class).contains_word("fa-#{locator}"))
  end
end

Bonus: non-leaking test IDs with Slim

And here goes the last code snippet for now. For Slim users only, sorry.

Imagine the following situation: you started using test_id everywhere, and now your HTML contains some test-specific information—yet another leaky abstraction. Probably, your users should not be aware of test-specific markup. Who knows what evil things can they do with this information?

How can we leave test IDs in only when we run tests, and strip them when we serve HTML in production? Ideally, without rewriting our view templates? I’ve been thinking about an elegant way of solving this problem and suddenly had an idea.

Slim has shortcuts support (which I used forever ago in my frontend framework). The initial idea was to define a shortcut for test_id which does nothing in non-test environments. It turned out that it’s not possible to add a non-operational shortcut to Slim. But I didn’t give up! I took a look at the internal implementation of Temple—an abstract framework for compiling templates used by Slim—and found that it has filters support. Hence, we can write a filter to strip down test_id attributes from the resulting HTML!

The final implementation looks like this:

# config/initializers/slim.rb
shortcut = Slim::Parser.options[:shortcut]
shortcut['~'] = { attr: "test_id" }

return if Rails.env.test?

class SlimTestIdFilter < Temple::HTML::Filter
  def on_html_attrs(*attrs)
    [:html, :attrs, *attrs.map { |attr| compile(attr) }.compact]
  end

  def on_html_attr(name, value)
    return if name == "test_id"

    super
  end
end

Slim::Engine.after Slim::Controls, SlimTestIdFilter

As you can see, we use ~ (“tilde” and “test” both start with a “T”, so that’s sort of mnemonics) as a shortcut to define test_id attributes. Here is an example usage:

.container~main
  p Hello from Slim!

Feel free to use this code in your project!

To prism or not to prism?

In conclusion, I’d like to say that the approach described in this article wouldn’t fit everyone. It works great for large applications with a consistent UI: admin interfaces, CRMs, Basecamp-like projects, etc. That is, when we can maximize the reusability of pages and sections. Even if it doesn’t seem that you can share a lot of code, it still makes sense to introduce Page Objects to hide the frontend complexity (cryptic selectors and JS manipulation). Otherwise adding yet another layer of abstraction would only bring maintenance overhead.

For example, I decided not to use SitePrism in the AnyCable Demo application I used as the demonstration for the first article in this series: the UI is simple and easy to test without any additional hacks.

Whenever you think “I’m tired of writing system tests and keeping them up-to-date”, remember that you have heard of some “prisms or whatever 🤔”, take another look at this post and consider the described techniques to reduce your pain.

If you want to set up rock-solid development practices in your engineering team—regardless of the technical stack—feel free to drop us a line. Helping companies to improve their engineering culture is one of the favorite things we do!