Hire Martians
Hire Martians
1-
Feb
2
Meet us at FOSDEM in Brussels, Belgium!

Climbing Steep hills, or adopting Ruby 3 types with RBS

Topics

With Ruby 3.0 just around the corner, let’s take a look at one of the highlights of the upcoming release: Ruby Type Signatures. Yes, types are coming to our favorite dynamic language! Let’s see how to take advantage of them by adding types to a real-world open source project and looking at the finer points of the process.

It is not the first time I cover Ruby types: about a year ago, I had the first taste of Sorbet and shared my experience in the very same Martian blog. At the end of my Sorbet post, I promised to give another Ruby type checker a try: Steep. So, here I am, paying my debts!

RBS in a nutshell

RBS is an entirely different beast from Sorbet: first of all, it’s an official endeavor by the Ruby core team. Secondly, it takes a whole different approach to annotating programs: ideally, you can leave your .rb files entirely unchanged. The official Readme states that:

RBS is a language to describe the structure of Ruby programs.

The “structure” includes class and method signatures, type definitions, etc. Since it’s a stand-alone language, not Ruby, separate .rbs files are used to store typings.

Let’s jump right into an example:

# martian.rb
class Martian < Alien
  def initialize(name, evil: false)
    super(name)
    @evil = evil
  end

  def evil?
    @evil
  end
end

# martian.rbs
class Alien
  attr_reader name : String

  def initialize : (name: String) -> void
end

class Martian < Alien
  @evil : bool

  def initialize : (name: String, ?evil: bool) -> void
  def evil? : () -> bool
end

The signature looks pretty similar to the class definition itself, except that we have types specified for arguments, methods, and instance variables. So far, looks pretty Ruby-ish. However, RBS has some entities which are missing in Ruby, for example, interfaces. We’re gonna see some examples later.

RBS itself doesn’t provide any functionality to perform type checking; it’s just a language, remember? That’s where Steep comes into a stage.

In the rest of this article, I will describe the process of adding RBS and Steep to Rubanok—a magical, Russian-Pinnochio-inspired Ruby DSL for transforming params in HTTP controllers. In my Sorbet post, I used the same library as an example to give you a sense of continuity. If you want to learn more about Rubanok itself, check out my “Carve your controllers like Papa Carlo” blog post.

Scaffolding types

It could be hard to figure out how to start adding types to an existing project. Luckily, RBS provides a way to scaffold our way in. It comes with a CLI tool (rbs) which has a number of commands, but we’re interested only in the prototype:

$ rbs prototype -h
Usage: rbs prototype [generator...] [args...]

Generate prototype of RBS files.
Supported generators are rb, rbi, runtime.

Examples:

  $ rbs prototype rb foo.rb
  $ rbs prototype rbi foo.rbi
  $ rbs prototype runtime String

The description is pretty self-explanatory; let’s try it:

$ rbs prototype rb lib/**/*.rb

# Rubanok provides a DSL ... (all the comments from the source file)
module Rubanok
  attr_accessor ignore_empty_values: untyped
  attr_accessor fail_when_no_matches: untyped
end

module Rubanok
  class Rule
    # :nodoc:
    UNDEFINED: untyped

    attr_reader fields: untyped
    attr_reader activate_on: untyped
    attr_reader activate_always: untyped
    attr_reader ignore_empty_values: untyped
    attr_reader filter_with: untyped

    def initialize: (untyped fields, ?activate_on: untyped activate_on, ?activate_always: bool activate_always, ?ignore_empty_values: untyped ignore_empty_values, ?filter_with: untyped? filter_with) -> untyped
    def project: (untyped params) -> untyped
    def applicable?: (untyped params) -> (::TrueClass | untyped)
    def to_method_name: () -> untyped

    private

    def build_method_name: () -> ::String
    def fetch_value: (untyped params, untyped field) -> untyped
    def empty?: (untyped val) -> (::FalseClass | untyped)
  end
end

# <truncated>

The first option (prototype rb) generates a signature for all the entities specified in the file (or files) you pass using static analysis (more precisely, via parsing the source code and analyzing ASTs).

This command streams all the discovered typings to standard output. To save the output, we can use redirection:

rbs prototype rb lib/**/*.rb > sig/rubanok.rbs

I’d prefer to mirror signature files to source files (i.e., have multiple files). We can achieve this with some knowledge of Unix:

find lib -name \*.rb -print | cut -sd / -f 2- | xargs -I{} bash -c 'export file={}; export target=sig/$file; mkdir -p ${target%/*}; rbs prototype rb lib/$file > sig/${file/rb/rbs}'

In my opinion, it would be much better if we had the above functionality by default (or maybe that’s a feature—keeping all the signatures in the same file 🤔).

Also, copying comments from source files to signatures makes the latter less readable (especially if there are many comments, like in my case). Of course, we can add a bit more Unix magic to fix this…

Let’s try the runtime mode:

$ RUBYOPT="-Ilib" rbs prototype runtime -r rubanok Rubanok::Rule

class Rubanok::Rule
  public

  def activate_always: () -> untyped
  def activate_on: () -> untyped
  def applicable?: (untyped params) -> untyped
  def fields: () -> untyped
  def filter_with: () -> untyped
  def ignore_empty_values: () -> untyped
  def project: (untyped params) -> untyped
  def to_method_name: () -> untyped

  private

  def build_method_name: () -> untyped
  def empty?: (untyped val) -> untyped
  def fetch_value: (untyped params, untyped field) -> untyped
  def initialize: (untyped fields, ?activate_on: untyped, ?activate_always: untyped, ?ignore_empty_values: untyped, ?filter_with: untyped) -> untyped
end

In a runtime mode, RBS uses Ruby’s introspection APIs (Class.methods, etc.) to generate the specified class or module signature.

Let’s compare signatures for the Rubanok::Rule class generated with rb and runtime modes:

  • First, runtime generator does not recognize attr_reader (for instance, activate_on and activate_always).
  • Second, runtime generator sorts methods alphabetically while static generator preserves the original layout.
  • Finally, the first approach guesses a few types, while the latter leaves everything untyped.

So, why one may find runtime generator useful? I guess there is only one reason for that: dynamically generated methods. Like, for example, in Active Record.

Thus, both modes have their advantages and disadvantages, and using them both would provide a better signature coverage. Unfortunately, there is no good way to diff/merge RBS files yet; you have to do that manually. Another manual work is to replace untyped with the actual typing information.

But let’s not get our hands dirty just yet. There is one more player in this game—Type Profiler, yet another experimental tool from the Ruby core team.

Type Profiler infers a program type signatures dynamically during execution. It spies on all the loaded classes and methods and collects the information about which types have been used as inputs and outputs, analyzes this data, and produces RBS definitions. Under the hood, it uses a custom Ruby interpreter (so, the code is not actually executed). You can find more in the official docs.

The main difference between TypeProf and RBS is that we need to create a sample script to be used as a profiling entry-point.

Let’s write one:

# sig/rubanok_type_profile.rb
require "rubanok"

processor = Class.new(Rubanok::Processor) do
  map :q do |q:|
    raw
  end

  match :sort_by, :sort, activate_on: :sort_by do
    having "status", "asc" do
      raw
    end

    default do |sort_by:, sort: "asc"|
      raw
    end
  end
end

processor.project({q: "search", sort_by: "name"})
processor.call([], {q: "search", sort_by: "name"})

Now, let’s run typeprof command:

$ typeprof -Ilib sig/rubanok_type_profile.rb --exclude-dir lib/rubanok/rails --exclude-dir lib/rubanok/rspec.rb

# Classes
module Rubanok
  VERSION : String

  class Rule
    UNDEFINED : Object
    @method_name : String
    attr_reader fields : untyped
    attr_reader activate_on : Array[untyped]
    attr_reader activate_always : false
    attr_reader ignore_empty_values : untyped
    attr_reader filter_with : nil
    def initialize : (untyped, ?activate_on: untyped, ?activate_always: false, ?ignore_empty_values: untyped, ?filter_with: nil) -> nil
    def project : (untyped) -> untyped
    def applicable? : (untyped) -> bool
    def to_method_name : -> String
    private
    def build_method_name : -> String
    def fetch_value : (untyped, untyped) -> Object?
    def empty? : (nil) -> false
  end

  # ...
end

Nice, now we have some of the types defined (though most of them are still untyped), and TypeProf respects the visibility of our methods and even instance variables (something we haven’t seen before). The order of methods stayed the same as in the original file—that’s good!

Unfortunately, despite being a runtime analyzer, TypeProf is not so good at metaprogramming support. For example, the methods defined using iteration won’t be recognized:

# a.rb
class A
  %w[a b].each.with_index { |str, i| define_method(str) { i } }
end

p A.new.a + A.new.b
 
$ typeprof a.rb

# Classes
class A
end

That would work with rbs prototype runtime though 😉

So, even if you produce an executable that provides 100% coverage of your APIs but uses metaprogramming, TypeProf is still not enough to build a complete types scaffold for your program.

To sum up, all three ways to generate initial signatures have their pros and cons, but combining their results could give a very good starting point in adding types to existing code. Hopefully, we’ll be able to automate this in the future.

In Rubanok’s case, I did the following:

  • Generated initial signatures using rbs prototype rb.
  • Ran typeprof and used its output to add missing instance variables and update some signatures.
  • Finally, ran rbs prototype runtime for main classes.

I also discovered a bug when RBS failed to correctly attribute an attr_accessor defined within a singleton class (e.g., inside a class << self block). Luckily, it was fixed and merged super quick, so my main module signatures are now correct:

 module Rubanok
-  attr_accessor ignore_empty_values: untyped
-  attr_accessor fail_when_no_matches: untyped
+  def self.fail_when_no_matches: () -> untyped
+  def self.fail_when_no_matches=: (untyped) -> untyped
+  def self.ignore_empty_values: () -> untyped
+  def self.ignore_empty_values=: (untyped) -> untyped
 end

Introducing Steep

So far, we’ve only discussed how to write and generate type signatures. That would be useless if we don’t add a type checker to our dev stack.

As of today, the only type checker that supports RBS is Steep—developed, unsurprisingly by Soutaro Matsumoto, the Ruby core committer who was in charge of RBS.

steep init

Let’s add the steep gem to our dependencies and generate a configuration file:

steep init

That would generate a default Steepfile with some configuration. For Rubanok, I updated it like this:

# Steepfile
target :lib do
  # Load signatures from sig/ folder
  signature "sig"
  # Check only files from lib/ folder
  check "lib"

  # We don't want to type check Rails/RSpec related code
  # (because we don't have RBS files for it)
  ignore "lib/rubanok/rails/*.rb"
  ignore "lib/rubanok/railtie.rb"
  ignore "lib/rubanok/rspec.rb"

  # We use Set standard library; its signatures
  # come with RBS, but we need to load them explicitly
  library "set"
end

steep stats

Before drowning in a sea of types, let’s think of how we can measure our signatures’ efficiency. We can use steep stats to see how good (or bad?) our types coverage is:

$ bundle exec steep stats --log-level=fatal

Target,File,Status,Typed calls,Untyped calls,All calls,Typed %
lib,lib/rubanok/dsl/mapping.rb,success,7,2,11,63.64
lib,lib/rubanok/dsl/matching.rb,success,26,18,50,52.00
lib,lib/rubanok/processor.rb,success,34,8,49,69.39
lib,lib/rubanok/rule.rb,success,24,12,36,66.67
lib,lib/rubanok/version.rb,success,0,0,0,0
lib,lib/rubanok.rb,success,8,4,12,66.67

This command, surprisingly, outputs CSV 😯. Let’s add some Unix magic and make the output more readable:

$ bundle exec steep stats --log-level=fatal | awk -F',' '{ printf "%-28s %-9s %-12s %-14s %-10s\n", $2, $3, $4, $5, $7 }'
File                         Status    Typed calls  Untyped calls  Typed %
lib/rubanok/dsl/mapping.rb   success   7            2              63.64
lib/rubanok/dsl/matching.rb  success   26           18             52.00
lib/rubanok/processor.rb     success   34           8              69.39
lib/rubanok/rule.rb          success   24           12             66.67
lib/rubanok/version.rb       success   0            0              0
lib/rubanok.rb               success   8            4              66.67

UPDATE (2021-03-30): Steep 0.43 added the table formatting support out-of-the-box. No need to deal with Unix, simply run bundle exec steep stats --log-level=fatal --format=table.

Ideally, we would like to have everything typed. So, I opened my .rbs files and started replacing untyped with the actual types one by one.

It took me about ten minutes to get rid of untyped definitions (most of them). I’m not going to describe this process in detail; it was pretty straightforward except for the one thing I’d like to pay attention to.

Let’s recall what Rubanok is. It provides a DSL to define data (usually, user input) transformers of a form (input, params) -> input. A typical use case is to customize an Active Record relation depending on request parameters:

class PagySearchyProcess < Rubanok::Processor
  map :page, :per_page, activate_always: true do |page: 1, per_page: 20|
   # raw is a user input
   raw.page(page).per(per_page)
  end

  map :q do |q:|
    raw.search(q)
  end
end

PagySearchyProcessor.call(Post.all, {q: "rbs"})
#=> Post.search("rbs").page(1).per(20)

PagySearchyProcessor.call(Post.all, {q: "rbs", page: 2})
#=> Post.search("rbs").page(2).per(20)

Rubanok deals with two external types: input (which could be anything) and params (a Hash with String or Symbol keys). Also, we have a notion of field internally: a params key used to activate a particular transformation. A lot of Rubanok’s methods use these three entities. To avoid duplication, I decided to use a type aliases feature of RBS:

module Rubanok
  # Transformation parameters
  type params = Hash[Symbol | String, untyped]
  type field = Symbol
  # Transformation target (we assume that input and output types are the same)
  type input = Object?

  class Processor
    def self.call: (params) -> input
                 | (input, params) -> input
    def self.fields_set: () -> Set[field]
    def self.project: (params) -> params

    def initialize: (input) -> void
    def call: (params) -> input
  end

  class Rule
    attr_reader fields: Array[field]

    def project: (params) -> params
    def applicable?: (params) -> bool
  end

  # ...
end

That allowed me to avoid duplication and indicate that they are not just Hashes, Strings, or whatever passing around, but params, fields, and inputs.

Now, let’s check our signatures!

Fighting with the signatures, or make steep check happy

It’s very unlikely that we wrote 100% correct signatures right away. I got ~30 errors:

$ bundle exec steep check --log-level=fatal

lib/rubanok/dsl/mapping.rb:24:8: MethodArityMismatch: method=map (def map(*fields, **options, &block))
lib/rubanok/dsl/mapping.rb:38:10: NoMethodError: type=(::Object & ::Rubanok::DSL::Mapping::ClassMethods), method=define_method (define_method(rule.to_method_name, &block))
lib/rubanok/dsl/mapping.rb:40:10: NoMethodError: type=(::Object & ::Rubanok::DSL::Mapping::ClassMethods), method=add_rule (add_rule rule)
lib/rubanok/dsl/matching.rb:25:10: MethodArityMismatch: method=initialize (def initialize(id, fields, values = [], **options, &block))
lib/rubanok/dsl/matching.rb:26:26: UnexpectedSplat: type= (**options)
lib/rubanok/dsl/matching.rb:29:12: IncompatibleAssignment:  ...
lib/rubanok/dsl/matching.rb:30:32: NoMethodError: type=::Array[untyped], method=keys (@values.keys)
lib/rubanok/dsl/matching.rb:42:8: MethodArityMismatch: method=initialize (def initialize(*, **))
lib/rubanok/dsl/matching.rb:70:8: MethodArityMismatch: method=match (def match(*fields, **options, &block))
lib/rubanok/dsl/matching.rb:71:17: IncompatibleArguments: ...
lib/rubanok/dsl/matching.rb:73:10: BlockTypeMismatch: ...
lib/rubanok/dsl/matching.rb:75:10: NoMethodError: type=(::Object & ::Rubanok::DSL::Matching::ClassMethods), method=define_method (define_method(rule.to_method_name) do |params = {}|)
lib/rubanok/dsl/matching.rb:83:12: NoMethodError: type=(::Object & ::Rubanok::DSL::Matching::ClassMethods), method=define_method (define_method(clause.to_method_name, &clause.block))
lib/rubanok/dsl/matching.rb:86:10: NoMethodError: type=(::Object & ::Rubanok::DSL::Matching::ClassMethods), method=add_rule (add_rule rule)
lib/rubanok/dsl/matching.rb:96:15: NoMethodError: type=(::Object & ::Rubanok::DSL::Matching), method=raw (raw)
lib/rubanok/processor.rb:36:6: MethodArityMismatch: method=call (def call(*args))
lib/rubanok/processor.rb:56:13: NoMethodError: type=(::Class | nil), method=<= (superclass <= Processor)
lib/rubanok/processor.rb:57:12: NoMethodError: type=(::Class | nil), method=rules (superclass.rules)
lib/rubanok/processor.rb:67:13: NoMethodError: type=(::Class | nil), method=<= (superclass <= Processor)
lib/rubanok/processor.rb:68:12: NoMethodError: type=(::Class | nil), method=fields_set (superclass.fields_set)
lib/rubanok/processor.rb:78:21: ArgumentTypeMismatch: receiver=::Hash[::Symbol, untyped], expected=::Array[::Symbol], actual=::Set[::Rubanok::field] (*fields_set)
lib/rubanok/processor.rb:116:6: NoMethodError: type=::Rubanok::Processor, method=input= (self.input =)
lib/rubanok/processor.rb:134:6: NoMethodError: type=::Rubanok::Processor, method=input= (self.input = prepared_input)
lib/rubanok/rule.rb:11:6: IncompatibleAssignment: ...
lib/rubanok/rule.rb:20:8: UnexpectedJumpValue (next acc)
lib/rubanok/rule.rb:48:12: NoMethodError: type=(::Method | nil), method=call (filter_with.call(val))
lib/rubanok/rule.rb:57:8: MethodArityMismatch: method=empty? (def empty?)
lib/rubanok/rule.rb:63:8: MethodArityMismatch: method=empty? (def empty?)
lib/rubanok/rule.rb:69:4: MethodArityMismatch: method=empty? (def empty?(val))

Let’s take a closer look at these errors and try to fix them.

1. Refinements always break things.

Let’s start with the last three reported errors:

lib/rubanok/rule.rb:57:8: MethodArityMismatch: method=empty? (def empty?)
lib/rubanok/rule.rb:63:8: MethodArityMismatch: method=empty? (def empty?)
lib/rubanok/rule.rb:69:4: MethodArityMismatch: method=empty? (def empty?(val))

Why Steep detected three #empty? methods in the Rule class? It turned out that it considers an anonymous refinement body to be a part of the class body:

using(Module.new do
  refine NilClass do
    def empty?
      true
    end
  end

  refine Object do
    def empty?
      false
    end
  end
end)

def empty?(val)
  return false unless ignore_empty_values

  val.empty?
end

I submitted an issue and moved refinements to the top of the file to fix those errors.

2. Superclasses don’t cry 😢

Another interesting issue relates to the superclass method usage:

lib/rubanok/processor.rb:56:13: NoMethodError: type=(::Class | nil), method=<= (superclass <= Processor)
lib/rubanok/processor.rb:57:12: NoMethodError: type=(::Class | nil), method=rules (superclass.rules)
lib/rubanok/processor.rb:67:13: NoMethodError: type=(::Class | nil), method=<= (superclass <= Processor)

The corresponding source code:

@rules =
  if superclass <= Processor
    superclass.rules.dup
  else
    []
  end

It’s a very common pattern to inherit class properties. Why doesn’t it work? First, the superclass signature says the result is either Class or nil (though it could be nil only for the BaseObject class, as far as I know). Thus, we cannot use <= right away (because it’s not defined on NilClass).

Even if we unwrap superclass, the problem with .rules would still be there—Steep’s flow sensitivity analysis currently doesn’t recognize the <= operator. So, I decided to hack the system and explicitly define the .superclass signature for the Processor class:

# processor.rbs
class Processor
  def self.superclass: () -> singleton(Processor)
  # ...
end

That way, my code stays the same; only the types suffer 😈.

3. Explicit over implicit: Handling splats.

So far, we’ve seen pretty much the same problems as I had with Sorbet. Let’s take a look at something new.

Consider this code snippet:

def project(params)
  params = params.transform_keys(&:to_sym)
  # params is a Hash, fields_set is a Set
  params.slice(*fields_set)
end

It produces the following type error:

lib/rubanok/processor.rb:78:21: ArgumentTypeMismatch: receiver=::Hash[::Symbol, untyped], expected=::Array[::Symbol], actual=::Set[::Rubanok::field]

The Hash#slice method expects an Array, but we pass a Set. However, we also use a splat (*) operator, which implicitly tries to convert an object to an array—seems legit, right? Unfortunately, Steep is not so smart yet: we have to add an explicit #to_a call.

4. Explicit over implicit, pt. 2: Forwarding arguments.

I used the following pattern in a few places:

def match(*fields, **options, &block)
  rule = Rule.new(fields, **options)

  # ...
end

This DSL method accepts some options as keyword arguments and then passes them to the Rule class initializer. The possible options are strictly defined and enforced in the Rule#initialize, but we would like to avoid declaring them explicitly just to forward down. Unfortunately, that’s only possible if we declare **options as untyped—that would make signatures kinda useless.

So, we have to become more explicit once again:

-        def map(*fields, **options, &block)
-          filter = options[:filter_with]
-          rule = Rule.new(fields, **options)

+        def map(*fields, activate_on: fields, activate_always: false, ignore_empty_values: Rubanok.ignore_empty_values, filter_with: nil, &block)
+          filter = filter_with
+          rule = Rule.new(fields, activate_on: activate_on, activate_always: activate_always, ignore_empty_values: ignore_empty_values, filter_with: filter_with)

# and more...

I guess it’s time to add my Ruby Next compiler to the mix and use the shorthand Hash notation 🙂

5. Variadic arguments: annotations to the rescue!

In the recent Rubanok release, I added an ability to skip input for transformations and only use params as the only #call method argument. That led to the following code:

def call(*args)
  input, params =
  if args.size == 1
    [nil, args.first]
  else
    args
  end

  new(input).call(params)
end

As in the previous case, we needed to make our signature more explicit and specify the actual arguments instead of the *args:

# This is our signature
# (Note that we can define multiple signatures for a method)
def self.call: (input, params) -> input
             | (params) -> input

# And this is our code (first attempt)
UNDEFINED = Object.new

def call(input, params = UNDEFINED)
  input, params = nil, input if params == UNDEFINED

  raise ArgumentError, "Params could not be nil" if params.nil?

  new(input).call(params)
end

This refactoring doesn’t pass the type check:

$ bundle exec steep lib/rubanok/processor.rb

lib/rubanok/processor.rb:43:24: ArgumentTypeMismatch: receiver=::Rubanok::Processor, expected=::Rubanok::params, actual=(::Rubanok::input | ::Rubanok::params | ::Object) (params)

So, according to Steep, param could be pretty match anything. We need to help Steep make the right decision. I couldn’t find a way to do that via RBS, so my last resort was to use annotations.

Yes, even though RBS itself is designed not to pollute your source code, Steep allows you to do that. And, in some cases, that is the necessary evil.

I came up with the following:

def call(input, params = UNDEFINED)
  input, params = nil, input if params == UNDEFINED

  raise ArgumentError, "Params could not be nil" if params.nil?

  # @type var params: untyped
  new(input).call(params)
end

We declare params as untyped to silence the error. The #call method signature guarantees that the params variable satisfies the params type requirements, so we should be safe here.

6. Dealing with metaprogramming: Interfaces.

Since Rubanok provides a DSL, it uses metaprogramming heavily.
For example, we use #define_method to dynamically generate transformation methods:

def map(*fields, activate_on: fields, activate_always: false, ignore_empty_values: Rubanok.ignore_empty_values, filter_with: nil, &block)
  # ...
  rule = Rule.new(fields, activate_on: activate_on, activate_always: activate_always, ignore_empty_values: ignore_empty_values, filter_with: filter_with)

  define_method(rule.to_method_name, &block)

  add_rule rule
end

And here’s the error we see when running steep check:

lib/rubanok/dsl/mapping.rb:38:10: NoMethodError: type=(::Object & ::Rubanok::DSL::Mapping::ClassMethods), method=define_method (define_method(rule.to_method_name, &block))
lib/rubanok/dsl/mapping.rb:40:10: NoMethodError: type=(::Object & ::Module & ::Rubanok::DSL::Mapping::ClassMethods), method=add_rule (add_rule rule)

Hmm, looks like our type checker doesn’t know that we’re calling the .map method in the context of the Processor class (we call Processor.extend DSL::Mapping).

RBS has a concept of a self type for module: a self type adds requirements to the classes/modules, which include, prepend, or extend said module. For example, we can state that we only allow using Mapping::ClassMethods to extend modules (and not objects, for example):

# Module here is a self type
module ClassMethods : Module
  # ...
end

That fixes NoMethodError for #define_method, but we still have it for #add_rule—this is a Processor self method. How can we add this restriction using module self types? It’s not allowed to use singleton(SomeClass) as a self type; only classes and interfaces are allowed.

Yes, RBS has interfaces! Let’s give them a try!

We only use the #add_rule method in the modules, so we can define an interface as follows:

interface _RulesAdding
  def add_rule: (Rule rule) -> void
end

# Then we can use this interface in the Processor class itself
class Processor
  extend _RulesAdding

  # ...
end

# And in our modules
module Mapping
  module ClassMethods : Module, _RulesAdding
    # ...
  end
end

7. Making Steep happy

I added a few more changes to the signatures and the source code to finally make a steep check pass. The journey was a bit longer than I expected, but in the end, I’m pretty happy with the result—I will continue using RBS and Steep.

Here is the final stats for Rubanok:

File                         Status    Typed calls  Untyped calls  Typed %
lib/rubanok/dsl/mapping.rb   success   11           0              100.00
lib/rubanok/dsl/matching.rb  success   54           2              94.74
lib/rubanok/processor.rb     success   52           2              96.30
lib/rubanok/rule.rb          success   31           2              93.94
lib/rubanok/version.rb       success   0            0              0
lib/rubanok.rb               success   12           0              100.00

Runtime type checking with RBS

Although RBS doesn’t provide static type checking capabilities, it comes with runtime testing utils. By loading a specific file (rbs/test/setup), you can ask RBS to watch the execution and check that method calls inputs and outputs satisfy signatures.

Under the hood, TracePoint API is used along with the alias method chain trick to hijack observed methods. Thus, it’s meant for use in tests, not in production.

Let’s try to run our RSpec tests with runtime checking enabled:

$ RBS_TEST_TARGET='Rubanok::*' RUBYOPT='-rrbs/test/setup' bundle exec rspec --fail-fast

I, [2020-12-07T21:07:57.221200 #285]  INFO -- : Setting up hooks for ::Rubanok
I, [2020-12-07T21:07:57.221302 #285]  INFO -- rbs: Installing runtime type checker in Rubanok...
...

Failures:

  1) Rails controllers integration PostsApiController#planish implicit rubanok with matching
     Failure/Error: prepare! unless prepared?

     RBS::Test::Tester::TypeError:
       TypeError: [Rubanok::Processor#prepared?] ReturnTypeError: expected `bool` but returns `nil`

Oh, we forgot to initialize the @prepared instance variable with the boolean value! Nice!

I found a couple of more issues by using rbs/test/setup, including the one I wasn’t able to resolve:

Failure/Error: super(fields, activate_on: activate_on, activate_always: activate_always)

RBS::Test::Tester::TypeError:
  TypeError: [Rubanok::Rule#initialize] UnexpectedBlockError: unexpected block is given for `(::Array[::Rubanok::field] fields, ?filter_with: ::Method?, ?ignore_empty_values: bool, ?activate_always: bool, ?activate_on: ::Rubanok::field | ::Array[::Rubanok::field]) -> void`

And here is the reason:

class Clause < Rubanok::Rule
  def initialize(id, fields, values, **options, &block)
    # The block is passed to super implicitly,
    # but is not acceptable by Rule#initialize
    super(fields, **options)
  end
end

I tried to use &nil to disable block propagation, but that broke steep check 😞. I submitted an issue and excluded Rule#initialize from the runtime checking for now using a special comment in the .rbs file:

# rule.rbs
class Rule
  # ...
  %a{rbs:test:skip} def initialize: (
    Array[field] fields,
    ?activate_on: field | Array[field],
    ?activate_always: bool,
    ?ignore_empty_values: bool,
    ?filter_with: Method?
  ) -> void
end

Bonus: VS Code integration

Steep comes with an official VS Code plugin, which just works:

Steep in VS Code

There is also a language server available (steep langserver), so other tools could integrate with Steep.

Bonus 2: Steep meets Rake

I usually run bundle exec rake pretty often during development to make sure that everything is correct. The default task usually includes RuboCop and tests.

Let’s add Steep to the party:

# Rakefile

# other tasks

task :steep do
  # Steep doesn't provide Rake integration yet,
  # but can do that ourselves
  require "steep"
  require "steep/cli"

  Steep::CLI.new(argv: ["check"], stdout: $stdout, stderr: $stderr, stdin: $stdin).run
end

namespace :steep do
  # Let's add a user-friendly shortcut
  task :stats do
    exec %q(bundle exec steep stats --log-level=fatal | awk -F',' '{ printf "%-28s %-9s %-12s %-14s %-10s\n", $2, $3, $4, $5, $7 }')
  end
end

# Run steep before everything else to fail-fast
task default: %w[steep rubocop rubocop:md spec]

Bonus 4: Type Checking meets GitHub Actions

As the final step, I configure GitHub Actions to run both static and runtime type checks:

# lint.yml
jobs:
  steep:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - uses: ruby/setup-ruby@v1
      with:
        ruby-version: 2.7
    - name: Run Steep check
      run: |
        gem install steep
        steep check

# rspec.yml
jobs:
  rspec:
    # ...
    steps:
      # ...
    - name: Run RSpec with RBS
      if: matrix.ruby == '2.7'
      run: |
        gem install rbs
        RBS_TEST_TARGET="Rubanok::*" RUBYOPT="-rrbs/test/setup" bundle exec rspec --force-color
    - name: Run RSpec without RBS
      if: matrix.ruby != '2.7'
      run: |
        bundle exec rspec --force-color

Although there are still enough rough edges, I enjoyed using RBS/Steep a bit more than “eating” Sorbet (mostly because I’m not a big fan of type annotations in the source code). I will continue adopting Ruby 3 types in my OSS projects and reporting as many issues to RBS/Steep as possible 🙂

P.S. You can find the full source code for Rubanok’s transition to Ruby types in this PR.

Join our email newsletter

Get all the new posts delivered directly to your inbox. Unsubscribe anytime.