From Ruby to Crystal? Writing and distributing a CLI tool

Let’s say you’ve been given the task of writing a cross-platform CLI tool, and you’ll be required to work with a network, file system, parsing and all that stuff. Additionally, you’re going to need unit tests and extensible architecture. What language should you choose? Well, Go, Rust, Deno or Node.js, C or C++… OK, is there anything else? Yes, there is! Well, with a few reservations…

Imagine you know Ruby well and don’t want to spend too much time learning a new language. Also, let’s say that you want to build the tool in Ruby but you want it compiled into a no-dependency binary file. So, in this case, the best choice for you is… to do it in Crystal!

The story

Coveralls is a code coverage service that allows you to control coverage in your project. You measure the coverage, send it to Coveralls, and receive a coverage status. Previously, Coveralls had integrations for numerous languages (about 30), with the ability to report the coverage results to coveralls.io.

However, it became difficult to add and deliver new features, since it’s necessary to add these features to all integrations. So, they decided to create their own coverage reporter called coverage-reporter; it supports most popular coverage formats and also introduces some new features.

That project was initially written in Crystal to make it easy for Ruby developers to contribute.

But then, due to a priority shift, it was left untouched for 2 years. However, in 2023, Coveralls asked the Evil Martians team to help with their coverage reporter. The main goals:

• Refactoring the code
• Adding support for a few coverage formats
• Setting up continuous delivery of new binaries
• Integrating the tool into existing GitHub Actions and CircleCI Orb

So, with our target set, our team of Ruby-loving Backened Engineers jumped into the glistening world of Crystal.

Magic spells

The Crystal language has some differences from Ruby, and you’ll need to learn some new keywords, method names, and concepts.

Jumping into Crystal with a purely Ruby mindset will set you off on a long trek frought with a lot of compilation errors.

If you want to use Crystal’s goodies you’ll have to read its docs, but nevertheless, let’s cover some basics.

For example, Crystal has property instead of attr_accessor, and it has Tuples, which are more performant than Arrays because they allocate in the stack and are immutable.

Let’s compare the Crystal and Ruby versions of 3 classes. One is data storage, the next is an abstract class, and the last does some work.

This is the Ruby version:

class FileReport
  def initialize(name, coverage)
    @name = name
    @coverage = coverage
  end

  def to_h
    { name: @name, coverage: @coverage }
  end
end

class BaseParser
  def self.name
    self.to_s.gsub(/(\w+)Parser/, "\\1").downcase
  end

  def matches?(_filename); raise NotImplementedError; end
  def parse(_filename); raise NotImplementedError; end
end

class MyformatParser < BaseParser
  def matches?(filename)
    File.readlines(filename).each do |line|
      line.chomp!
      next if /^\s*$/.match?(line)

      return true if /<myformat/.match?(line)

      return false
    end

    false
  rescue StandardError
    false
  end

  def parse(_filename)
    # ...
    []
  end
end

And this is the Crystal version:

class FileReport
  def initialize(@name : String, @coverage : Array(Int64?))
  end

  def to_h : Hash(Symbol, String | Array(Int64?))
    { :name => @name, :coverage => @coverage }
  end
end

abstract class BaseParser
  def self.name : String
    # Hey! Metaprogramming in Crystal is a huge topic and it's completely different from Ruby.
    {{ @type.stringify.gsub(/(\w+)Parser/, "\\1").downcase }}
  end

  abstract def matches?(filename : String) : Bool
  abstract def parse(filename : String) : Array(FileReport)
end

class MyformatParser < BaseParser
  def matches?(filename : String) : Bool
    File.each_line(filename, chomp: true) do |line|
      next if line.blank?

      return true if /<myformat/.matches?(line)

      return false
    end

    false
  rescue Exception
    false
  end

  def parse(filename : String) : Array(FileReport)
    # ...
    [] of ArrayReport
  end
end

As you can see, the code looks pretty much the same, but there are a few details to make note of: the types, syntactic sugar, as well as the presence of Exception instead of StandardError. Plenty of little differences.

The behavior of some methods might also surprise you: for example Enumerable#max will raise an exception, Enumerable::EmptyError, if the collection is empty. Enumerable#max? can be used if you don’t want that exception, and this approach, (appending ? to a method to allow nil value returns) is widely used in Crystal.

Beyond comparions with Ruby, Crystal also uses a type system that feels similar to TypeScript.

This allows you to provide multiple types for variables, parameters, and return values.

It’s quite a flexible system, that is, until you find yourself writing something like this:

json_type = Hash(String, Hash(String, Hash(String, Array(Int64?) | Hash(String, Array(Int64?) | Hash( # oh god, will this ever finish?
json_type.from_json(File.read(filename))

Phew, describing variants of Hash value types can be annoying, especially when value types are different.

However, Crystal provides aliases which make things a bit more clear:

alias Coverage = Array(Int64?)
alias Branches = Hash(String, Hash(String, Int64?))
alias LinesAndBranches = Hash(String, Array(Int64?) | Branches)
alias Timestamp = Int64
alias FileStats = Hash(String, Coverage | LinesAndBranches)
alias SimplecovFormat = Hash(String, Hash(String, FileStats | Timestamp))

...

SimplecovFormat.from_json(File.read(filename))

It’s never a bad idea to add more aliases since it’s much easier to hide complexity behide a reasonable name.

So, from my perspective, type-”overwhelmedness” is the only thing that can drive you mad in Crystal.

This is the price we pay for all the benefits static types give us (you probably know about these benefits if you’ve ever written in Go or Rust: overall performance, machine code optimizations, catching errors early, etc.).

Tell me what [coverage format] to swallow

One of the features we added to coverage-reporter was support for 6 new coverage formats (besides the existing LCOV format support). Coverage data is mostly of the line_number - covered/uncovered structure, so adding support for different formats was quite straightforward. The biggest task was to understand the structure of the format. It takes less than 50 lines of code to actually parse the file.

There are plenty of libraries to parse coverage formats written in JavaScript and Ruby, but for obvious reasons we couldn’t use them as dependencies; so, we had to write them ourselves.

Happily, since Ruby code can work for Crystal with just slight changes, I was able to use the best code writing practice in the world: Copy and Paste! This is how a parser for GCOV format was written in only half an hour (just compare it).

Sad [and red] Eyes

Successfully compiling a binary with no runtime dependencies is still a tricky thing for Crystal. There are some limitations to consider:

• On Linux, this can only be done in a special alpine-based Docker container
• On macOS, this is not supported at all
• On Windows, surprisingly, it works out of the box!

Crystal uses LLVM under the hood and has a few runtime libs that are platform-specific. So, in general, it’s possible to create no-dependency binaries for any architecture and any OS, but practically speaking, that’s a big task that would probably consume much more time than pure development in Crystal.

For the coverage reporter project, I decided to stick with static builds for Linux and Windows. For macOS, I added a Homebrew tap with bottles, so we can say that we have static builds for 3 platforms that GitHub uses as runners (which is enough for now, since coverage-reporter is mostly used in CI, and CI is mostly on Linux).

Since building binaries on Linux and Windows is rather straightforward, I’d like to describe the steps to create a Homebrew tap with a formula that uses bottles: a binary package which only needs linking to become a working executable. Adding bottles into a formula makes installation much faster since it skips the build step. GitHub provides a way to build bottles in GitHub Actions and store them as artifacts. However, since GitHub only supports x64 runners, you’ll only have bottles for x64 architecture.

Creating a Homebrew tap

Let’s say you want a Homebrew tap with a CLI tool built in Crystal. What follows is a recipe for just that.

First, initialize a Homebrew tap.

brew tap-new coverallsapp/homebrew-coveralls

cd $(brew --repository)/Library/Taps/coverallsapp/homebrew-coveralls

Next, edit GitHub Actions. This step is required since it fails to run the Github Action. Thus, without this fix, workflows will fail.

diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml
index 9e2a27e..9ef97fd 100644
--- a/.github/workflows/publish.yml
+++ b/.github/workflows/publish.yml
@@ -7,6 +7,8 @@ jobs:
   pr-pull:
     if: contains(github.event.pull_request.labels.*.name, 'pr-pull')
     runs-on: ubuntu-22.04
+    env:
+      HOMEBREW_NO_INSTALL_FROM_API: 1
     steps:
       - name: Set up Homebrew
         uses: Homebrew/actions/setup-homebrew@master
diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml
index ed26708..c11b113 100644
--- a/.github/workflows/tests.yml
+++ b/.github/workflows/tests.yml
@@ -10,6 +10,8 @@ jobs:
       matrix:
         os: [ubuntu-22.04, macos-12]
     runs-on: $
+    env:
+      HOMEBREW_NO_INSTALL_FROM_API: 1
     steps:
       - name: Set up Homebrew
         id: set-up-homebrew

Step three, commit the changes to main and create a branch for a new formula.

git commit -m 'fix: test and publish'
git push origin main
git checkout -b new-formula

Step four, add a formula.

# Formula/coveralls.rb

class Coveralls < Formula
  desc "Self-contained, universal coverage uploader for Coveralls"
  homepage "https://github.com/coverallsapp/coverage-reporter"
  url "https://github.com/coverallsapp/coverage-reporter/archive/refs/tags/v0.3.6.tar.gz"

  # run `brew fetch coveralls` to check SHA256
  sha256 "59fc991846d19556921ace19687f602d863f423ac93f2f50c5f6058b4a50391a"
  license "MIT"

  # use :build, so if there's a bottle it will be used
  depends_on "crystal" => :build

  # dependencies of Crystal
  depends_on "bdw-gc"
  depends_on "libevent"
  depends_on "libyaml"
  depends_on "pcre"

  # coverage-reporter dependencies
  depends_on "sqlite"
  uses_from_macos "libxml2"

  # installation instructions
  def install
    system "shards", "install", "--production"
    system "crystal", "build", "src/cli.cr", "-o", "./dist/coveralls", "--release", "--no-debug"
    system "strip", "./dist/coveralls"
    bin.install "./dist/coveralls"
  end

  # testing if binary is OK
  test do
    assert_match version.to_s, shell_output("#{bin}/coveralls --version").chomp
  end
end

And last, commit and push the changes. Next, go the the GitHub repo and open the PR. test.yml workflow should check if the formula is OK and prepare the bottles.

After all the checks have finished, set the pr-pull label and watch the magic unfold! You’ll see the PR closed and a new commit with your formula will have appeared; you will also notice bottles in the formula.

Writing in Crystal is fun.

It might seem similar to Ruby, since it has a very familiar syntax, but you’ll find yourself employing slightly different concepts.

But, to be frank, I will say that for writing a cross-platform tool, Crystal is probably not the perfect choice.

That said, it’s still an acceptable choice, especially if you want to write in a statically-typed language, and if you know Ruby.

The thing is, you’ll have to think about the types you pass and receive while also making sure your code is readable and simple.

I’d call Crystal a step away from Ruby, not a step forward, nor backwards, just, a step off to the side.

Building a statically-linked binary with Crystal in 2023 is possible but not user-friendly.

A final suggestion: use Go if support for many different platforms and architectures really matters for you!

Evil Martians are ready to assist all Earthlings in need! We’re poised to beam down and fix any issues, whether they be based in frontend, product design, backend, SRE, or anything else. If you’ve got a problem: don’t wait, and engage message transmission now, (that is, reach out to us, for more info)!