Active Storage meets GraphQL pt. 2: exposing attachment URLs

April 23, 2019

Topics

Vladimir Dementyev
Principal Backend Engineer

Other parts:

Active Storage meets GraphQL: Direct Uploads
Active Storage meets GraphQL pt. 2: exposing attachment URLs

In the previous post, I shared some tips on adding Active Storage’s direct uploads to Rails+GraphQL applications.

So, now we know how to upload files. What’s next? Let’s move to the next step: exposing attachments URLs via GraphQL API.

Seems like an easy task, right? Not exactly.

These are the challenges we faced when doing it in our own application:

Dealing with N+1 queries
Making it possible for clients to request a specific image variant.

N+1 problem: batch loading to the rescue

Let’s first try to add the avatarUrl field to our User type in a naïve way:

module Types
  class User < GraphQL::Schema::Object
    field :id, ID, null: false
    field :name, String, null: false
    field :avatar_url, String, null: true

    def avatar_url
    # That's an official way for generating
    # Active Storage blobs URLs outside of controllers 😕
      Rails.application.routes.url_helpers
           .rails_blob_url(user.avatar)
    end
  end
end

Assume that we have an endpoint which returns all the users, e.g., { users { name avatarUrl } }. If you run this query in development and take a look at the Rails server logs in your console, you will see something like this:

D, [2019-04-15T22:46:45.916467 #2500] DEBUG -- : User Load (0.9ms) SELECT users".* FROM "users"
D, [2019-04-15T22:46:45.919362 #2500] DEBUG -- : ActiveStorage::Attachment Load (0.9ms) SELECT "active_storage_attachments".* FROM "active_storage_attachments" WHERE "active_storage_attachments"."record_type" = $1 AND "active_storage_attachments"."name" = $2 AND "active_storage_attachments"."record_id" = $3 [["record_type", "User"], ["name", "avatar"], ["record_id", 12]]
D, [2019-04-15T22:46:45.922420 #2500] DEBUG -- : ActiveStorage::Blob Load (1.0ms) SELECT "active_storage_blobs".* FROM "active_storage_blobs" WHERE "active_storage_blobs"."id" = $1 [["id", 9]]
D, [2019-04-15T22:46:45.919362 #2500] DEBUG -- : ActiveStorage::Attachment Load (0.9ms) SELECT "active_storage_attachments".* FROM "active_storage_attachments" WHERE "active_storage_attachments"."record_type" = $1 AND "active_storage_attachments"."name" = $2 AND "active_storage_attachments"."record_id" = $3 [["record_type", "User"], ["name", "avatar"], ["record_id", 13]]
D, [2019-04-15T22:46:45.922420 #2500] DEBUG -- : ActiveStorage::Blob Load (1.0ms) SELECT "active_storage_blobs".* FROM "active_storage_blobs" WHERE "active_storage_blobs"."id" = $1 [["id", 10]]
D, [2019-04-15T22:46:45.919362 #2500] DEBUG -- : ActiveStorage::Attachment Load (0.9ms) SELECT "active_storage_attachments".* FROM "active_storage_attachments" WHERE "active_storage_attachments"."record_type" = $1 AND "active_storage_attachments"."name" = $2 AND "active_storage_attachments"."record_id" = $3 [["record_type", "User"], ["name", "avatar"], ["record_id", 14]]
D, [2019-04-15T22:46:45.922420 #2500] DEBUG -- : ActiveStorage::Blob Load (1.0ms) SELECT "active_storage_blobs".* FROM "active_storage_blobs" WHERE "active_storage_blobs"."id" = $1 [["id", 15]]

For each user we load an ActiveStorage::Attachment and an ActiveStorage::Blob record: 2*N + 1 records (where N is the number of users).

We already discussed this problem in the “Rails 5.2: Active Storage and beyond” post, so, I’m not going to repeat the technical details here.

TL;DR For classic Rails apps we have a built-in scope for preloading attachments (e.g. User.with_attached_avatar) or can generate the scope ourselves knowing the way Active Storage names internal associations.

GraphQL makes preloading data a little bit trickier—we don’t know beforehand which data is needed by the client and cannot just add with_attached_<smth> to every Active Record collection (‘cause that would add an additional overhead when we don’t need this data).

That’s why classic preloading approaches (includes, eager_load, etc.) are not very helpful for building GraphQL APIs. Instead, most of the applications use the batch loading technique.

One of the ways to do that in a Ruby app is to add the graphql-batch gem by Shopify. It provides a core API for writing batch loaders with a Promise-like interface.

Although no batch loaders included into the gem by default, there is an association_loader example which we can use for our task (more precisely, we use this enhanced version which supports scopes and nested associations).

Let’s use it to solve our N+1 issue:

def avatar_url
  AssociationLoader.for(
    object.class,
  # We should provide the same arguments as
  # the `preload` or `includes` call when do a classic preloading
    avatar_attachment: :blob
  ).load(object).then do |avatar|
    next if avatar.nil?

    Rails.application.routes.url_helpers.rails_blob_url(avatar)
  end
end

NOTE: the then method we use above is not a #yield_self alias, it’s an API provided by the promise.rb gem.

The code looks a little bit overloaded, but it works and makes only 3 queries independently on the number of users. Keep on reading to see how we can transform this into a human-friendly API.

Dealing with variants

We want to leverage the power of GraphQL and allow clients to specify the desired image variants (e.g., thumbs, covers, etc.):

From the code perspective we want to do the following:

user.avatar.variant(:thumb) # == user.avatar.variant(resize_to_fill: [64, 64])

Unfortunately, Active Storage doesn’t have a concept of variants (predefined, named transformations) yet. That will likely be included in Rails 6.x (where x > 0) when this PR (or its variation) gets merged.

We decided not to wait and implement this functionality ourselves: this small patch by @bibendi adds the ability to define named variants in a YAML file:

# config/transformations.yml

thumb:
 convert: jpg
 resize_to_fill: [64, 64]

medium:
 convert: jpg
 resize_to_fill: [200, 200]

Since we have the same transformation settings for all the attachments in the app, this global configuration works for us well.

Now we need to integrate this functionality into our API.

First, we add an enum type to our schema representing a particular variant from the transformations.yml:

class ImageVariant < GraphQL::Schema::Enum
  description <<~DESC
    Image variant generated with libvips via the image_processing gem.
    Read more about options here https://github.com/janko/image_processing/blob/master/doc/vips.md#methods
  DESC

  ActiveStorage.transformations.each do |key, options|
    value key.to_s, options.map { |k, v| "#{k}: #{v}" }.join("\n"), value: key
  end
end

Thanks to Ruby’s metaprogramming nature, we can define our type dynamically using the configuration object—our transformations.yml and the ImageVariant enum will always be in sync!

Finally, let’s update our field definition to support variants:

module Types
  class User < GraphQL::Schema::Object
    field :avatar_url, String, null: true do
      argument :variant, ImageVariant, required: false
    end

    def avatar_url(variant: nil)
      AssociationLoader.for(
        object.class,
        avatar_attachment: :blob
      ).load(object).then do |avatar|
        next if avatar.nil?

        avatar = avatar.variant(variant) if variant
        Rails.application.routes.url_helpers.url_for(avatar)
      end
    end
  end
end

Bonus: adding a field extension

Adding this amount of code every time we want to add an attachment url field to a type doesn’t seem to be an elegant solution, does it?

While looking for a better option, I found a Field Extensions API for graphql-ruby. “Looks like exactly what I was looking for!”, I thought.

Let me first show you the final field definition:

field :avatar_url, String, null: true, extensions: [ImageUrlField]

That’s it! No more argument-s and loaders. Adding the extension makes everything work the way we want!

And here is the annotated code for the extension:

class ImageUrlField < GraphQL::Schema::FieldExtension
  attr_reader :attachment_assoc

  def apply
  # Here we try to define the attachment name:
  # - it could be set explicitly via extension options
  # - or we imply that is the same as the field name w/o "_url"
  # suffix (e.g., "avatar_url" => "avatar")
    attachment = options&.[](:attachment) ||
                 field.original_name.to_s.sub(/_url$/, "")

  # that's the name of the Active Record association
    @attachment_assoc = "#{attachment}_attachment"

  # Defining an argument for the field
    field.argument(
      :variant,
      ImageVariant,
      required: false
    )
  end

 # This method resolves (as it states) the field itself
 # (it's the same as defining a method within a type)
  def resolve(object:, **_rest)
    AssociationLoader.for(
      object.class,
      # that's where we use our association name
      attachment_assoc => :blob
    )
  end

 # This method is called if the result of the `resolve`
 # is a lazy value (e.g., a Promise—like in our case)
  def after_resolve(value:, arguments:, **_rest)
    return if value.nil?

    variant = arguments.fetch(:variant, :medium)
    value = value.variant(variant) if variant

    Rails.application.routes.url_helpers.url_for(value)
  end
end

Happy graphQLing and active storing!