Active Storage meets GraphQL pt. 2: exposing attachment URLs
Topics
Share this post on
![Vladimir Dementyev]()
Vladimir Dementyev
Principal Backend Engineer
Translations
Other parts:
- Active Storage meets GraphQL: Direct Uploads
- Active Storage meets GraphQL pt. 2: exposing attachment URLs
In the previous post, I shared some tips on adding Active Storage’s direct uploads to Rails+GraphQL applications.
So, now we know how to upload files. What’s next? Let’s move to the next step: exposing attachments URLs via GraphQL API.
Seems like an easy task, right? Not exactly.
These are the challenges we faced when doing it in our own application:
- Dealing with N+1 queries
- Making it possible for clients to request a specific image variant.
N+1 problem: batch loading to the rescue
Let’s first try to add the avatarUrl field to our User type in a naïve way:
module Types
class User < GraphQL::Schema::Object
field :id, ID, null: false
field :name, String, null: false
field :avatar_url, String, null: true
def avatar_url
# That's an official way for generating
# Active Storage blobs URLs outside of controllers 😕
Rails.application.routes.url_helpers
.rails_blob_url(user.avatar)
end
end
endAssume that we have an endpoint which returns all the users, e.g., { users { name avatarUrl } }. If you run this query in development and take a look at the Rails server logs in your console, you will see something like this:
D, [2019-04-15T22:46:45.916467 #2500] DEBUG -- : User Load (0.9ms) SELECT users".* FROM "users"
D, [2019-04-15T22:46:45.919362 #2500] DEBUG -- : ActiveStorage::Attachment Load (0.9ms) SELECT "active_storage_attachments".* FROM "active_storage_attachments" WHERE "active_storage_attachments"."record_type" = $1 AND "active_storage_attachments"."name" = $2 AND "active_storage_attachments"."record_id" = $3 [["record_type", "User"], ["name", "avatar"], ["record_id", 12]]
D, [2019-04-15T22:46:45.922420 #2500] DEBUG -- : ActiveStorage::Blob Load (1.0ms) SELECT "active_storage_blobs".* FROM "active_storage_blobs" WHERE "active_storage_blobs"."id" = $1 [["id", 9]]
D, [2019-04-15T22:46:45.919362 #2500] DEBUG -- : ActiveStorage::Attachment Load (0.9ms) SELECT "active_storage_attachments".* FROM "active_storage_attachments" WHERE "active_storage_attachments"."record_type" = $1 AND "active_storage_attachments"."name" = $2 AND "active_storage_attachments"."record_id" = $3 [["record_type", "User"], ["name", "avatar"], ["record_id", 13]]
D, [2019-04-15T22:46:45.922420 #2500] DEBUG -- : ActiveStorage::Blob Load (1.0ms) SELECT "active_storage_blobs".* FROM "active_storage_blobs" WHERE "active_storage_blobs"."id" = $1 [["id", 10]]
D, [2019-04-15T22:46:45.919362 #2500] DEBUG -- : ActiveStorage::Attachment Load (0.9ms) SELECT "active_storage_attachments".* FROM "active_storage_attachments" WHERE "active_storage_attachments"."record_type" = $1 AND "active_storage_attachments"."name" = $2 AND "active_storage_attachments"."record_id" = $3 [["record_type", "User"], ["name", "avatar"], ["record_id", 14]]
D, [2019-04-15T22:46:45.922420 #2500] DEBUG -- : ActiveStorage::Blob Load (1.0ms) SELECT "active_storage_blobs".* FROM "active_storage_blobs" WHERE "active_storage_blobs"."id" = $1 [["id", 15]]For each user we load an ActiveStorage::Attachment and an ActiveStorage::Blob record: 2*N + 1 records (where N is the number of users).
We already discussed this problem in the “Rails 5.2: Active Storage and beyond” post, so, I’m not going to repeat the technical details here.
TL;DR For classic Rails apps we have a built-in scope for preloading attachments (e.g. User.with_attached_avatar) or can generate the scope ourselves knowing the way Active Storage names internal associations.
GraphQL makes preloading data a little bit trickier—we don’t know beforehand which data is needed by the client and cannot just add with_attached_<smth> to every Active Record collection (‘cause that would add an additional overhead when we don’t need this data).
That’s why classic preloading approaches (includes, eager_load, etc.) are not very helpful for building GraphQL APIs. Instead, most of the applications use the batch loading technique.
One of the ways to do that in a Ruby app is to add the graphql-batch gem by Shopify. It provides a core API for writing batch loaders with a Promise-like interface.
Although no batch loaders included into the gem by default, there is an association_loader example which we can use for our task (more precisely, we use this enhanced version which supports scopes and nested associations).
Let’s use it to solve our N+1 issue:
def avatar_url
AssociationLoader.for(
object.class,
# We should provide the same arguments as
# the `preload` or `includes` call when do a classic preloading
avatar_attachment: :blob
).load(object).then do |avatar|
next if avatar.nil?
Rails.application.routes.url_helpers.rails_blob_url(avatar)
end
endNOTE: the then method we use above is not a #yield_self alias, it’s an API provided by the promise.rb gem.
The code looks a little bit overloaded, but it works and makes only 3 queries independently on the number of users. Keep on reading to see how we can transform this into a human-friendly API.
Dealing with variants
We want to leverage the power of GraphQL and allow clients to specify the desired image variants (e.g., thumbs, covers, etc.):
From the code perspective we want to do the following:
user.avatar.variant(:thumb) # == user.avatar.variant(resize_to_fill: [64, 64])Unfortunately, Active Storage doesn’t have a concept of variants (predefined, named transformations) yet. That will likely be included in Rails 6.x (where x > 0) when this PR (or its variation) gets merged.
We decided not to wait and implement this functionality ourselves: this small patch by @bibendi adds the ability to define named variants in a YAML file:
# config/transformations.yml
thumb:
convert: jpg
resize_to_fill: [64, 64]
medium:
convert: jpg
resize_to_fill: [200, 200]Since we have the same transformation settings for all the attachments in the app, this global configuration works for us well.
Now we need to integrate this functionality into our API.
First, we add an enum type to our schema representing a particular variant from the transformations.yml:
class ImageVariant < GraphQL::Schema::Enum
description <<~DESC
Image variant generated with libvips via the image_processing gem.
Read more about options here https://github.com/janko/image_processing/blob/master/doc/vips.md#methods
DESC
ActiveStorage.transformations.each do |key, options|
value key.to_s, options.map { |k, v| "#{k}: #{v}" }.join("\n"), value: key
end
endThanks to Ruby’s metaprogramming nature, we can define our type dynamically using the configuration object—our transformations.yml and the ImageVariant enum will always be in sync!
Finally, let’s update our field definition to support variants:
module Types
class User < GraphQL::Schema::Object
field :avatar_url, String, null: true do
argument :variant, ImageVariant, required: false
end
def avatar_url(variant: nil)
AssociationLoader.for(
object.class,
avatar_attachment: :blob
).load(object).then do |avatar|
next if avatar.nil?
avatar = avatar.variant(variant) if variant
Rails.application.routes.url_helpers.url_for(avatar)
end
end
end
endBonus: adding a field extension
Adding this amount of code every time we want to add an attachment url field to a type doesn’t seem to be an elegant solution, does it?
While looking for a better option, I found a Field Extensions API for graphql-ruby. “Looks like exactly what I was looking for!”, I thought.
Let me first show you the final field definition:
field :avatar_url, String, null: true, extensions: [ImageUrlField]That’s it! No more argument-s and loaders. Adding the extension makes everything work the way we want!
And here is the annotated code for the extension:
class ImageUrlField < GraphQL::Schema::FieldExtension
attr_reader :attachment_assoc
def apply
# Here we try to define the attachment name:
# - it could be set explicitly via extension options
# - or we imply that is the same as the field name w/o "_url"
# suffix (e.g., "avatar_url" => "avatar")
attachment = options&.[](:attachment) ||
field.original_name.to_s.sub(/_url$/, "")
# that's the name of the Active Record association
@attachment_assoc = "#{attachment}_attachment"
# Defining an argument for the field
field.argument(
:variant,
ImageVariant,
required: false
)
end
# This method resolves (as it states) the field itself
# (it's the same as defining a method within a type)
def resolve(object:, **_rest)
AssociationLoader.for(
object.class,
# that's where we use our association name
attachment_assoc => :blob
)
end
# This method is called if the result of the `resolve`
# is a lazy value (e.g., a Promise—like in our case)
def after_resolve(value:, arguments:, **_rest)
return if value.nil?
variant = arguments.fetch(:variant, :medium)
value = value.variant(variant) if variant
Rails.application.routes.url_helpers.url_for(value)
end
endHappy graphQLing and active storing!
