# Active Storage meets GraphQL: Direct Uploads

> Integrating Active Storage direct uploads into GraphQL Ruby project

- Date: 2019-04-02T00:00:00.000Z
- Authors: Vladimir Dementyev
- Categories: 
- URL: https://evilmartians.com/chronicles/active-storage-meets-graphql-direct-uploads

---

## Happy code for happy people

Let me start with a strong statement: **I am a happy person**.

This happiness is multi-dimensional, with some dimensions bringing more value than others (and I'm not talking about the most valuable ones today, sorry).

For example, I'm happy at work because I finally got an opportunity to build a project from scratch using such "cutting-edge" (if we can say so about 15-years old framework 😉) technologies, as [**Rails 6**](https://edgeguides.rubyonrails.org/6_0_release_notes.html).

Believe it or not, I've been working on Rails production projects for 5 years, and haven't even touched Rails 5, only Rails 4!

"Legacy Rails applications" was a kind of my specialization (by the way, that's the topic of my upcoming [RailsConf](https://railsconf.com/program/sessions#session-832) session).

These dark times have ended [this January](https://weblog.rubyonrails.org/2019/1/18/Rails-6-0-Action-Mailbox-Action-Text-Multiple-DBs-Parallel-Testing/): I ran `gem install rails --prerelease && rails new ***`.

(Actually, it was `rails new *** -d postgresql --skip-action-mailbox --skip-action-text --skip-action-cable --skip-sprockets --skip-spring --skip-test --skip-bundle`.)

The project I'm working on is not a 100% new codebase; it has a lot of code initially written for Rails 4.

And it has [GraphQL](https://graphql.org) API as the main entry point for clients (web and mobile applications).

As a part of porting the old codebase into the new app, we migrated from [CarrierWave](https://github.com/carrierwaveuploader/carrierwave) to [Active Storage](https://edgeguides.rubyonrails.org/active_storage_overview.html). The experience was smooth. And though Active Storage has [some](https://github.com/rails/rails/pull/34935) [missing](https://github.com/rails/rails/pull/35390) [parts](https://github.com/rails/rails/pull/35290), it has its advantages and a _Rails-way_ simplicity.

*Read also Evil Martians' article*: https://evilmartians.com/chronicles/rails-5-2-active-storage-and-beyond

**NOTE:** If you're new to Active Storage, check out the post I wrote a year ago with [my colleague](https://twitter.com/progapandist): ["Rails 5.2: Active Storage and beyond"](https://evilmartians.com/chronicles/rails-5-2-active-storage-and-beyond).

So, it's time to move to the most notable advantage of Active Storage: [direct uploads](https://edgeguides.rubyonrails.org/active_storage_overview.html#direct-uploads) implemented out-of-the-box.

## Life before Active Storage

First, let me tell you how we dealt with file uploads in Rails 4. Neither GraphQL specification nor [`graphql`](http://graphql-ruby.org/) Ruby gem specifies a way to cook file uploads properly.

There is an [open source specification](https://github.com/jaydenseric/graphql-multipart-request-spec), which has implementations in different languages, including [Ruby](https://github.com/jetruby/apollo_upload_server-ruby). It "describes" the [`Upload`](https://github.com/jetruby/apollo_upload_server-ruby/blob/2.0.0.beta.3/lib/apollo_upload_server/upload.rb) scalar type, does some [Rack middleware](https://github.com/jetruby/apollo_upload_server-ruby/blob/2.0.0.beta.3/lib/apollo_upload_server/middleware.rb) magic to pass uploaded files as variables and _kinda works_ transparently.

Sounds like a "plug-n-play." In theory. In practice, it transformed into "plug-n-play-n-fail-n-fix-n-fail-n-fix":
- Buggy client implementations (especially for React Native)
- Side-effects due to a non-strict [`Upload` type](https://github.com/jetruby/apollo_upload_server-ruby/blob/2.0.0.beta.3/lib/apollo_upload_server/upload.rb) (which doesn't care about the actual object _type_)
- Apollo dependency (yes, we said "Good-bye!" to Apollo in the new version; but that's another story).

No surprises (and [no alarms](https://en.wikipedia.org/wiki/No_Surprises) 😉), we decided to get rid of this hack and use a good old REST for uploading files.

And here comes Active Storage with direct uploads.

## Directing uploads 🎥

What is "direct upload" by the way?

This term is usually used in conjunction with cloud storage services (e.g., Amazon S3) and means the following: **instead of uploading a file using the API server, the client uploads it directly to the cloud storage using credentials generated by the API server**.

*Image: Direct uploads diagram*

Good news—Active Storage provides a server-side API to handle direct uploads and a frontend JS client [out-of-the-box](https://edgeguides.rubyonrails.org/active_storage_overview.html#direct-uploads).

Another good news—this API is abstract, works with any [_service_](https://edgeguides.rubyonrails.org/active_storage_overview.html#setup) supported by Active Storage (i.e., filesystem, S3, GCloud, Azure). And that's great: you can use filesystem locally and S3 in production without all that `if`-s and `else-`s.

Good news rarely comes without bad news, though. And the bad news is that Active Storage (and Rails in general) does not know anything about GraphQL and relies on its own REST API for retrieving direct upload credentials.

What do we need to make it all happen in GraphQL?

First of all, an ability to get direct upload credentials using GraphQL API (via [_mutation_](https://graphql.org/learn/queries/#mutations)).

Secondly, it would be great to re-use as much JavaScript code from a framework as possible to avoid re-inventing the wheel.

### `createDirectUpload` mutation...

*Image: mutation preview in GraphiQL*

Unfortunately, Rails doesn't have any documentation for the server-side direct uploads implementation.

All we have is the [source code](https://github.com/rails/rails/blob/v6.0.0.beta3/activestorage/app/controllers/active_storage/direct_uploads_controller.rb) for `DirectUploadsController`:

```ruby
def create
  blob = ActiveStorage::Blob.create_before_direct_upload!(blob_args)
  render json: direct_upload_json(blob)
end

private

def blob_args
  params.require(:blob).permit(:filename, :byte_size, :checksum, :content_type, :metadata).to_h.symbolize_keys
end

def direct_upload_json(blob)
  blob.as_json(root: false, methods: :signed_id).merge(
    direct_upload: {
      url: blob.service_url_for_direct_upload,
      headers: blob.service_headers_for_direct_upload
    }
  )
end
```

Take a look at the `checksum` parameter: this is one of mane _hidden gems_ of Active Storage—a built-in file contents verification.

When a client requests a direct upload, it can specify the checksum of the file ([MD5](https://en.wikipedia.org/wiki/MD5) hash encoded as [base64](https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=3&cad=rja&uact=8&ved=2ahUKEwia_MD3mqvhAhUEVN8KHR22DBkQFjACegQIAhAB&url=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FBase64&usg=AOvVaw1QZr6n6e41rkPczxpoNpjT)), and the service (e.g., Active Storage itself or S3) will use this checksum later to verify the uploaded file contents.

Let's come back to GraphQL.

GraphQL mutations are pretty much like Rails controllers, so transforming the above code into a mutation is a straightforward:

```ruby
class CreateDirectUpload < GraphQL::Schema::Mutation
  class CreateDirectUploadInput < GraphQL::Schema::InputObject
    description "File information required to prepare a direct upload"

    argument :filename, String, "Original file name", required: true
    argument :byte_size, Int, "File size (bytes)", required: true
    argument :checksum, String, "MD5 file checksum as base64", required: true
    argument :content_type, String, "File content type", required: true
  end

  argument :input, CreateDirectUploadInput, required: true

  class DirectUpload < GraphQL::Schema::Object
    description "Represents direct upload credentials"

    field :url, String, "Upload URL", null: false
    field :headers, String,
          "HTTP request headers (JSON-encoded)",
          null: false
    field :blob_id, ID, "Created blob record ID", null: false
    field :signed_blob_id, ID,
          "Created blob record signed ID",
          null: false
  end

  field :direct_upload, DirectUpload, null: false

  def resolve(input:)
    blob = ActiveStorage::Blob.create_before_direct_upload!(input.to_h)

    {
      direct_upload: {
        url: blob.service_url_for_direct_upload,
        # NOTE: we pass headers as JSON since they have no schema
        headers: blob.service_headers_for_direct_upload.to_json,
        blob_id: blob.id,
        signed_blob_id: blob.signed_id
      }
    }
  end
end

# add this mutation to your Mutation type
field :create_direct_upload, mutation: CreateDirectUpload
```

Now, to retrieve a direct upload payload from the server, your GraphQL client must perform the following request:

```js
mutation {
 createDirectUpload(input: {
   filename: "dev.to", # file name
   contentType: "image/jpeg", # file content type
   checksum: "Z3Yzc2Q5iA5eXIgeTJn", # checksum
   byteSize: 2019 # size in bytes
 }) {
   directUpload {
     signedBlobId
   }
 }
}
```

### ...and some JavaScript

**Disclaimer:** the JS implementation below is just a sketch and hasn't been tested in reality (since in my project we don't use any of Rails' JS code). All I checked for is that it compiles.

To upload a file, a client must perform the following steps:
- Obtain the file metadata (filename, size, content type and checksum)
- Request direct upload credentials and a blob ID via API—`createDirectUpload` mutation
- Upload the file using the credentials (no GraphQL involved, HTTP PUT request).

For step 1 and 3 we can re-use some of the code from the JS library that comes along with Rails (don't forget to add `"@rails/activestorage"` to your `package.json`).

Let's write a `getFileMetadata` function:

```js
import { FileChecksum } from "@rails/activestorage/src/file_checksum";

function calculateChecksum(file) {
  return new Promise((resolve, reject) => {
    FileChecksum.create(file, (error, checksum) => {
      if (error) {
        reject(error);
        return;
      }

      resolve(checksum);
    });
  });
}

export const getFileMetadata = (file) => {
  return new Promise((resolve) => {
    calculateChecksum(file).then((checksum) => {
      resolve({
        checksum,
        filename: file.name,
        content_type: file.type,
        byte_size: file.size
      });
    });
  });
};
```

[`FileChecksum`](https://github.com/rails/rails/blob/v6.0.0.beta3/activestorage/app/javascript/activestorage/file_checksum.js) class is responsible for calculating the required checksum and is used by Active Storage in [`DirectUpload`](https://github.com/rails/rails/blob/v6.0.0.beta3/activestorage/app/javascript/activestorage/direct_upload.js#L16) class.

Now you can use this function to build your GraphQL query payload:

```js
// pseudo code
getFileMetadata(file).then((input) => {
  return performQuery(
    CREATE_DIRECT_UPLOAD_QUERY,
    variables: { input }
  );
});
```

Now it is time to write a function to upload files directly to the storage service!

```js
import { BlobUpload } from "@rails/activestorage/src/blob_upload";

export const directUpload = (url, headers, file) => {
  const upload = new BlobUpload({ file, directUploadData: { url, headers } });
  return new Promise((resolve, reject) => {
    upload.create(error => {
      if (error) {
        reject(error);
      } else {
        resolve();
      }
    })
  });
};
```

Our complete client side code example would be the following:

```js
getFileMetadata(file).then((input) => {
  return performQuery(
    CREATE_DIRECT_UPLOAD_QUERY,
    variables: { input }
  ).then(({ directUpload: { url, headers, signedBlobId }) => {
    return directUpload(url, JSON.parse(headers), file).then(() => {
      // do smth with signedBlobId—our file has been uploaded!
    });
  });
});
```

Looks like we did it! Hope that helps you to build your awesome new Rails + GraphQL project)

> For a more realistic example, [check out this snippet](https://gist.github.com/Saionaro/7ee0e2c02749e2729dc429c9e9bfa7f3) from our React Native application.

## In Conclusion, or what to do with `signedBlobId`

Let me provide just a quick example of how we use signed blob IDs in the app—the `attachProfileAvatar` mutation:

```ruby
class AttachProfileAvatar < GraphQL::Schema::Mutation
  description <<~DESC
    Update the current user's avatar
    (by attaching a blob via signed ID)
  DESC

  argument :blob_id, String,
           "Signed blob ID generated via `createDirectUpload` mutation",
           required: true

  field :user, Types::User, null: true

  def resolve(blob_id:)
    # Active Storage retrieves the blob data from DB
    # using a signed_id and associates the blob with the attachment (avatar)
    current_user.avatar.attach(blob_id)
    { user: current_user }
  end
end
```

That's it!

*Read also Evil Martians' article*: https://evilmartians.com/chronicles/active-storage-meets-graphql-pt-2-exposing-attachment-urls

---

This article was written by [Evil Martians](https://evilmartians.com), a product development consultancy trusted by teams from early-stage startups to public companies. If you're working through a hard problem, [we can help](https://evilmartians.com/contact-us)