Reporting non-nullable violations in graphql-ruby properly

GraphQL encourages you to decouple your frontend from your backend. You can think more about your types’ “correctness” and less about particular view layout. To achieve that GraphQL provides strong type system. GraphQL doesn’t allow to return String instead of Integer and also doesn’t allow return nulls where API declares that type is non-nullable.

But sometimes your data will accidentally contain empty values where even you don’t expect them. Some external APIs will return incorrect or missing data, even if its contract disallows that. Sometimes it will be your own bug. But in any case, it is your responsibility to track for such failures and correct them.

If you use Ruby for developing GraphQL API, the chances are high that you use graphql-ruby. It is standard de-facto in Ruby because it is a very mature and feature-rich gem with a whole ecosystem of plugins. But “out of the box” there it doesn’t inform you about violations of these “non-nullable type” rules and just silently returns a partial response with error messages to the client.

But we want to know about non-null violations because it is our responsibility to develop bullet-proof applications. And according to the graphql-ruby’s documentation about type errors, there is a special hook to catch such errors. Let’s use it!

We’re using Honeybadger, so we’ll report this offense there:

class YourAppSchema < GraphQL::Schema
  def self.type_error(exception, query_context)
    if exception.is_a?(::GraphQL::InvalidNullError)
      Honeybadger.notify exception, context: { query: query_context.query.query_string }
    end
    super
  end
end

So far, so good. Soon we’ll able to see our first error in error tracker UI.

But wait, what’s it? There are errors about non-null violations in multiple different fields mixed in one error field. Aren’t they different bugs? What’s wrong?

It turns out that by default Honeybadger groups errors together by exception class, component (controller and action), and backtrace fragment. Please note that the exception message doesn’t count. In the case of GraphQL, it is always a single controller and action, so exceptions for absolutely different queries are getting grouped together. Let’s fix that!

According to the same documentation page, we need to calculate unique fingerprint string for every different error. So, what we need to include to this fingerprint for non-null violation?

• Exception class? Yes, we need to distinguish GraphQL::InvalidNullError from other types of errors
• Controller and action? No, they’re always the same. And even if an error will occur during triggering subscription, will it make any difference?
• Backtrace? Hmm… I’m not sure. What do you think?
• Type and field names, of course! We need to separate errors in different fields.

Let’s use an exception class name, GraphQL type name, and field name:

class ApplicationSchema < GraphQL::Schema
  class << self
    def type_error(exception, query_context)
      fingerprint = honeybadger_fingerprint(exception)
      Honeybadger.notify exception, fingerprint: fingerprint, context: {
        query:   query_context.query.query_string,
        user_id: query_context[:user]&.id
      }
      super
    end

    private

    def honeybadger_fingerprint(error)
      case error
      when GraphQL::InvalidNullError
        "#{error.class}:#{error.parent_type.name}.#{error.field.name}"
      end
    end
  end
end

And that’s it: from now on exceptions for different fields lives in different Honeybadger errors.

Thank you for your attention. Let’s build reliable APIs!