The SDK Imperative: why developer tools live or die by their SDKs

Some backstory: We were searching for an open-source connectivity solution for AnyCable Pro running on Fly.io to enable synchronization between nodes in a cluster. Our journey led us to NATS, a highly performant connectivity technology for distributed systems.

NATS was a joy to work with, so we couldn’t help but offer our consulting services.

It turned out that the Ruby SDK of NATS wasn’t on par performance-wise, and we ended up speeding it up by more than three times for high-load cases–a significant achievement for NATS!

This was a major enhancement to the client so I appreciate the practical solution that was implemented to boost the client’s performance and scalability.

---

This experience made us realize a critical aspect of developer tools that is often overlooked: the quality and performance of their SDKs can make or break adoption and usability.

Irina Nazarova CEO at Evil Martians

Schedule call

Why do SDKs matter so much?

Before we dive into this case in detail, let’s elaborate a little bit on the topic at hand and why it is important: SDKs are the primary interface between developers and a developer tool’s capabilities, and an inefficient SDK can undermine the performance benefits of even the most optimized product.

Further, poor SDKs lead to frustration, increased development time, and the potential that users will just abandon the tool.

An SDK represents a product to the entire ecosystem. Ideally, an SDK should feel native to its users, adhering to language-specific notations. This is why companies often hire consultants with deep language expertise to handle the heavy lifting; an approach which can lead to amazing developer experience and manageable future maintenance.

The NATS Ruby SDK case study

As previously mentioned, the NATS Ruby client was not performance-optimized, and it was quickly becoming a bottleneck.

While one might be tempted to blame Ruby for this slowness, the issue was actually not with Ruby itself but rather, the problem related to how it was being used within the SDK. Luckily, Ruby has a few efficiency-oriented cards up its sleeve!

But what was the problem? For every subscription, the NATS Ruby client was creating a separate thread to handle incoming messages. This is a simple and reliable solution. However, Ruby threads, being an underlying OS primitive, are relatively expensive to maintain: when there are tens of thousands mostly idle threads, the OS scheduler is busy switching between them leaving no CPU time for executing the application code!

We considered a few solutions: switching to a thread pool, fibers, and async Ruby. We ended up going with thread pools to optimize messaging performance, resulting in a more than 3x improvement for high-load cases, which are typical thing for NATS.

Choosing the right solution is not an easy task: one needs to balance their urge to rewrite everything from the scratch with their available time, taking into account:

• Fibers, while seemingly the best choice, as they are the most modern and lightweight primitives, were still too new at the moment and lacked adoption in applications and frameworks. It was just too risky to adopt them now.
• The Async library is neat, and hides the complexity of underlying fibers, providing a nice API. However, frankly speaking, it requires rewriting applications to use it. That’s way too high of a price for an SDK. That said, it is a good idea to make another version of NATS Ruby SDK specifically for it.
• Thread pools, while being the least “fancy” solution, are the most reliable one: we continue to use the same underlying primitives, just doing this wisely and responsibly. There are little changes required to the SDK itself. And, most importantly, no changes required in client apps or any notable backward compatibility issues expected.

So it’s no surprise that we chose thread pools to fix the issue, prioritizing backward compatibility over hype. But did you notice that word “notable” in the previous paragraph? Actually, it turns out that there are some possible changes in behavior of client app subscription handlers that could be unexpected; you can find more specific technical details in this talk: Threads, callbacks, and execution context in Ruby. (However, while this presents a curious detail for discussion, the actual chances to encounter this issue in real apps are negligible.)

(And, because we have the pleasure of working in the world of open source, you can see exactly what we did: check out our pull request for handling subscription messages in a thread pool.)

So, what can we learn from this? The biggest takeaway for teams building for engineers is that we need to treat SDKs as first-class citizens in developer tools. That means following these practices:

• Optimizing SDKs to leverage the strengths of each programming language
• Actively seeking and incorporating community feedback on usability and performance
• Regularly updating SDKs to stay in sync with core product developments and language ecosystem changes

And, we have to address the ripple effect! Improving an SDK doesn’t just benefit individual developers–it can elevate the entire ecosystem around a tool:

• Better SDKs lead to wider tool adoption and community growth
• Efficient SDKs encourage more developers to contribute to the core project
• Well-designed SDKs facilitate the creation of complementary tools and integrations

Closing thoughts

Our experience with the NATS Ruby SDK reinforced a crucial lesson: in the world of developer tools, SDKs are not just accessories—SDKs are essential components that can determine a tool’s success or failure.

As creators and contributors, we need to recognize that our tools are only as strong as the SDKs that developers use to interact with them.

Further, prioritizing SDK development and optimization will not only improve individual developer experiences, it will also contribute to the overall health and growth of the open-source ecosystem!