March 7, 2024
Remember that feeling of moving from your tiny dorm room to your first real apartment? That’s kind of what our SafeBoda backend has gone through over the years. We started off 8 years ago as a monolithic PHP application, which was great at first, but as we grew, things got a little cramped (think too many roommates and not enough space for your dreams).
Now, while PHP is a solid language (it used to be the king of the website castle!), tech moves fast these days. So, we had to adapt. But here’s the thing: great engineers are like culinary wizards — they can build amazing things with almost any set of tools (or programming languages). Sure, I wouldn’t recommend our team programming in Fortran and Cobol in 2024, but with the right talent, any language can work.
As our team grew, the monolithic monster became a maintenance nightmare. Imagine ten people trying to cook in a tiny kitchen — chaos! We needed to be agile and push out clean code multiple times a week. And on top of that ensure there were no bugs and avoid deploying code that crashed the system. That’s when we embraced the microservices architecture, the talk of the tech town (and every conference ever).
Here’s the thing, though: talking about microservices is easy, but actually doing it? That’s a whole other story. Lots of CTOs say they are moving to micro services but most startups don’t make the transition. This is because you need a rockstar DevOps team and the right mindset. Breaking an app into microservices is like splitting an atom — exciting, but comes with a whole lot of new challenges.
For example, one of our microserves is a driver management service that we wanted to keep separate from our financial transaction service and ride dispatch engine. One of the main things we wanted to do for our drivers is to not let them take cash trips when they are in debt to us. Why you may ask, is because as they take more cash trips they get further into debt with us owing us commission and they eventually can’t get out of debt. But keeping these three services in sync across thousands of drivers, in real-time, is super challenging. It involves message queues, separate databases , and a system that can heal itself if one of the services hiccups.
As part of this microservices makeover, we decided to upgrade our backend language to Elixir. It’s the language of choice for big names like WhatsApp and Discord, and we were impressed by its benefits. Here are some of the key features that originally lured us to Elixir.
As SafeBoda’s engineering team ventured into Elixir, the language’s potential was undeniable. Its scalability and ability to handle our real-time traffic volumes lived up to the hype. However, unforeseen challenges emerged during implementation.
The most significant hurdle, as previously mentioned, was the critical dependence on the quality and experience of our engineering team, regardless of the programming language chosen. Imagine if you were forming a rock band. You would rather have Slash from Guns N Roses as your guitarist using a $100 guitar vs. having a 12 year old beginner using a $6,000 guitar. A fancy high performance tool still won’t get the job done without being in the hands of the correct people.
Our predicament was further compounded by the limited availability of Elixir developers. Despite the presence of highly skilled individuals, the overall talent pool, especially in East Africa where Safeboda resides, was concerningly small. This scarcity often translated into significantly higher compensation demands to secure their expertise.
Furthermore, integrating junior engineers proved problematic. The transition to Elixir’s functional programming paradigm, a concept not readily covered in basic courses, presented a significant learning curve. This resulted in a potential team divide between seasoned functional programmers and those accustomed to traditional object-oriented approaches. While some engineers adapted seamlessly, many others struggled despite extensive training, ultimately restricting our talent pool for specific projects.
These limitations, coupled with the strategic objective of fostering a robust engineering talent pool within East Africa, compelled us to explore a more sustainable solution. Consequently, the decision was made to transition our microservices to Node.js, specifically leveraging the NestJS framework.
This choice, unsurprisingly, sparked debate within the tech community. Many questioned the rationale behind Node.js, particularly with compelling alternatives like Go’s speed, Java’s enterprise-grade solutions via the JVM, and Rust’s burgeoning potential. As the CTO, navigating such complex choices is a fundamental responsibility. While numerous languages presented themselves as viable options, I ultimately selected Node.js based on the following key considerations.
By carefully considering these factors, we concluded that NestJS offered the perfect blend of a large talent pool, familiarity with existing skillsets, a mature ecosystem, and easy integration with our existing tools. This combination creates a solid foundation for efficient development, team collaboration, and long-term maintainability of our backend infrastructure.
The past two years have witnessed SafeBoda’s backend infrastructure undergo a significant transformation, with many services migrating to Node.js. As the CTO, this journey has provided valuable insights, not only regarding the technical merits of Node.js but also in reaffirming the crucial role of our skilled development team.
While Elixir remains a cornerstone for our high-concurrency and scalability requirements, Node.js has proven to be an excellent choice for a substantial portion of our microservices. And we continue to move more services from Elixir to Node.js every day. This experience has underscored a fundamental truth: the foundation of a robust development environment lies not solely in the technology stack, but in fostering a team of strong developers. Working with a team that can tackle challenges with agility is critial to our success.
Furthermore, adopting a unified and widely accessible language like Node.js simplifies the onboarding process for new developers, fostering a more streamlined and efficient development experience. This, coupled with the advantages of TypeScript, which provides enhanced type safety, empowers our team to deliver scalable solutions while maintaining code quality.
The success of this migration is evident in our ability to manage a solution exceeding a million app downloads, and processing hundreds of thousands of rides monthly, with minimal scalability concerns. This journey has illustrated the potential of Node.js for many of our major microservices. But most importantly the most important part of our success is the importance of cultivating a skilled development team as a key driver of long-term success. This journey has been a super exciting one for me with many adventures left to come for our team.
March 7, 2024