Designing Scalable Systems: Practical Approach
When I first joined GitHub, scaling quickly became more than a buzzword—it was a daily reality. Handling millions of requests per second isn't just an engineering feat; it's essential. Let me walk you through how I approach system design, scaling, and the real-world trade-offs we face. This isn't about theory; it's about what works in the trenches. We'll cover vertical vs. horizontal scaling, the importance of simplicity in code, and even hiring the right tech team. Because ultimately, it's the practical solutions that matter.
When I first joined GitHub, scaling wasn't just a buzzword—it was the daily grind. Handling millions of requests per second isn't merely an engineering feat; it's a necessity. So, how do I think about designing systems at scale? Let me share my approach: system design, the real trade-offs we face, and the delicate balance between business impact and technical achievements. This isn't about theory; it's about what actually works in the field. We'll dive into vertical vs. horizontal scaling, the importance of keeping code simple, and even how we recruit the right talent for a tech team. Because at the end of the day, it's the practical solutions that make a difference. Ready to dive in?
Scaling and System Design: Where to Start
When I first dove into scalable system design, my primary question was: how much should my solution scale? Sometimes, we anticipate a 10x or even 100x scaling factor for future-proofing. But be careful, it's easy to get lost in assumptions and design for non-existent needs. Start with what you know. A simple design for 100 users can be more effective than a complex, premature architecture.
Why Kubernetes? Because it allows managing small yet powerful clusters. At GitHub, I've seen that even with just five or six containers, we can handle millions of requests per second. Simplicity is your best ally. A simple code means fewer bugs, less maintenance, and more scalability. But watch out, don't confuse simplicity with naivety.
Vertical vs Horizontal Scaling: Making the Right Choice
Vertical scaling means adding power (CPU, RAM) to an existing machine. Meanwhile, horizontal scaling involves adding more machines. I've witnessed many projects fail by opting for one over the other without analyzing real needs. Take a cloud-native service for instance: sometimes vertical scaling can be more efficient initially.
But watch out for costs! Adding machines often seems simpler, but it can quickly become expensive. One of my projects saw its AWS bill skyrocket due to poor estimation.
- Vertical scaling: Increases the power of a machine.
- Horizontal scaling: Adds more machines.
- Costs and infrastructure: Analyze them before deciding.
Balancing Business Impact and Technical Achievements
I've often found that technical excellence doesn't always equate to direct business impact. Take GitHub, managing millions of requests per second. Technical decisions are driven by real business constraints. The key is understanding these constraints to guide your design.
"Sometimes solving business problems means building software that is just good enough for today."
Communicating these technical choices to non-tech stakeholders is crucial. You need to be able to explain why a solution was chosen without complex jargon.
- Understand business constraints before deciding.
- GitHub: An example of effectively managing massive requests.
- Communication is key to stakeholder buy-in.
Continuous Learning and Adaptation in Tech
In tech, staying updated is second nature. I remember learning to use new tools and adjusting my designs to accommodate user growth. GitHub has shown that you can learn a lot from past mistakes and successes. Experimenting and iterating is key. I've often had to revisit my initial designs to align them with new market realities.
What tools to use for continuous learning? Here are some resources:
Hiring and Team Management: Building the Right Tech Team
Building a team capable of tackling scalability challenges is no easy feat. When hiring, I look for candidates who not only have the technical skills but can also integrate into the company's learning culture. A team needs to be adaptable and ready for continuous learning.
Watch out for common pitfalls: hiring solely for technical skills without considering cultural fit can be disastrous. Ensure each team member understands business objectives to align technical achievements with the company's real needs.
- Look for candidates with a balance of technical skills and business understanding.
- Foster a culture of continuous learning.
- Avoid common hiring mistakes by also evaluating cultural fit.
In system design, grasping business needs is as crucial as technical skills. Here's what I've nailed down:
- Practical first: I always prioritize a pragmatic approach. We build to scale, but also to drive direct business impact. When I think '10x', I don't just stick to technical prowess; aligning with strategic goals is key.
- Continuous adaptation: Successful systems aren't static. I've often had to tweak my scaling strategies, whether vertical or horizontal, to stay in line with market changes and objectives.
- Concrete impact: GitHub services handle millions of requests per second, but what truly matters is how these capacities translate into business value.
Looking ahead, see system design as a driver of change. Ready to take things to the next level? Start by evaluating your current scaling strategy and aligning it with business goals.
For deeper insights, I recommend watching the original video: Watch here. It's like chatting with a colleague who's been down the road already.
Frequently Asked Questions
Thibault Le Balier
Co-fondateur & CTO
Coming from the tech startup ecosystem, Thibault has developed expertise in AI solution architecture that he now puts at the service of large companies (Atos, BNP Paribas, beta.gouv). He works on two axes: mastering AI deployments (local LLMs, MCP security) and optimizing inference costs (offloading, compression, token management).
Related Articles
Discover more articles on similar topics
Merging Product and Engineering: Reinventing Processes
I was skeptical at first. Going from six engineers to just two seemed like a recipe for disaster. But merging product and engineering not only worked but thrived. In today's fast-paced tech environment, efficiency isn't just a buzzword—it's a necessity. By integrating roles and leveraging AI, we've redefined our workflow. This has transformed team dynamics, strengthening feedback loops and collaboration. Now, with one frontend and one backend engineer, we balance developing new features with maintenance. We experiment, adapt, and innovate. But watch out, the risk is moving too fast and losing sight of key objectives. When it works, the impact is tangible.
Mastering Software Engineering: From Junior to Architect
I remember when I first dipped my toes into software engineering. It felt like a maze of code and concepts. Fast forward to today, I'm orchestrating complex systems as an architect. Moving from a junior developer to an architect is about more than just coding. It's about mastering the art of software engineering, understanding real project demands, and navigating integration challenges. Let's talk about the core skills: continuous learning, reading and understanding code, facing middleware and DevOps challenges, and effectively communicating in tech. Let's dive in, I'll show you the way.
Building a 20M€ ARR App: My Journey
I didn't set out to build a 20M€ ARR app. It all started with a list of 100 dreams after Christmas 2019. Fast forward to today, and we're in 52 countries, tackling urban loneliness head-on. I orchestrated the development, navigated through financial challenges, and adjusted our business model to reach a global audience. This journey is more than just code; it's about human connection. In this article, I take you behind the scenes of our entrepreneurial journey, from the initial idea to managing global expansion. If you're ready to see how an idea turns into success, dive in with me.
Raise Funds Without Losing Control: Key Strategies
Raising funds is like sailing through a storm. I've been through the fundraising wringer more times than I can count, and if there's one thing I've learned, it's this: preparation is everything. Whether you're dealing with European or American investors, every detail matters. Let me guide you through this maze without losing your shirt—or your company. We're diving into everything: the differences between European and American investors, legal prep, term sheets, and even using BSR as a fundraising tool. Don't get burned by antidilution clauses or inflated valuations. Let's explore key strategies to raise funds without losing control.
Deploying Kimmy K2: My Workflow Experience
I've been hands-on with AI models for years, and when I got my hands on the Kimmy K2 Thinking model, I knew I was diving into something potent. This model marks a significant evolution, especially coming from a Chinese company. With its impressive technical capabilities and implications for the future of AI, Kimmy K2 isn't just another model; it's a tool that excels in real-world applications. Let me walk you through how it stacks up against others, its technical features, and why it might be a game changer in your workflow.