I’m not exactly a large-scale DevOps guy, but I can tell ya we’ve been moving back toward a monorepo at CodePen and it’s rife with advantages over a system with lots of smaller repos. For us, I mean. It’s very likely that you have entirely different challenges and have come to entirely different conclusions at your place. 🤙
I was thinking about this after reading Ben Nadel’s “Why I’ve Been Merging Microservices Back Into The Monolith At InVision.” Even though our conclusions are similar, I can tell he faces an entirely different set of problems.
Microservices Solve Both Technical and People Problems
A technical problem is one in which an aspect of the application is putting an undue burden on the infrastructure; which, in turn, is likely causing a poor user experience (UX). For example, image processing requires a lot of CPU. If this CPU load becomes too great, it could start starving the rest of the application of processing resources. This could affect system latency. And, if it gets bad enough, it could start affecting system availability.
A people problem, on the other hand, has little to do with the application at all and everything to do with how your team is organized. The more people you have working in any given part of the application, the slower and more error-prone development and deployment becomes. For example, if you have 30 engineers all competing to “Continuously Deploy” (CD) the same service, you’re going to get a lot of queuing; which means, a lot of engineers that could otherwise be shipping product are actually sitting around waiting for their turn to deploy.
Advantages of the Monorepo (for us)
- One ring to rule them all. You
git pullone repo and you are 100% up to date with everyone else and have everything you need for a complete dev environment.
- No stray puppies. There is no confusion on where the action happens on GitHub. You do pull requests against the monorepo. You open issues on the monorepo. This avoids scattered activity that gets lost.
- Kumbaya. You can share code. It can be particularly helpful to share utilities or components anywhere in the codebase. We poked at ideas like publishing shared bits to npm for other repos to use, but that workflow was janky compared to having the code together in on place.
- Growing old together. There are no old and neglected repos, because it’s just one. For our small team, having dozens of repos meant some of them had old outdated dependencies, ancient versions of Node, linting and formatting rules that were out of sync with other repos, etc.
Disadvantages of the Monorepo (for us)
- Deployment trickiness. I think the main reason we split off repos originally is that the code in those repos needed to go to unique places. They might have represented an individual Lambda or individual service on some other server. An individual repo means it’s easier to hook up stuff that is unique to that server/service, like CI/CD.
Yes, I get that this is controversial.
I actually don’t care that much. I’m not gonna get all intense about this like air fryer people and CrossFit zealots. Here’s a full-throated argument against monorepos from Matt Klein.
I’m just saying: it’s been clearly useful for us. I can see how things play out differently for other companies. I can see how a company that works with contractors might want to limit their access to something less than an entire monorepo. I can see how a git repo might become unwieldy and large. Those aren’t problems for us at CodePen right now, so the advantages of a monorepo win.
Is really interesting that you point out the difference in benefits gained through a monorepo. I have been using a monorepo as a solo dev for 3+ years now and I never want to go back. The main advantage for me was the speed at which I could start a new project. I’ve had so many ideas, but I prefer a very specific workflow that constantly evolves so setting it up everytime and having quality engineering fall out of sync was a nightmare for returning to projects.
As you mentioned though, different types of projects are probably the most difficult problems to solve. When I started my monorepo I ended up writing my own CI/CD tool to be aware of what type of project the folder was based on structure and building/deploying it differently depending on that structure. As well as being aware of how each project interconnects dependency wise. It uses a plugin system so when I want to cover a new type of project I can write a plugin for it.
It’s been quite the experience using the monorepo and having huge gains in creating projects quickly and especially in refactoring. I don’t think I’ll ever go back. It’s so nice having my tool notice when I change a single module that is a dependency of say 4 other ones and it auto deploys each of those. It makes refactoring a not so scary feat as I never need to know what is being affected. The truck is keeping good testing practices. Which I’m still improving over time.
Really a great read and perspective to see especially from other Oregonians.
I really like the distinction between using microservices to solve technical problems vs. using microservices to solve people problems. I personally see using microservices to solve people problems as the happy path and using microservices to solve technical problems as more of an edge case. I get the impression though that most developers’ experiences are the opposite of that.
Monorepo has been really helpful for us as well, everything got so simplified and quick.
Also we have tackled the deployment problem by creating separate binary directory for each microservice during build and then using deployment script which does n deployments instead of 1 during CD.
I’m curious why you don’t mention dependency versioning. That’s usually touted as the main advantage of the monorepo: there’s no need to manage versions (bump versions, update dependencies, etc). Did anything change for you in that regard?
It feels like thats sometimes a cause of pain. For example, to share things across sub-projects, we were using Lerna. But it was causing issues with dependencies having weird conflicts. Like a React component in a design system library that uses some version of React and then a sub-project in Next.js using a different version of React and having that be hard to chase down. Yarn Workspaces solved some of it. But yes, I’d say overall, it’s kinda nice to have one repo where you’re doing chore-version-bumps.