The Problem with Monorepos and Shared Libraries

There’s a unique problem with monorepos and shared libraries that often gets overlooked. It happens when:

• You have a large monorepo, worked on by several teams of developers
• You have shared libraries that are sym-linked to apps or services
• You need to make a breaking change to a shared library

I mentioned this in a reply to @mattpocockuk who recently joined turborepo, which is a part of Vercel and develops monorepo tooling.

I’m not sure many people are aware of this problem, so I wanted to expand on it further.

The Problem

In a monorepo, libraries are typically sym-linked to consuming applications or services.

What this means is that every change to a shared library is immediately used by consumers.

For non-breaking changes, this is seen as a huge benefit because it significantly reduces maintenance overhead.

But for breaking changes, it introduces a problem.

First let me recap what a “breaking change” is:

A breaking change is any modification made to a library that requires consumers of that library to make their own changes to accommodate that modification.

Examples: removing a function, changing a function signature, adding/removing side-effects

Anti-examples: adding a new function, adding a new parameter to an existing function

Because consumers immediately consume the breaking change, this leads to the following conclusion:

The breaking change and any remedies for the breaking change, inside every consumer, needs to be made in the same pull request

Here are the problems with that fact:

1. This can be a massive task and the difficulty scales with the size of the monorepo. In some cases, it can be an infeasibly large task to do manually and you’d have to write a code-mod.

2. It can lead to very large PRs. Large PRs are typically seen as a bad practice. As the size of the PR increases, the quality of code reviews decreases and the deploy risk increases. A quick google says PRs less than 400 lines are ideal.

3. Testing the change can be very difficult. Many companies have significant portions of their monorepo that are missing code coverage. If you’re the one making the breaking change to the library, you may have to manually test many consumers instead of relying on automated tests.

   If you have to manually test 30-40 consumers, you may have to follow 30-40 READMEs to get the consumers running locally. I don’t know about you, but getting my local setup for even 1 project can be a nightmarish task of battling out-dated READMEs, fixing DB issues, turning on feature flags, getting seed data, setting up authentication, etc.

The Options

There are a few ways to deal with this:

Option 1: Resign yourself to making large PRs

This is a valid option if you have the time and your monorepo is not too big that it becomes infeasible. If your monorepo is very large, this option may still be possible if you write a codemod to automatically create the changes.

However, this option becomes riskier the less test coverage your monorepo has. A major benefit of comprehensive test coverage is so that you can make far-reaching changes like this with confidence. It’s another good reason to strive for high test coverage.

Option 2: Use versioning instead of sym-links

By allowing consumers to specify the version of the library they want, each consumer can upgrade to new versions at their own pace. However, there are some major drawbacks to this:

1. You lose the productivity benefits when making non-breaking changes. Consumers no longer automatically get improvements to your libraries, and you introduce an upgrade process with maintenance overhead.

   This is especially a problem if a consumer gets stuck on an older version due to a breaking change. Since they’re stuck on this version, they don’t get any future non-breaking changes.

   This point specifically is why a lot of people use monorepos and it almost defeats the purpose of a monorepo to use versioning.

2. Some technical problems arise when multiple copies of the same library are in your dependency tree and if that library is stateful.

   For example, many react libraries (and react itself) require only one version of themselves exist in the tree. Because of the way react’s context mechanism works, you can introduce complex bugs when installing multiple versions

3. For a frontend application, multiple versions of the same library cause the bundle size to increase unnecessarily.

4. Managing and maintaining multiple copies of libraries is more complex than a single version.

Option 3: Don’t add shared libraries to monorepos

This is a valid option, but it has most of the same drawbacks as above.

I included this option mostly to make a secondary point: shared libraries are the only real complexity of monorepos.

If you don’t include shared libraries in your monorepo, there isn’t really any compelling argument left against monorepos.

For example, if you put all your micro-services in one monorepo and no shared libraries, you will still see good productivity benefits over a poly-repo approach with few downsides.

Option 4: Make breaking changes in multiple steps, using deprecation

The last option is to make breaking changes in multiple steps:

1. Create the new way of doing things and deprecate the old way of doing things
2. Migrate consumers to the new way of doing things, in 1-many PRs
3. Remove the old way of doing things

This procedure is meant to de-risk the large PR option while keeping all the benefits of monorepos. It essentially spreads out the hardest part of making breaking changes into multiple PRs. Then the actual breaking change is made after the risky part is already done.

There’s a couple trade-offs with this approach:

• You need to support 2 ways of doing things at once for a transitionary period
• It’s more effort for the library maintainer
• The entire process may take longer than the single PR approach

It’s worth noting this type of deprecation process is a common procedure even outside monorepos. Package maintainers often accept the extra work required for this process to give consumers more time to more gradually migrate away from the old way of doings things.

Design your library so more changes are non-breaking than breaking

In either a monorepo or polyrepo setup, breaking changes are burdensome. While non-breaking changes require no work from consumers, the effort required for a breaking change scales with the number of consumers.

There are some things you can do to design your library such that more changes are non-breaking than breaking:

1. More long-term planning upfront and a longer review process, with a focus on careful API design. After all, breaking changes are primarily API changes
2. Aim for as small as possible API surface and don’t try to predict all use-cases upfront

Conclusion

In summary;

1. Monorepos shine when making non-breaking changes
2. Monorepos are more complex for breaking changes
3. Breaking changes are manageable and there are strategies to deal with them, but it requires more than surface-level consideration
4. There are ways to design a shared library to minimize breaking changes and optimize for stability
5. Shared libraries are the only significant complexity of monorepos