Legibility and Software

Oct, 30 2020
7 Minutes

I'm beginning to familiarize myself with the topic of legibility - a concept from the book "Seeing Like a State". While I haven't dug in deep, it appears the following are "cannon" for the topic:

And these are great accompaniments:

So let me take a quick stab at defining legibility, then take a glance at complexity, and finally bring this back to software.

Legibility - the idea that complex systems are inefficient and can be rearranged with authoritarian mandates to optimize for ease of governance (in other words, when central authorities desire complex systems to be simplified for easier management, they desire legibility)

Alright, that was quick - onto complexity.

Complexity

I expect to write about this in the future, but here's a quote I've begun to cite with religious zeal and frequency:

For the simplicity that lies this side of complexity, I would not give a fig, but for the simplicity that lies on the other side of complexity, I would give my life

This quote explains that there are two forms of simplicity.

The first is the type possessed by those who suffer from Dunning-Kruger's: you know too little, and thus you think you know it all.

The second is the type wielded by experts who truly understand. This form of simplicity has "scaled" the complexity curve. If information is wild, then this form of simplicity is information tamed into a mental model.

For humans, complexity is not legible, whereas simplicity is. We desire legibility, but as you can see, one form is good. The other.. is quite bad.

Complexity, Legibility, and Software

Software tends towards entropy. With time, as each feature gets added, as each bug gets "fixed", as developers come and go, the software system inevitably becomes a mess - no matter how pristine it started.

At some point, this complexity and entropy slows the team down. The company yearns for legibility!

In Alexander's review of "Seeing Like A State", he describes a similar situation in 18th century Prussia:

Enlightenment rationalists noticed that peasants were just cutting down whatever trees happened to grow in the forests, like a chump. They came up with a better idea: clear all the forests and replace them by planting identical copies of Norway spruce (the highest-lumber-yield-per-unit-time tree) in an evenly-spaced rectangular grid

And then?

This went poorly. The impoverished ecosystem couldn’t support the game animals and medicinal herbs that sustained the surrounding peasant villages, and they suffered an economic collapse. The endless rows of identical trees were a perfect breeding ground for plant diseases and forest fires. And the complex ecological processes that sustained the soil stopped working, so after a generation the Norway spruces grew stunted and malnourished. Yet for some reason, everyone involved got promoted, and “scientific forestry” spread across Europe and the world.

Sadly, this tale does not only apply to these Enlightenment rationalists. Anyone who has worked on a software team long enough to witness a regime change might resonate with this hypothetical:

Imagine a software engineering team who is growing fast, but lacks a CTO. The previous VP of Engineering was well liked, but resigned because he wanted to live in Boulder, CO and be among the rock climbing trails. And so, someone new comes in.

In order to govern their empire, this new CTO yearns for legibility. After 7 weeks on the job, they demand that all systems be rewritten in Java 7. Because Java 8+ and functional programming is "too hard to hire for" (which is code for the leadership by insecurity).

On the bright side, they've decided to start small and break up the monolith into microservices - and have chosen a couple of pilot projects to start.

On the downside, these projects will be led by the Council of The CTO, which is comprised of a few senior engineers the new CTO has spent the last 7 weeks hiring.

You can't help but notice that this collection of senior "talent" bears striking resemblance to Wormtongue whispering sweet sycophantic yes-man, yes-boss, reality-ignoring stupidity into the ears of the CTO.

Note, however, there is a difference between King Theoden and our hypothetical CTO: the causal relationship of incompetence doesn't run in the direction of powerful brownnosers leading to degraded leadership skills. Rather, it's ego-centric leadership leading to a festering entourage of way-too-highly-compensated henchmen, who reign for a brief moment of time.

And thus, we begin yet another "High Modernist" attempt at enforcing top down, rectangular grid, central planning on a software codebase. We all know how this disaster ends. The complexity wins, and the floundering CTO and their Council has found themselves thrashing about in the tar pit.

Sigh, one should not dive head first into the tar pit - it's best to "lay some boardwalks across the tar".

Boardwalks for Complexity

So if hiring Mr. CTO isn't the way to go, how can we balance our need for legibility without causing an ecological meltdown?

I can't answer this conclusively, but I'm offering "boardwalks" - some tools that might help this dangerous traversal.

1) Archaeologists not Architects

When it comes to reining in the complexity of software, I believe that there is one truth we are trying to optimize for. It comes from the line in "How to Build Good Software":

The main value in software is not the code produced, but the knowledge accumulated by the people who produced it.

In many cases, the root cause of disastrous rewrites simply occurs because the people who wrote the first version of the software did not leave a legible trail for the people behind them. In fact, it's often the case that the trail blazers/code cowboys and cowgirls who wrote the code never even understood the domain themselves, simply brute-forcing, fire fighting, bug squashing, and praying their way to something that was Good Enough™.

But the onus of legibility is never on the original crew, the onus is on the team doing the rewrite. This squad must understand the situation on the ground before attacking. Whether or not the original crew left breadcrumbs is irrelevant. They left code and that, sadly, is Your King.

At this point, you want archaeologists and anthropologists, not architects. Anyone who starts architecting before digging is preparing to swim in the tar pit.

2) Clean Code and Legibility

At its core, I believe clean code is simply code that is "legible to humans". If code is quickly understood by the engineers on your team, and can be modified safely, then I believe it is "clean".

I like this concept because it shows us that legibility CAN be imposed on something, without sacrificing correctness and robustness. In fact, clean code is often more correct and more robust - because the abstractions are in place to add optimizations over time.

The concept here is that legibility can be achieved at the micro level if we strive for clean code. If new hires can make progress on your code base without much polite complaining, then your code is reasonably clean (i.e. they don't say things like "The code's pretty good, although A, B, C, D, E, F, G, H, I, J, K... has slowed me a bit down this week"). If they're complimenting the codebase, then you know it's clean (the joke here is that this doesn't happen).

So micro-legibility is great and all, but we're really looking for something bigger. Where else can we increase legibility in software without sacrificing correctness?

3) The micro in Microservices

As software systems get more complex, they often face "The Decision". That's the point in a startup's lifecycle where there are just too many engineers - and not everyone can coexist happily under one roof.

How a company negotiates the transition to microservices is important. Having seen a failed attempt at this, I have increased conviction that the right way to think about microservices (as with most new things, really) is to start super small.

Here's how you might start:

  • Start a new feature as its own tiny microservice (typically when you need new resources and endpoints, not when you're adding fields to an existing resource)
  • Refactor tiny, stateless portions of your application into Serverless functions (anything that uses ImageMagick is a fantastic example - thumbnail generation, image resizing, etc. Here's some more)

In these cases, it's possible to chip away at pieces of the monolith while still preserving most of the hard earned "learnings" of the legacy code. That's because the ambitions are small. This avoids the hubris of diving straight into the tar pit.

4) The Strangler

If your ambitions are not small, and you want to finish this rewrite before the next promotion cycle - then good luck, I can't help you. But if you're open to the idea that these things will take time, let's consider, The Strangler.

I've written about this in detail before, and I don't wish to rehash everything from then. That said, let me pluck a few choice quotes so you don't have to redirect off this page:

Thankfully, Azure has put out a great description of the Strangler Pattern... Here’s their illustration of how to use it:
The key point to note is that in this case, the Strangler is a Facade, a  wrapper around both the legacy and “modern” systems. This differs  slightly from Martin Fowler’s formulation. For Fowler, there isn’t  necessarily a Strangler Facade, just a legacy and modern system that  take different amounts of traffic.

Conclusion

Legibility is a fascinating idea, especially in the context of complexity. And there's one thing I haven't really spoken about yet, which is human systems that are too complex tend to become brittle. Was the Fall of Rome simply the Empire reaching a tipping point of complexity?

And thus, we reach for legibility, not necessarily because we like it and want to look like geniuses, but sometimes, because we have to.

I don't have any answers, but I have offered "boardwalks" that you can place atop the tar pit:

  1. Favor Archaeologists, not Architects
  2. Strive for Clean Code and Legibility
  3. Put the micro in Microservices
  4. Use the Strangler

Whether or not we can achieve legibility in software is not something I can answer. But hopefully, this article helps us to understand how we can "See Like a Software Engineer".