This post is a bit unpolished. I’m still working on a good way to explain the intuition behind abstractions and how to use them effectively.
I’ve published it anyway because having a basic understanding of abstraction in our shared lexicon will be important for future posts.
In The first principle of software design, I introduced the core “principle” that underlies software design (and, really, design in general): All software is created to fit a context.
We also identified one of the key problems that software design has to deal with: the exceptional complexity, multidimensionality, and fractal-like nature of the context the software is designed for. Not only that, but the context of a software solution is constantly changing over time.
As a result of this complexity, any useful piece of software will be too complicated to keep the whole design, from the high-level purpose down to each line of code, in your head at once.
The fundamental technique that software developers use to tackle this deep complexity is called abstraction:
Abstraction is encapsulating complexity within a simpler interface.
Abstraction is a difficult concept to wrap one’s head around at first, precisely because it’s so generalized. But we’re surrounded by abstractions all the time, in our software projects and in life in general.
The classic example is a car. You don’t need to know how a car works — the differential, power steering, combustion engine, etc. — to drive one. The complexity is literally hidden “under the hood”. All you need to interact with is the interface that the car exposes: pedals, steering wheel, etc. This is the fundamental idea of abstraction: providing a simple interface to a complex system.
Without abstraction, software engineering at the current scale wouldn’t be possible.
Consider how complicated it would be to design a software system if we had to operate at the level of individual transistors. Only small problems would be solvable at that level of abstraction. This is why we invented logic gates, CPUs, machine code, assembly code, low-level programming languages, and finally high-level programming languages like JavaScript, Python, etc.
However, writing code, line by line, in a high-level, general-purpose programming language is still operating at a lower level of abstraction than we want.
This is why all such languages include tools for building your own abstractions — for example, functions, procedures, processes, actors, classes, servers, etc. Your job is to make effective use of these tools to implement abstractions that make sense for your specific application.
This figure implies that application-specific abstractions are just one layer, but in fact you’ll build as many layers as you need to make it tractable to comprehend your system.
Carefully building exactly the right number of abstraction layers is a core competency for designing software systems. This is one of those cases where “a failure to plan is a plan to fail”, so think hard.
Investing some time in building proper abstractions might seem slower than just diving into implementing features, but it will pay off in the long run. Especially if you want your software to be comprehensible and maintainable to people other than yourself (or to yourself in 12 months when you’ve forgotten everything)!
The exact abstractions to use may not be obvious at first. There’s nothing wrong with starting the software design with either preliminary abstractions, or none at all, as long as you refactor the system repeatedly as it gets more complex to introduce abstractions where they’re needed.
Another example of abstraction in the real world is a map.
The map is a simplified representation of something or somewhere (which we’ll call the territory). It only includes the details of the territory that are expected to be relevant. It also provides an appropriate scale to allow all the relevant parts of the territory to be perceived at once.
A map that contained all the information of the territory would be uselessly detailed, since it wouldn’t simplify anything. Understanding the map would be just as difficult as understanding the actual territory. But if the map is simpler — an abstraction — then it can be useful.
For example, a road map lists the major roads. It doesn’t list every house, cul-de-sac, tree, and so on. It’s just the roads. This is what makes the map useful for people driving through.
That’s why, even though Google Maps has satellite images for the whole world, the default maps view shows something similar to the familiar old road maps, instead of showing the actual imagery. It’s an acknowledgement of the fact that for most people looking at Google Maps, what matters are the locations of roads, buildings, and water features. Individual trees, the color of the ground, and so on, is rarely relevant and introduces unnecessary complexity in the map. (Of course, the satellite imagery is still available for when you do care about that level of detail, effectively allowing the user to choose what “level of abstraction” the map should display.)
Domain-specific languages (DSLs) are an example of a carefully designed abstraction that builds on top of a general-purpose programming language to make it easier to express the behavior you want for a specific domain (i.e. problem context).
A “true” custom DSL is often overkill, but conversely, most complex software designs benefit from a DSL-like collection of high-level functions to make it easier to configure and understand the system’s behavior.
An alternative to using abstractions to tame complexity is to make the complexity as easy as possible to see and comprehend. Languages like APL are illustrative of this idea. A whole program that does real work can fit into a single screen, and can be viewed and edited as a “unit.”
However, even this approach is just a stopgap measure that only works to a certain point. It’s enabled by the fact that these languages expose a very high level of abstraction to the programmer to begin with, but eventually you will need abstractions as your program gets more complex.
The unique power of abstractions is that by layering them atop each other, you can make problems with any level of complexity become tractable.
In software development we also talk about leaky abstractions. These are situations where the interface does not fully encapsulate the complexity of the underlying system. The complexity “leaks out”, so to speak.
To use our car analogy again: If your car battery dies, you may need to give your car a jump start. But there’s no “jump start” button on your car’s dash (unless you have an RV with two battery systems!) Instead, you need to actually open the hood, find your car battery, connect leads to the right terminals, and so on.
Because the interface has no way to perform the jump start operation, you need to look under the hood. This is a “leak” in the abstraction.
Practically speaking, all abstractions have some level of leakage. In fact, software engineering is one of the places where you’ll encounter the most complete abstractions out there.
On a philosophical note, abstraction is essential to how we understand the world around us.
From an outside perspective, what we “see” is a vast array of photons of different frequencies that activate receptors in our eyes. At first (as a baby), these colors and lights are perceived just as they are. But as we mature, we learn to understand the idea of individual objects — a rock, a box, a plank, and so on. And perhaps multiple planks in a certain configuration looks like a table, or chair.
These impressions — rock, plank, chair — are mental models, in other words, abstractions, that we automatically use to understand the sensory input we receive from the world around us.
