Too Smart For Their Own Good

The Allure of Complexity

I have been fortunate to work with brilliant people over my last almost 20 years of creating software. Sharp folk who anticipate every flaw, spot patterns faster, see downstream consequences that others miss. They speak confidently and authoritatively, are invigorating and usually cause others around them to sharpen up. They are an obvious asset but, if you're not careful, a huge liability.

I won't touch the subject of personality traits that come alongside such intellect. It is a bit of a stereotype to expect such people to somewhat be lacking in empathy or humility. I have to say that has rarely been my experience - fortunately. What I have observed instead, time and again, is a concerning acceptance and coexistence with complexity.

Software and system design is fundamentally about balancing constraints—it is rarely about creating the "perfect" solution. Tradeoffs have to be made between performance, complexity, cost, security, and many other factors. My observation is that more than being at ease with complex solutions and designs, brilliant people are often attracted to them. Without sufficient self-regulation, small decisions snowball—until complexity exerts a gravitational pull that drags down your whole project or system.

When accidental complexity is normalized, it starts to not look like much of a tradeoff. This is the complexity that we create, often unnecessarily. And some brilliant people tend to create a lot of it - because they can handle it, because it might even be intellectually rewarding.

To make matters worse, these people often design or engineer critical parts of huge systems, the primitives around which everything is built up. They are attracted to what is deemed to be the most important work, and often assigned to it precisely because of their perceived ability to tackle such challenges.

Don't get me wrong, they can be relied upon to get projects to completion. However, at best they introduce a lot of friction, and at worst they can derail the whole thing.

Making the most of your brightest

Obviously this embracing of accidental complexity is not something that affects everyone. It's critical not to let silos form within your organization, where this complexity orthodoxy exists. The key is to surround them with pragmatic engineers who have a strong bias for simplicity, creating a healthy tension that regulates design choices. Some attrition can be expected and is in fact desired.

Those more problematic individuals also benefit from being often pulled from their narrow slice of the system that they focus on, and put in situations where they need to deal with the broader system. Overwhelm them, put them in uncomfortable situations, try to get them to zoom out and realize that the complexity budget is already well overrun.

If you're not careful, over time cognitive monocultures can take hold. Complexity becomes accepted - not as a problem to solve, but as a natural state of good engineering. People stop asking how things can be made simpler. The cost of this accidental complexity becomes invisible over time, dismissed as simply 'the way things are done.'

We need smart people to solve the hardest problems, and smart people need hard problems to work on. They also need regulation. The goal is to find simple designs that effectively address complex problems.