Design in Construction

Teaching about design is tricky because the processes and methods shown to students tend to be optimized solutions, whereas the reality is often a lot messier. What gets presented as an ideal route from A to B is something of a simplification—a fantasy of how design would work if we all had prescient knowledge. Design problems are wicked paradoxes, in that we usually need to solve what we think our problem is in order to actually define it (and then solve it again). You might recall from a couple weeks ago the importance of problem definition as a prerequisite to software construction. As a mission statement for a project, problem definition can provide a clear understanding of what our goal is (our "Point B"), lest we find ourselves tilting at windmills. As a guideline for detailed design of a complex system—be it a software program, a building, an assembly line, or an oil painting—the very concept of a clear problem is often the problem.

This week's chapter is titled "Design in Construction" and it marks the beginning of "Part II: Creating High-Quality Code." In contrast to earlier chapters where we examined the software architecture planning process (high-level design), this chapter is concerned with actively designing the structure and multi-leveled logic of the code itself (low-level design). It turns out that the best way to get started with writing precise, rigorous code is a lot of sloppy, non-deterministic, meandering design work. This chapter was fun to read, but it's tricky to write about. On one hand, Steve McConnell's best practices for design in software construction read like the ten commandments of object-oriented programming. On the other hand, this chapter offers a wonderful conceptual framework for managing a design process with sanity and grace. Readers familiar with OOP would no doubt get some value out of me describing in detail what each of those best practices are (and I may yet do so), but I do think that the more abstract concepts of system design make for more interesting (and accessible) reading.

At its core, software development is the process of taking some problem from the messy real world, identifying not only the typical implications of that problem but also all the exceptional cases, and then designing solutions that are exactly correct (not mostly right most of the time). It's a daunting task made possible only through the aggressive and relentless management of complexity. Therein lies another paradox: as software accommodates more depth and nuance, its real-world applicability grows, thereby raising the bar on how much complexity must be accommodated by the next generation of systems.

Managing complexity is "Software's Primary Technical Imperative," according to McConnell. Writing in the late eighties, the Dutch computer scientist Edsger Dijkstra noted that software engineering was perhaps the only human endeavor where such a scope of reason was required:

From a bit to a few hundred megabytes, from a microsecond to a half an hour of computing confronts us with completely baffling ratio of 10⁹! The programmer is in the unique position that his is the only discipline and profession in which such a gigantic ratio, which totally baffles our imagination, has to be bridged by a single technology. He has to be able to think in terms of conceptual hierarchies that are much deeper than a single mind ever needed to face before.¹

At the time of publication of the second edition of Code Complete in 2004, that ratio was something closer to 1:10¹⁵. I can hardly imagine what power of ten would be required today. Suffice it to say, it's impossible to contain an entire computer program within your head. The point of low-level design in computer programming is to minimize the amount of stuff that the programmer has to consider at any given moment and to prevent the needless proliferation of "accidental complexity."

The primary method for compartmentalizing a design into manageable chunks is to leverage hierarchy. At the top level, there is the system itself, followed by important subsystems, the classes and objects that make up the subsystems, the logical routines that form the working parts of each class, and finally the internal routine design (how the code is written).

Not every level is designed or defined on every project; smaller projects might lack subsystems and just employ a collection of classes. Internal routine design must happen by necessity, but it might not be consciously done by the programmer who dives right in. One common thread among all levels of design is the employment of heuristics to drive decision-making; indeed, subjective and non-committal heuristics lend themselves well to the messy and inquisitive process of design. For the sake of brevity, I won't delve deeply into these methods, but they are listed here:

I would very much love to delve into each of these points, but the exigencies of my self-imposed weekly publishing schedule prevent me from getting too far into the weeds on OOP (at least for this week). In concert with design heuristics, there are also different design practices that can be employed:

The messy, recursive design process affords programmers the best opportunity to make mistakes early, before implementing anything in a programming language. Some of the methods listed here are contradictory—not all are applicable to every project. The main takeaway is that low-level design, of any type, is primarily concerned with the compartmentalization and careful management of complexity. The closer you inspect a thing, the more overwhelming it can become. Anyone can observe a skyscraper from afar and easily grasp its whole. When you start to pull apart the curtain wall and must consider (in the simultaneous contexts of the entire building, the facade design, its structural system, and its constituent wall panels) extruded aluminum mullions, glass coatings, sealants, anchoring hardware, and shading systems, the complexity starts to become very overwhelming indeed. Computers are fantastically well-suited to considering 10⁹ elements at a time—humans less-so. If you're a designer working closely with fine-grained things, you owe it to yourself (and certainly to your collaborators!) to bear in mind the Primary Technical Imperative. Next week we'll thoroughly investigate level three of the system hierarchy and the cornerstone of OOP: classes and objects.

References:

¹ Dijkstra, Edsger W. "On the Cruelty of Really Teaching Computing Science." 1988. EWD 1036. University of Texas at Austin, https://www.cs.utexas.edu/~EWD/transcriptions/EWD10xx/EWD1036.html.