Four Qualities of Good Models

Although modeling has always played a role in engineering, it has become a central focus with the advent of model-based systems engineering. While we as engineers are focused on engineering and building systems, sometimes the topic of modeling, as both an art and a science, still gets short shrift.

If we are to take full advantage of the power of modeling, there are important questions to be answered. How do we know if our models are adequate? How can we compare the “fitness” of different models?

Modeling is actually a very common experience across a wide variety of disciplines. Engineering, management, economics, psychology, architecture, even English and history, all use models. I learned modeling through my university studies of math and physics, but it wasn’t until my studies in operations research that I fully understood what makes a good model. I’ve come up with four top qualities of modeling that I use as a guide when modeling.

1. Model with Purpose
First and foremost, I’ve learned that it’s essential that we understand the primary purpose of the models we create. How well we meet that purpose is a measure of the quality of our model. A model with too many purposes can become unwieldly, overly complicated, and essentially useless. As systems engineers, we need to keep in mind that the purpose of our system model is to show that a system design will meet the needs of the user. Detailed engineers also have models that are designed with the purpose of meeting requirements supplied by systems engineers. We can then integrate our system model with the more detailed models to form the digital engineering enterprise.

This brings us to my favorite definition of modeling—and there are many definitions! It is from Professor Brian Wilson at the University of Lancaster who, along with Dr. Peter Checkland, studied modeling in the 1980s in the context of management science. Dr. Wilson states (emphasis added):

A model is the explicit interpretation of one’s understanding of a situation or merely of one’s ideas about that situation. It can be expressed in mathematics, symbols or words, but it is essentially a description of entities, processes or attributes and the relationships between them. It may be prescriptive or illustrative, but above all, it must be useful.

From Wilson’s definition of modeling, we learn that usefulness is the measure of good models. Models are useful to the extent that they serve a purpose. This comports with another well-known adage from the statistician George E. P. Box that states:

All models are wrong, but some are useful. The practical question is how wrong do they have to be to not be useful?

According to Box, some models are useful and some are not, meaning that some models serve their purpose and others do not. As we model, we must keep an eye on our purpose, and ensure that our models continue to serve that purpose and remain useful as we represent aspects of reality.

2. Model with Simplicity
The second part of Box’s statement brings us to the next quality of modeling, simplicity. It is not practical or desirable to model every aspect of reality, so we need to determine how much of reality we must retain in order to construct a useful model. As we eliminate elements of reality from our models, our models become simpler. As long as this simplicity retains all that is relevant to the model’s purpose, the model will prove useful.

Taking advantage of simplicity allows us to deal more effectively with the complexity we find in our systems. Simplicity is an important aspect of modeling, particularly because beginners in the field of modeling tend to believe that a good model is one that mimics reality as closely as possible. However, modeling aims at simplification held in balance with relevance, rather than as a useless production of complex copies of a complex reality.

Simplifying our models is also a loose application of the “Law of Parsimony.” Attributed to William of Ockham (1287-13¬47), the heuristic was originally used to guide scientific understanding of causation when modeling the physical world. Also referred to as Occam’s Razor, it can be summarized as: “Among competing hypotheses, the one with the fewest assumptions should be selected.” In simpler language, Occam’s razor can be stated as: “The simplest explanation is preferable to one that is more complex.” Note the term “razor” was used to reflect the heuristic’s recommendation to cut out unnecessary assumptions, or constructs of a model. A simpler model with fewer elements is “more wrong,” as Box would say, but still preferred if it meets the intended purpose. In other words, the preferred model is the simplest model that still serves its purpose.

An example of using simpler models can be taken from statistical forecasting techniques based on correlation of data. A model with fewer data elements that makes accurate predictions can be preferred over a correlation model with more data elements with no more accuracy. This is not an absolute, but it is another example about “how wrong” a model can be and still be useful.

Of course there are limits. The Rutherford-Bohr model of the atom with its concentric orbits of electrons around a central nucleus is simpler than Schrodinger’s wave-function model of the atom. However, the Bohr model became less useful as the need for predictive accuracy increased, making the Schrodinger model’s wave-function aspects relevant, at which point the (much) more complicated wave-function model prevailed.

As systems engineers, we strive for the simplest engineering design that meets the needs of the users. Too many unnecessary parts create unnecessary interfaces, and unnecessary risk. When we see a design with unnecessary parts, we call it “gold-plating” for this very reason. A car, for example, intended for commuting will be less complex than a race car, but it cannot meet the same needs. A race car could be used to commute, but we realize it is markedly over-designed. (It is “gold-plated.”)

3. Model with Abstraction
Another important consideration when developing a quality model is the level of abstraction used within the model. The level of abstraction is also referred to as the level of granularity. Both terms refer to the degree of detail used in a model. Abstraction and granularity are inversely proportional. A model with more abstraction has a lower level of granularity, and fewer entities than a more granular, less abstract model.

Modeling with abstraction is an application of the Law of Parsimony. In some cases, a model uses only a single layer of abstraction. Other times, the notions of decomposition and aggregation are used to develop a hierarchical model with multiple levels of abstraction. Hierarchical models can be developed top-down using decomposition, but also bottom-up, using the notion of aggregation. Aggregation refers to the conceptual task of taking a given set of model entities at a given level of abstraction and generating a set of higher level modeling entities that capture the same meaning. Decomposition, or de-aggregation, is the opposite process, whereby a set of model entities at a given level of abstraction are examined, and a set of lower level elements derived that contain more information about each of the entities in the original set.

In some cases, high levels of granularity can be preferred, because they may represent the close approximation to “real engineering.” Some might even propose that a prototype or lab simulation be used instead of modeling. We evaluate these concerns by reflecting on cost, scale, time, and purpose, with the understanding that we model to reduce, not mimic complexity.

Abstraction can make our models much more simple and yet, just as accurate, if we give thought to how the system behaves and is designed at each layer of the hierarchy. In this way, we come up with a scaled-down data model, so that at the higher layers, we deal with data elements that represent the aggregation of more detailed data exchanges. In other words, we can model “status” exchange at one layer of the model, and model more granular types of “status” at deeper layers in a hierarchy. Modeling at various layers like this helps us as systems engineers ensure that our design is adequately complete and captures the intended behavior of the systems. It can be much more difficult to see this view with only highly granular models. The adage “Don’t lose the forest for the trees” comes to mind.

4. Model for Understanding
Finally, we come to the audience of the model. If no one but you can understand your model, it is likely that it will not be very useful. If no other stakeholders beyond your organization understand your model, it will likely not be very useful. Of course, we can insist that everyone learn the intricacies of the model, but more likely, we need to adjust our modeling approach to accommodate the audience.

When our models are not understood, our efforts can become stove-piped. We lose the ability to communicate across engineering disciplines. When models are understood, we break down these communication barriers, allowing us to become truly transdisciplinary in our engineering efforts and better able to achieve concurrent engineering—so necessary in tackling the complexity we find in today’s systems.

Modeling for understanding is just as important as modeling for accuracy and precision. In engineering, we deal with scientific and mathematical models that strive to predict actual outcomes, but we also deal with conceptual models that strive to align the understanding of a set of stakeholders. When you model for understanding, your stakeholders will thank you.

Summary
What does this mean for systems engineers, and more generally for engineering? Our system model should be the foundation of our model-based systems engineering approach. A logical, hierarchical model, primarily directed at the high-level design, is necessary to ensure that our design is complete, captures all the intended behaviors, and meets the needs of the user. The model should also help us track our systems engineering journey to track the rationale for decisions and help lay down an audit trail for later examination.

More detailed models that serve more specific purposes can be developed for the design and development of hardware and software components. We break down these silos of engineering when we connect the systems engineering model with detailed engineering models to create a fully integrated digital engineering environment.

One final warning: modeling can be fun, and it can easily become an end unto to itself, so remember these four characteristics of good models. Focus on a single purpose, use simplicity when you can, leverage abstraction, and of course, make sure the models are understood by all stakeholders. In this way, we can model to advance our purposes rather than for the sake of the modeling exercise itself.

If you have any questions about modeling, feel free to contact me at mark.simons@vitechcorp.com.