The quest for a “single source of truth” and a model-based systems engineering (MBSE) practice grounded in integrated data is underway across the systems engineering world. The goals are laudable and the work being done is important, but there are some pitfalls along the way. It is critical that, in our enthusiasm over the digital thread, we don’t lose sight of the basic aim of model-based systems engineering.
In the Systems Engineering Handbook (v. 4) (SEHv4), INCOSE states that the goal of systems engineering is “providing a quality product that meets the user needs.” To put the bottom line up front, then, the fundamental purpose of systems engineering is to design/architect a solution to our customer’s problem that can be realized within the constraints of the problem space to meet the customer’s needs. This involves making predictions about possible solutions as to whether a particular solution will solve the problem(s) at hand and meet the users’ needs.
In an MBSE setting, the basis for making that prediction is a model of the system of interest that represents the solution under consideration. It is important to remember that we are talking about model-Based SE and not merely model-Involved SE. A number of models may be involved in – and make important contributions to – the prediction (e.g.- analytic models), but, in the end, it is the system model (variously referred to as the descriptive system model or the architectural model) that is the basis of model-based systems engineering. The purpose of that model is to provide the systems engineer with the basis for her prediction.
George E. P. Box is often quoted as saying, “All models are wrong, but some are useful.” He was, of course, referring to the nature of models as a limited representation of reality, where their limitations make models “wrong” as judged by their fidelity to the underlying reality. But, more instructively, Box elaborated, “Remember that all models are wrong; the practical question is how wrong do they have to be to not be useful.” The answer to his question about models in general is that they cease to be useful when they are limited in some way or ways that render them unable to serve their purpose.
To answer to Box’s question in the context of systems engineering, the model that is the basis of the prediction regarding the sufficiency of the solution to meet the needs of the users – the M in MBSE – loses its usefulness at the point where it cannot support the critical prediction. At that point, it can’t be the basis of providing an assurance of “providing a quality product that meets the user needs.” INCOSE SEHv4
As David Long has pointed out in an entry in this blog, “There is one and only one architectural model – broad in scope, fundamentally interconnected in nature – and that architectural model connects and coordinates the diverse analytic models. Done well, the architectural model addresses both the problem and solution, reflecting and integrating the key dimensions of both in a manner that clearly reflects the interconnected nature of the system. Done well, the architectural model aligns and maps key terminology across disciplines and concerns, connecting the various perspectives and analytical considerations. In addressing needs, logical solution, physical solution, and V&V, the descriptive model is highly connected.” Long, One Model to Coordinate Them All (2016).
The essence of this model is its system view of the problem and solution. It is useful to the systems engineer in determining that the needs are accurately and completely captured in the requirements, that every requirement is met by the functional architecture, that every function is allocated to the physical architecture and performed satisfactorily by it. Without that systems view the engineer cannot give the users the assurance that the system of interest will perform in ways that will solve the problem(s) they are seeking to solve.
If we divide this model into fragmented views or sets of data, we lose the system view, and with it, the ability to predict its quality and behavior. The temptation to fragment the system model into models that represent one aspect of the system of interest (e.g. – a requirements model) or a particular view or views comes from the use of the definition of a model as a limited representation of an underlying reality. In a hyper technical sense, a view or a physics based analytical model are “limited representations of the underlying system reality. But this definition must be applied with Box’s question of usefulness in mind. Without the system view that is critical to the system engineer’s purpose, the model is limited by its fragmentary nature in a way that makes it “wrong” to the point of not being useful. In other words, it cannot be the M in MBSE.
In order to provide this system view, the model must be constructed in a way that assures that all of the entities, their attributes and relationships are captured and made available for representation in a variety of ways. As Long puts it in his blog entry, the model, “. . . covers the space from concept of operations through requirements, behavior, physical architecture, and verification & validation. This is where innovation occurs. This is the one chance to manage complexity and address key systems characteristics such as resilience and security. This is what systems engineers represent with a broad set of representations tuned to the audience and the engineer – from IDEF0 to traditional structured representations to SysML and beyond.” Long, One Model to Coordinate Them All (2016). All this is in the service of the purpose of systems engineering.
The guarantor of a consistent and complete system model is the database schema or metamodel that organizes the model data to provide the engineer with the picture of the system of interest “from concept of operations through requirements, behavior, physical architecture, and verification & validation.” The metamodel provides the framework for the structure and relationships of the data that comprise the model. The framework also makes it possible for a robust modeling tool to examine and track the consistency and completeness of the model.
In their excellent whitepaper, Integrated Data as a Foundation of Systems Engineering (INCOSE 2018), the Requirements Working Group pointed out another important value of an organizing schema, “Having a master schema which other databases are consistent with is important from an interoperability, consistency, and data sharing perspective. Currently, various tool vendors define a proprietary schema for the data and information developed and managed within their tool’s database. This can make sharing data and information between tools problematic and often requires some sort of “middleware” to extract data from one database, transform the data and information to be consistent with the project’s master schema, and then load the data and information into the project’s sharable data base.” Integrated Data Sec 3.4.2 Note, INCOSE (2018)
The working group also noted the effects of a major pitfall in the effort to develop a common basis for organizing and sharing the model data. Encouraging the development of a profession-wide approach, they wrote, “Note that another part of the problem is standards proliferation by which multiple ontologies and exchange approaches are advocated. To make progress, the profession should step back and establish the common data model required to engineer any system (hopefully building upon work from the past such as AP-233). While that data model would require extensions based upon the specific domain and system of interest, it would establish the necessary core to reasonably underpin data interoperability and the practice of Systems Engineering.” Integrated Data Sec 4.7.2 Note, INCOSE (2018)
Fig. 1
As an example of a non-proprietary schema, Vitech’s metamodel was developed out of research done by TRW for the U.S. Army in the 1960s. It is a part of the public domain and throughout its life across the ensuing decades Vitech has maintained its public availability. It has been tested and refined through its use commercially since Vitech’s founding by David Long in 1992, and it remains publicly available as a basis for developing the common metamodel sought by the Requirements Working Group. Pictured above in Fig. 1, it is extensible and has been proven in decades of use across a variety of industry sectors.
The move to complete data integration and the collection and organization of the wide variety of data around the system lifecycle holds the promise of improving the quality and effectiveness of the systems engineering journey across the system lifecycle. But there are pitfalls that await us.
The first is that in the effort to capture catalog and manage data across not only the engineering aspects of the lifecycle, but including the project, program, and enterprise management and governance areas as well, we can easily lose sight of the necessity of providing a database schema that promotes the development of models that enable the systems engineering purpose.
In the process of capturing all of the data relevant to the system lifecycle we can succumb to the temptation to accept as models any representation or analytical model without subjecting their limitations to the scrutiny of George Box’s question, “how wrong do they have to be to not be useful” and its answer in the purpose to which the MBSE model is put.
Finally, we can fall prey to the idea that all that is needed for interoperability is the establishment of a common standard for the form of the data or the way it is represented and ignore the necessity of standardizing the structure of the database schema. This will leave us passing data that the destination tool can “read” but has no idea what to do with.
These pitfalls lurk in the temptation to do too much. While areas like project, program, and enterprise management impact the system lifecycle, they do not directly contribute data to the critical system model of systems engineering. The data associated with those areas should absolutely be catalogued, managed and preserved. But it should not be allowed to obscure or sacrifice the structure and content of the systems model. In that respect we should be careful not to allow the data integration effort to lose focus on the purpose of systems engineering, using a model to provide an assurance that we have designed “a quality product that meets the user needs.”
