What is interoperability? How do you measure and then test it? Interoperability is a difficult concept to understand and test. First, we have to define interoperability. Interoperability is the connectedness of systems and components to provide end-to-end operational effectiveness. But how is that interconnectedness defined within the system? The best way to understand it is through architecture.
Architectures define the way systems fit together like puzzle pieces. Think of your house. There are many layers of systems working together, such as the plumbing, heating and electricity sub-systems to make the house as a whole operate as one system. Each of those sub-systems is a layer, or view, of the entire house architecture. The places where the pieces of the architecture components fit together to interact are called the interfaces within the architecture. The interfaces must work in certain ways to perform the overall mission, and this is the crux of interoperability.
The interfaces must have well-defined functions for how the coupling between the components perform to ensure that the overall functions of the system work correctly. Think of it as a series of messages along a telephone system. If the message breaks down or is corrupted by one part of the system interfaces, then the message will be jumbled at the other end and the interoperability will be corrupted. To keep the interfaces working correctly, most interfaces follow a standard for their operations. The standard defines how the interface will perform its operations to function between the multiple components in the same way every time. For example, a power wall outlet is an interface, and its operation is defined by electrical codes or standards to ensure it performs its function correctly.
An important aspect of interoperability is that it comes in the form of both internal and external system interfaces. Internal interfaces are those that connect component to component and sub-system to sub-system. External interfaces connect one system to another and a system to the infrastructure. External components are more difficult to work with because, unlike the usual internal interfaces, they are not controlled by one system program. This leads to imposing greater workloads on understanding who defines those external interfaces and who has responsibility for them in terms of configuration management and control. This greater workload is also seen in the testing of the interfaces because there are more entities and programs to work with to ensure that the interfaces perform effectively. To accomplish this for external interfaces, we must ensure that we are involved in the design, development, testing, control and configuration management of those interfaces to the greatest extent possible.
To ensure that interfaces are well-defined in their operations, we must shift the development of the interfaces as far left in the acquisition development timeline as possible, allowing for early development (prior to Milestone B) of interface definition and standards in conjunction with development of the requirements. Additionally, interoperability requirements for network interfaces must be defined in accordance with the mandatory Net-Ready (NR) Key Performance Parameter (KPP) that defines the way network interfaces will operate for the system’s performance.
There are three main attributes to the NR KPP. They are: Support to Military Operations, Entered and Managed on the Network, and Effectively Exchanges Information. The NR KPP is an interesting KPP because it is not defined by a standard set of rules but rather is developed in accordance with the way the system’s network infrastructure is expected to perform the mission. So the NR KPP must be defined very early in the requirements development process to ensure the network is designed and developed with well-defined and controlled interfaces for network performance. Those interfaces will form the backbone of the system and how it interoperates. The performance of the system will be built on top of that infrastructure architecture to perform its mission with well-defined interoperability across the network interfaces. So, how do we test or verify that the system is interoperable? We have interoperability requirements in terms of the NR KPP and other technical interface requirements. But how do we verify that they perform correctly?
The Department of Defense Instruction (DoDI) 5000.02 states that “program managers will design, develop, test and evaluate systems to ensure IT [information technology] interoperability requirements are achieved.” Additionally, DoDI 8330.01 states: “All IT, including defense acquisition and procurement programs and enterprise services, must have a net ready key performance parameter (NR KPP) as part of its interoperability requirements documentation. The NR KPP consists of measurable and testable performance measures and metrics derived from associated DoD architectures, and is used to assess both the technical exchange of information, data, and services, and the end-to-end operational effectiveness of those exchanges.” This ensures that the requirements exist for interoperability across network interfaces and testing or that verification is performed against the requirements.
The Joint Interoperability Test Command (JITC) is the lead Operational Test Agency for interoperability and testing against the NR KPP. The best way to get involved with JITC to develop test plans and procedures is to work with JITC early in the requirements development process to ensure that the requirements developed are verifiable. JITC should then continue to be involved in the development and testing of the system for interoperability across the network.
To perform effective testing for interoperability, developmental and operational testing should be performed across the interfaces and from end to end to ensure mission effectiveness. The testing should follow the requirements set forth in the NR KPP as well as any other interoperability requirements and standards developed for each interface in the system architecture. Testing should be performed in accordance with the associated standard and requirements for each interface as soon as possible in development to ensure problems are identified and fixed early rather than later. Many problems within interface performance can cause major problems for the total system performance as well as ripple effects throughout the system across components. It is important to root out these issues early in the development life cycle.
Metrics and Measures of Performance should be established for each interface and controlled through an associated standard or requirement. They will help establish correct operation of the system across each interface and will ensure that testing is performed against a performance standard or requirement.
Ultimately, testing of interfaces for interoperability as well as testing end-to-end performance can be a time-consuming and daunting task. It is important that JITC be involved up front and early and that developmental rigor are followed in the systems engineering processes to ensure good development and management of the interfaces throughout requirements generation, development and testing. If developmental processes, requirements and standards are designed early and followed throughout the development timeline, the interfaces will be well defined and controlled throughout the life cycle. This will ensure that testing is performed early, consistently, continuously and rigorously to ensure that there is a well-defined and managed interface control scheme throughout development and testing of the system and its architecture.