O-PAS Factory Acceptance Testing: What Should an Open-System FAT Actually Include?

A project team arrives at Factory Acceptance Testing with six suppliers, multiple control platforms, a common communications architecture, and a test procedure developed around conventional DCS acceptance criteria. Every supplier has tested its equipment, the control applications are running, and the operator displays look correct. The integrated system completes the functional tests, the punch list is manageable, and the project prepares for shipment.

But nobody has demonstrated whether a component can be replaced with an equivalent product from another supplier. Nobody has tested whether a control application can be transferred between runtime environments, or whether the system recovers correctly when a multi-vendor interface fails. The project has proven that the installed system works under the conditions tested, but it has not established whether the architecture will deliver the flexibility the owner purchased.

That is the problem with applying traditional FAT methods to an open automation architecture.

Factory Acceptance Testing has always been about demonstrating that a system satisfies its specified requirements before it leaves the factory. Open Process Automation does not change that fundamental objective. What changes is the range of requirements that need to be demonstrated and the evidence the owner should expect to receive.

Traditional FAT proves functionality. O-PAS FAT must prove more.

In a conventional Distributed Control System project, FAT typically concentrates on control logic, operator interfaces, alarms, sequences, communications, system configuration, and other project-specific functional requirements. Testing is often conducted within an environment where the primary automation supplier controls most of the hardware, software, engineering tools, and integration infrastructure.

That model has served the industry for decades. It provides a structured way to identify problems before equipment arrives at the plant, reducing commissioning risk and giving owners confidence that the delivered system performs as specified.

An O-PAS implementation introduces another dimension. The architecture is intended to support interoperable components from multiple suppliers, with greater flexibility to introduce, upgrade, or replace technology over the system lifecycle. These capabilities are not automatically demonstrated by running the same functional tests used for a proprietary DCS.

A system can pass every control-loop and operator-display test while retaining significant dependencies on particular suppliers, engineering tools, interfaces, or runtime environments. Those dependencies may not affect initial operation, but they can become expensive when the owner attempts to modernize or replace part of the system.

The FAT strategy must therefore address two separate questions: Does the system perform its required functions, and has the project demonstrated the open-system capabilities it claims to provide?

Both deserve acceptance criteria.

The seven essential elements of an open-system FAT

There is no single universal FAT procedure suitable for every O-PAS implementation. Test scope must reflect the architecture, project requirements, criticality of the functions, and the components being delivered. However, seven categories provide a practical framework for developing a comprehensive acceptance strategy.

1. Architecture and interface verification

The first task is to establish that the delivered system matches the approved architecture. This includes identifying the participating components, their suppliers, relevant interfaces, communications paths, configuration baselines, and dependencies.

In an open architecture, this information becomes particularly important because the owner needs to understand where responsibilities and technical boundaries exist. An interface that appears straightforward on an architecture diagram may depend on configuration details, supporting services, or implementation choices that are not obvious from the drawing.

Testing should verify that the required interfaces are implemented and configured correctly, that expected information is exchanged, and that the resulting behavior matches the system design. Any claimed O-PAS conformance should be supported by appropriate evidence for the specific products, versions, and applicable requirements.

The deliverable should be more than a completed checklist. The owner needs an accurate record of the architecture that was actually tested.

2. Application portability testing

In our previous discussion of application portability, we examined the difference between declaring an application portable and demonstrating that it can actually be moved between environments. FAT provides an opportunity to put that distinction into practice.

Select a representative control application and attempt to transfer it from its original engineering and execution environment to another defined target environment. The test should identify the required source artifacts, libraries, configuration information, interfaces, and other dependencies.

Then measure the engineering work required to complete the transfer. Record changes to application logic, library substitutions, interface remapping, configuration modifications, and the time required to restore correct operation. Successful import or compilation is not sufficient; the transferred application must also pass the defined functional regression tests.

Not every O-PAS project will require unrestricted application portability across every available platform. The acceptance criteria should reflect the capabilities the project has specified and the target environments selected for testing.

What matters is that portability is demonstrated against a defined requirement rather than inferred from standards compliance alone.

3. Multi-vendor interoperability testing

The next category examines how independently supplied components behave when assembled into the complete control system.

A basic communications test establishes whether components can exchange information. A meaningful interoperability test goes further by verifying that the information is interpreted correctly and that the integrated functions operate as intended.

Consider a control function involving an application executing on one supplier’s platform, information supplied through another component, and operator interaction provided through a separate environment. The test should exercise the complete functional path rather than verifying each component independently.

Data quality, timestamps, state information, engineering units, interface behavior, and application dependencies may all influence the result. This is where a multi-vendor FAT becomes fundamentally different from a collection of supplier demonstrations.

The project must prove that the assembled components work together under the intended configuration.

4. Failure and recovery testing

Normal operation is generally the easiest condition to demonstrate. The more revealing tests often begin when a component or interface stops working.

A comprehensive FAT should include representative failure scenarios involving communications interruptions, component restarts, unavailable services, degraded information quality, and recovery after the original condition has been restored.

The expected response must be defined before the test begins. Depending on the function, that could involve maintaining the last valid value, transitioning to a defined operating state, generating an alarm, rejecting invalid information, or requiring operator intervention.

For functions with safety implications, the test must respect the relevant safety requirements and system boundaries. An O-PAS interoperability demonstration does not replace the independent verification required for safety instrumented functions or other protective systems.

The important question is whether the assembled system behaves predictably when something goes wrong, and whether recovery introduces any unexpected operating conditions.

5. Cybersecurity and access control verification

An open, multi-vendor architecture also requires a coordinated cybersecurity strategy. The presence of standardized interfaces does not eliminate security responsibilities, and introducing components from multiple suppliers can complicate identity management, configuration control, patching, and vulnerability management.

FAT should verify the security controls specified for the delivered system. Depending on project scope, these may include authentication, authorization, network segmentation, secure communications, logging, account management, and configuration integrity.

Testing should also examine the operational consequences of security-related changes. What happens when credentials are updated, for example? What happens when a certificate expires or a required security service becomes unavailable?

Cybersecurity acceptance should align with the owner’s risk assessment and applicable industrial cybersecurity requirements, including relevant IEC 62443 practices where specified. The objective is not to declare the system secure because a checklist has been completed. It is to establish documented evidence that the required security controls have been implemented and tested.

6. Component replacement testing

This is one of the most revealing tests an owner can request.

Suppose the architecture includes a component from Supplier A performing a defined function. During the test, that component is replaced with an alternative from Supplier B intended to satisfy the same requirements.

What happens?

Does the replacement preserve the required interfaces? Must application logic change? Are additional configuration tools needed? Does the replacement introduce new dependencies? Can the original functional tests be repeated successfully?

A replacement demonstration does not need to establish that every component is interchangeable with every possible alternative. It should test a defined substitution scenario that is meaningful to the owner’s lifecycle objectives.

The engineering effort should also be recorded, including modifications to surrounding components or applications. This provides a practical measurement of supplier-switching friction. If changing one component requires extensive redesign elsewhere in the system, the owner needs to understand that dependency before accepting the architecture.

7. Change management and regression testing

A successful FAT demonstrates the system’s condition at a particular point in time. It does not guarantee that the same behavior will be preserved after future changes.

Industrial control systems evolve continuously. Applications are modified, firmware is updated, security patches are introduced, and components are replaced. Each change can affect the behavior of interfaces and functions that depend on it.

An open-system FAT should therefore establish a repeatable regression-testing approach. The project should identify which tests must be repeated following different categories of change, what configuration baselines are required, and how test results will be recorded.

This is especially important in a multi-vendor environment because an update to one component may affect another supplier’s implementation.

The owner should leave FAT with more than evidence that the current system works. The project should also deliver the procedures and information required to verify that it continues to work as the architecture evolves.

Who owns the integrated FAT?

This is a question that needs an answer long before testing begins.

In a conventional DCS project, the primary automation supplier frequently carries substantial responsibility for coordinating the integrated system test. In a multi-vendor O-PAS project, that responsibility may be assigned to a system integrator, an engineering contractor, the owner, or another designated party.

What matters is that the responsibility is explicit.

Somebody must maintain the integrated test plan, coordinate supplier participation, control configuration baselines, manage discrepancies, and determine whether the complete system satisfies the acceptance criteria.

Without clear ownership, a failed interoperability test can quickly become a debate about which supplier is responsible for the interface. Supplier A may demonstrate that its component operates correctly, while Supplier B produces equivalent evidence for its own equipment. Neither result necessarily resolves the integrated system problem.

The project needs a designated integration authority with sufficient technical visibility and contractual authority to manage these boundaries. That responsibility should be established during procurement, not negotiated during FAT.

What should the owner receive when FAT is complete?

The final acceptance package should preserve the evidence needed to support the system throughout its operating life.

At a minimum, it should include the tested architecture and configuration baselines, interface definitions, approved test procedures, results, unresolved discrepancies, application dependencies, and the regression tests required for future changes. Where portability or component replacement has been demonstrated, the package should also document the engineering effort, modifications, tools, and procedures involved.

A simplified acceptance matrix might look like this:

These records are not simply documentation for project closeout. They become reference material when a supplier changes a product, a component approaches obsolescence, or the owner considers introducing new technology.

In a conventional project, FAT documentation often becomes something engineers consult when troubleshooting. In an open architecture, it should also support future engineering decisions about interoperability, replacement, and modernization.

FAT is where architectural promises become engineering evidence

Open Process Automation is intended to give industrial owners greater flexibility in how control systems are designed, supplied, maintained, and modernized. Standards and conformance provide essential foundations for that objective, but they do not eliminate the need for disciplined system engineering.

The real value of an open architecture emerges when the owner can make changes without unnecessarily rebuilding the surrounding system. That requires well-defined interfaces, portable application assets where specified, predictable failure behavior, controlled configurations, and repeatable acceptance procedures.

These capabilities cannot be established solely through product demonstrations or supplier declarations. They need to be incorporated into project requirements and verified through testing appropriate to the delivered architecture.

A successful O-PAS FAT should therefore answer more than whether the system is ready to ship. It should establish what the owner can reasonably expect to preserve, modify, and replace over the system lifecycle.

The most valuable outcome of FAT may not be proof that the system works today. It may be the evidence that allows the owner to change it tomorrow.

That is where the long-term economic value of Open Process Automation begins to become measurable.

For teams evaluating or implementing Open Process Automation, the free Introduction to Open Process Automation course and practical OPA/O-PAS resources at OPAcommunity.com are good starting points.

Next in the series: Who Owns System Integration in a Multi-Vendor O-PAS Architecture?