Implementation
The Implementation phase is where most of what we’re ordinarily tempted to think of as the “actual work” of the project takes place. However, as essential as the work done during this phase truly is, we’ve already seen how much work must be done to set it up for success – and we’ll see shortly how much must be done to capitalize on its success in order for the project to succeed overall. One aspect of the System Engineer’s job is to restrain the team’s temptation to view Implementation as the “real work” when the other phases are equally as important.
The System Engineer is able to play this role, in part, through resisting the impulse to roll up sleeves and plunge personally into the daily work of implementation. Instead, during this phase, the System Engineer:
- Helps the Project Manager with setting/adjusting project milestones and the details of sprint-to-sprint planning
- Oversees the implementation review process, including Static & Dynamic Analyses, Unit Testing, and Security Testing, and may personally conduct some or all technical reviews
- Provides compliance guidance on Implementation-specific Standards.
Finally, the Systems Engineer owns the final output of the phase, which is the Final Technical Review & Sign-off. This review is directed at answering three key questions:
- Did we do it?
- Did we build the solution we set out to build, as described by the User Needs and Requirements?
- Did we do it right?
- Was our process controlled? Are our outputs compliant? Are we on track to mitigate the risks and vulnerabilities we identified? Note: mitigations are not considered fully accounted for until they are addressed by the System Test Report, produced during the next phase (Integration).
- Did we do enough?
- Do we need to stop fiddling, tweaking, and improving, because the project is ready to ship? Or are we not fully feature-complete according to the agreed-upon plan, and do we need to adjust resources, methods, tools, and the project phase schedule itself to ensure we get there?
One important note here: depending on the device or component safety classification, traceability for artifacts produced during the Implementation phase is not always a “must” from a regulatory perspective. However, Velentium recommends producing traceable artifacts in every phase regardless of product classification, because it will make later phases (e.g., Verification) and submission package preparation MUCH faster and easier when the time comes.
Integration
Here’s where it all starts to come together. First, the System Engineer prepares a set of Reports to pave the way for Compliance Testing and Formal Design Verification. For software/firmware elements, these include test reports for Code Review, Unit Testing, and Integration Testing, during which units which have been exercised and proven to work in isolation are systematically integrated into more and more complex structures with other known-good units and exercised to confirm that they work correctly in integration as well as creating.
The Integration Test series culminates in System Testing, which follows a test method that the Systems Engineer helped design and describe in the project Test Plan (produced as part of the Design Phase). The output or artifact from System Testing is a System Test Report that traces back to the Risk Assessment and Security Assessment to prove mitigation efficacy. If there are any previously-identified risks or vulnerabilities not addressed by System Testing, the Systems Engineer must prepare an additional Anomaly Report to account for them.
From there, the system moves into Formal Design Verification. Design Verification demonstrates that the (tentatively) finished system meets all requirements. No matter the system’s safety classification, traceability is now a “must” – and you’ll be glad you saved yourself the trouble of not skipping over traceability during Implementation!
An integral part of Formal Design Verification is Compliance Testing, which demonstrates conformity to all applicable Standards. The System Engineer scopes out the standards which apply and ensures test coverage and may help coordinate the test schedule, although actual testing may be performed by a reputable 3rd party. Often, it’s desirable to involve a professional test house not simply because of their familiarity, tools, and resources with Standards-based testing, but also because they are a knowledgeable neutral party, able to examine the system with a fresh set of critical eyes.
The inputs, outputs, and tracing for Formal Design Verification are owned by the Systems Engineer. He or she is responsible not only for producing the required artifacts but for ensuring any lingering gaps between Requirements, Design, and Implementation are addressed.