Proving Due Diligence for FDA Audits
VectorCAST automated testing tools are successfully used by clients demonstrating due diligence with FDA software quality requirements. Whether your device is Class I, II or III, the VectorCAST tools provide a dependable and repeatable testing process for your medical device software development. And the VectorCAST coverage and test case pass/fail reports are exportable to regular HTML or text for easy integration into product documentation delivered to the FDA.
FDA Developer Testing Levels
|FDA Developer Testing Levels||Satisfied with VectorCAST and DT10|
|Unit (module or component) level testing focuses on the early examination of sub-program functionality and ensures that functionality not visible at the system level is examined by testing. Unit testing ensures that quality software units are furnished for integration into the finished software product.||
|Integration level testing focuses on the transfer of data and control across a program's internal and external interfaces. External interfaces are those with other software (including operating system software), system hardware, and the users and can be described as communications links.||
|System level testing demonstrates that all specified functionality exists and that the software product is trustworthy. This testing verifies the as-built program's functionality and performance with respect to the requirements for the software product as exhibited on the specified operating platform(s).||
According to the FDA General Principles of Software Validation Guidance:
|All production and/or quality system software, even if purchased off-the-shelf, should have documented requirements that fully define its intended use, and information against which testing results and other evidence can be compared, to show that the software is validated for its intended use.|
Vector Software has developed an extensive software medical device certification kit VectorCAST to meet FDA and IEC 62304:2006 standards.
Further, the FDA document provides a recommended approach to Testing by the Software Developer in section 5.2.5.The various VectorCAST test tools address the software testing referred to in this section and provide an automated capability to satisfy each of the document's requirements.
Section 5.2.5: Testing by the Software Developer
Software testing entails running software products under known conditions with defined inputs and documented outcomes that can be compared to their predefined expectations.
Code Coverage Analysis
The level of structural testing can be evaluated using metrics that are designed to show what percentage of the software structure has been evaluated during structural testing. These metrics are typically referred to as "coverage" and are a measure of completeness with respect to test selection criteria. The amount of structural coverage should be commensurate with the level of risk posed by the software.
Regression analysis and testing are employed to provide assurance a change did not create problems elsewhere in the software product. Regression testing is the rerunning of test cases that a program has previously executed correctly and comparing the current result to the previous result in order to detect unintended effects of a software change.
Test reports should comply with the requirements of the corresponding test plans.
Improving Code Quality While Complying with IEC 62304
Any company that wants to sell a medical device in Europe needs to comply with the IEC 62304 standard.The standard provides a list of tasks and activities that support the safe design and maintenance of medical device software. The goal of this is to ensure the software does what is intended without causing any unacceptable risks. Therefore, IEC 62304 relies heavily on Risk Management strategies.
There are several ways that VectorCAST and DT10 tools can increase code quality and therefore reduce risk, regardless of whether you developing Class A, B or C medical devices:
VectorCAST/Cover provides standalone code coverage analysis during system testing to identify not only what areas of the application are being executed during system testing but more importantly, which areas are not being executed. Those areas have a higher risk of failure. A typical risk mitigation strategy is to increase the tests being performed on the software to achieve 100% code coverage.
VectorCAST/C++ allows engineers to set up both low level unit tests, which test a single function, as well as higher level software module integration tests, which test a logical grouping of functions. These tests consist of specific inputs to stimulate a function and the expected results indicating successful execution of the code based on the inputs. These tests can be run natively, on a target simulator, or on live target hardware. VectorCAST provides pass/fail analysis to show which expected results were achieved, meaning a passing test case, and which were not, a failing test case. Passing 100% of the tests leads to a higher quality product, and when combined with the code coverage metrics that VectorCAST/C++ also offers, can reduce the risk of failure for code that achieves 100% coverage.
The other area where risk is introduced is when the software changes and the changes are not thoroughly analyzed as to their impact on the overal software application. Regression analysis and testing are employed to provide assurance a change did not create problems elsewhere in the software product. Regression testing is the rerunning of test cases that a program has previously executed correctly and comparing the current result to the previous result in order to detect unintended effects of a software change. With VectorCAST/Manage, regression testing is automated and can be set up as frequently as you want, including nightly.
DT10's real-time target trace debug allows further risk mitigation by providing developers & testers the birds eye view of the whole system execution. The unique dynamic testing approach allows developers & testers to uncover hard-to-reproduce and hard-to-detect defects. This real-time testing on target provides the necessary means to meet compliance requirement.
Commercial Test Tool Validation
Software testing tools are frequently used to ensure consistency, thoroughness, and efficiency in the testing of such software products and to fulfill the requirements of the planned testing activities. These tools may include supporting software built in-house to facilitate unit (module) testing and subsequent integration testing (e.g., drivers and stubs) as well as commercial software testing tools. Such tools should have a degree of quality no less than the software product they are used to develop. Appropriate documentation providing evidence of the validation of these software tools for their intended use should be maintained, known as "Intended Use Validation."
FDA and IEC 62304 Compliance
VectorCAST provides tool validation in the form of qualification documents. The qualification documents consist of Tool Operational Requirements (TOR) and Tool Qualification Data (TQD). TOR describes requirements for VectorCAST tools and TQD, the associated tests and test results. These documents are built for each project and the tests execute on the same compiler/target/runtime environment used by the customer.
The VectorCAST Medical Device Software Compliance Kit for FDA and IEC 62304 includes all necessary documentation for validating embedded software components used on all classes of medical devices.
The kit includes test procedures, plans, coverage and test case reports necessary for Intended Use Validation, and to demonstrate due diligence with FDA and IEC 62304 software quality requirements.