In our previous posts, we looked at the importance of integrating human factors engineering (HFE) processes throughout product development. FDA approval for your device may require a human factors validation test (HFVT), which will demonstrate just how well you've applied HFE all along. You'll want to plan ahead for an HFVT early on, designing validation test parameters concurrently with product design and development.
Remember, the goal of HFE is to reduce hazardous risks from normal use and reasonably foreseeable misuse of your device through superior user-interface design.
When looking ahead to your HFVT, there are a couple of “big picture” concepts to keep in mind.
- The purpose of an HFVT is to demonstrate the success of your HFE.
By the time you run your final HFVT, you should have already done multiple rounds of design and use testing throughout development. You know your product is effective and safe, and that its design and interface encourages safe and correct use by intended users, for intended uses, under intended use conditions. Now it's time to demonstrate that.
Ideally, an HFVT should tell your team nothing about how users interact with your product that you don't already know. By the time you run an HFVT, your UI is in its final, market-ready form. You've designed your test to include all critical tasks, and you've clearly identified and described what “success” looks like for each test. This is where you prove your device is ready for the real world.
The process is both similar and dissimilar to an experienced lawyer cross-examining a witness. It is similar, in that the lawyer only asks questions to which she already knows the answers: she isn't cross-examining for her own benefit, but for the benefit of observers – the judge and jury.
But it's also dissimilar, in that the expert lawyer will only choose questions (and answers) that support the overall argument she is making, and that's not what HFVT does. For HFVT, you've got to design a robust set of tests that demonstrate your new medical device is ready for market, no matter what “questions” (i.e., normal uses and reasonably foreseeable misuses) it may be subjected to. That's where the second big-picture concept comes in:
- Testing should simulate real-world conditions in every way possible.
Device testers must be representative of anticipated actual users at every level, from highly trained medical staff familiar with a wide array of similar devices, to patients who may be encountering this kind of device for the first time. Just as you've got to know your users for HFE, you've got to know them for HFVT.
Demographics matter! Even nationality is important. Regulatory bodies like the FDA will want to see that your testers were of the same nationality as your intended users. Nationality differences can introduce unforeseen effects, due to differences in clinical practice, language use impacting labeling and training, common units of measure, and so on. If you intend to market your device to multiple countries, you should plan on a separate HFVT for each one. And best practices will dictate that you factor nationality-based differences into your HFE and UI designs all along.
Pre-test training should mirror anticipated actual training for your device. This should be true of conditions as well as content: test training should take place under the same conditions as device training would, and time should lapse between training and testing to simulate “knowledge decay” so you can evaluate training retention.
(If you've followed the FDA's HFE prescription of “safety through design” as Job #1, this should pose no problem! Remember, users are more likely to interact with the device based on their intuition rather than on your training).
Test environments should be as close to actual use environments as possible, and the testing process itself should be designed so as to bias or interfere with testers' interaction with your device as little as possible.
Lastly, your tests should include rare and unusual use scenarios not based on probability of occurrence, but based instead on the severity of potential consequences of rare and unusual usage. This is a general principle driving all HFE: we classify and deal with design-sourced hazards based on severity, rather than probability, because user behavior is so difficult to predict prior to testing. Even though your HFE developmental testing will help you predict normal usage behavior to a great extent, you should still include high-risk hazard testing during rare and unusual use scenarios in your HFVT.
There are some devices or use scenarios which cannot be simulated. In these cases, HFVT will have to take place under conditions of actual use. You'll need additional permission to run an actual-use test, and of course you will need to have run simulated tests as far as they can go to ensure that your device will be safe in actual use. And before you run an HFVT, whether simulated-use or actual, you'll need to submit a draft of your testing protocol to the FDA for pre-approval.
Human factors validation testing is the final phase of human factors engineering, and a major part of securing FDA approval for your device. Incorporating HFVT design into your product design-and-development process early prepares your team and your device to excel when test day comes.