In our previous posts, we looked at the importance of integrating human factors engineering (HFE) processes throughout product development. FDA approval for your device may require a Human Factors Validation Test (HFVT), which will demonstrate just how well you've applied HFE all along. You'll want to plan ahead for an HFVT, ideally from project start, designing validation test parameters concurrently with product design and development.
Remember, the goal of HFE is to reduce hazardous risks from regular use and reasonably foreseeable misuse of your device through superior user-interface design.
When looking ahead to your HFVT, there are a couple of “big picture” concepts to keep in mind.
1. The purpose of HFVT is to demonstrate the success of your HFE.
By the time you run your HFVT, you should have already done multiple rounds of design and use testing throughout development. You have reason to believe your product effective and safe, and that its design and interface encourages safe and correct use by intended users, for intended purposes, under intended use conditions. Now it's time to validate that.
Ideally, HFVT should tell your team nothing about how users interact with your product that you don't already know. By the time you run an HFVT, your UI is in its final, market-ready form. You've designed your test to include all critical tasks, and you've clearly identified and described what “success” and “failure" looks like for each test. This is where you prove your device is ready for the real world.
The process is both similar and dissimilar to an experienced lawyer cross-examining a witness. It is similar, in that the lawyer only asks questions to which she already knows the answers: she isn't cross-examining for her own benefit, but for the benefit of observers – the judge and jury. HFVT is similarly done for the benefit of stakeholders and regulators: it should provide no results that surprise your design team, and must prove that your product will be used as you’ve designed.
But it's also dissimilar, in that the expert lawyer will only choose questions (and answers) that support the overall argument she is making, and that's not what HFVT does. For HFVT, you’ve got to design a robust set of tests that demonstrate your new medical device is ready for market, no matter what “questions” (i.e., typical uses and reasonably foreseeable misuses) it may be subjected to. That's where the second big-picture concept comes in:
2. Testing should simulate real-world conditions in every way possible.
Device testers must be representative of anticipated actual users at every level: from highly trained medical staff familiar with a wide array of similar devices to patients who may be encountering this kind of device for the first time.
Demographics matter! Even nationality is important. Regulatory bodies like the FDA will want to see that your testers were of the same nationality as your intended users. National background can introduce unforeseen effects, due to differences in clinical practice, language use impacting labeling and training, standard units of measure, and so on. If you intend to market your device to multiple countries, you should plan on a separate HFVT for each one. Best practices dictate that you factor nationality-based differences into your HFE and UI designs all along.
Pre-test training should mirror anticipated actual training for your device. This should be true of conditions as well as content: test training should take place under the same conditions as device training would, and time should lapse between training and testing to simulate “knowledge decay” so you can evaluate training retention.
(If you've followed the FDA's HFE prescription of “safety through design” as Job #1, this should pose no problem! Remember, users are more likely to interact with the device based on their intuition rather than on your training).
Test environments should be as close to actual use environments as possible, and the testing process itself should be designed so as to bias or interfere with testers' interaction with your device as little as possible.
Lastly, your tests should include rare and unusual use scenarios not based on probability of occurrence, but based instead on the severity of potential consequences of rare and unique usage. This is a general principle driving all HFE: we classify and deal with design-sourced hazards based on severity, rather than probability, because user behavior is so difficult to predict prior to testing. Even though your HFE developmental testing will help you predict normal usage behavior to a great extent, you should still include high-risk hazard testing during rare and unusual use scenarios in your HFVT.
There are some devices or use scenarios which cannot be simulated. In these cases, HFVT will have to take place under conditions of actual use. You'll need additional permission to run an actual-use test, and of course, you will need to have run simulated tests as far as they can go to ensure that your device will be safe in actual use. And before you execute an HFVT, whether simulated-use or actual, you'll need to submit a draft of your testing protocol to the FDA for pre-approval.
Human Factors Validation Testing is the final phase of Human Factors Engineering and a significant part of securing approval for your device. Anticipating HFVT early in your product design-and-development process prepares your team and your device to excel when test day comes.