Risk Assessments play a central role in Medical Device development. All medical device manufacturers apply risk management (they should because they have to!). All of them claim to be compliant with ISO 14971. And all of them do it differently.
I have worked with a large number of clients and I have seen more Risk Assessment variants than I can count. Some are good, some have, let's say, "potential".
From this experience, I can deduce a few best practices that will reduce the risk assessment effort considerably.
Here are my top five tips:
Don't brainstorm to identify risks
You are required to identify and assess ALL potential risks. How do you find them ALL? That can be a daunting question for someone new to the medical device industry.
However, the solution is to be structured i.e. to use a structured approach to systematically identify risks. There exists several known methods to do this, including:
- Task Analysis (analysing the use process)
- System Analysis (analysing the system through decomposition)
- Using the ISO 14971 annex questions
- Using existing risk reports of similar devices
Regardless of the approach selected, brainstorming should not be one of them. There are a number of well-known reasons for this, the most important one being that you will miss important risks.
Next time around, try a structured technique. You will identify more risks. I promise.
Use both top-down and bottom-up Risk Assessments
Some companies rely on EITHER bottom-up OR top-down risk assessment techniques and miss out on the fact that both approaches deliver vital and often DIFFERENT risks.
Top-down risk assessment techniques (such as PHAor Task Analysis) can be done early in the development process without much knowledge about the actual design of the device. It is a great tool for early identifying use errors and probably misuse.
Once the device design is known, the selected design itself must be analysed for risks (such as materials used, geometry, movements and energy emittance etc.) through a bottom-up risk assessment. FMEA'sare very popular and well designed for this purpose. Both these techniques complement each other and should be conducted by any serious medical device manufacturer.
Don't keep Design Controls and Risk Management in separate systems
Design drives risk. And Risk drives design. This will become apparent when you need to follow up on the implementation and verification of mitigations as well as the further analysis if mitigations introduce new risks. The glue between the design and the risks is the traceability. The effort of managing this traceability in a paper-based documentation system will be VERY high (those of you who have done it will nod now!).
So is applying software tools the solution? Not necessarilly, since proper traceability monitoring can not be done until the requirement management tool is integrated with the risk management tool (or vice versa). Only by automatically managing the traceability between the Risk Assessment Items and the Design Items, preferably in a single tool, can true trace monitoring be obtained.
Use reasonable probability and severity scales
I am glad to see a clear trend of tightening down the probability and severity scales during the risk evaluations. From previously having used up to 10 steps, the current trend tends towards five to six steps or less. People simply have a very hard time to judge whether a probability should be six or seven on a 1-10 scale and spend too much time pondering such questions. The option range is simply too large to be effective!
For the probability axis, I would like to endorse Dr. Johner approach of having each step representing 2 orders of magnitude. He explains this very well by saying, that apart from such a an approach lets the probability axis span over more than 8 order of magnitudes, "...the factor 100 indicates the precision which we can appreciate... If you ask a group of people, how long it takes (on average) for a hard disk to be defective, the estimates vary between 2 years and 10 years. But everyone realizes that this average is greater than one month and less than 10 years. And between these two values is about a factor of 100."
Make use of existing mitigations
In many cases, the risk assessment is carried out when the design is already known. In such cases: when coming up with mitigations for your identified risks, use the already existing mitigations in your current design!
I bet your current design already contains a whole bunch of design decisions that are risk mitigations without you really considering them as such. The absolute majority of design teams I have encountered are very, very good in designing innovative and safe devices. However, many of the design decisions taken are based on previous experience, industry state-of-the-art, or simply old habits having been refined over time. Since these engineer are often better designers than document writers, they simply do not see their design (often already in place) through the lens of risk management.
Bottom line: your current design already contains of an uncovered treasure of existing mitigations. Try to use your existing design as mitigations when performing your next risk assessment.
Risk Identification is an early and essential part of the risk management process and ISO 14971 requires us to make a complete risk assessment, to identify ALL hazards.
But, how do we know if all of the hazards have been identified? How can we prove this?
You could brainstorm or have a white board session gathering ideas that pop up, but the only way to truly achieve confidence in your risk identification process is by using a structured approach.
There are several techniques available depending on the assessed source, including:
- Assessing established potential hazards from internal records or published standards
- Analysis of the manufacturer's experience with similar medical devices
- Conducting a User Task Analysis on the user’s interaction with the device to uncover use errors
- Assessing Field data and published incidents from similar devices in use
- Assessing critical components for safe an effective use
Because of the difficulty involved with thoroughly identifying all of the hazards, ISO 14971 provides a number of aides – such as Annex C (2012) (becoming the ISO 24791 Annex A in the 2019 edition) – which provide a list of questions to assist in establishing device characteristics that may impact safety. Although not exhaustive, these question can serve as starting point and become of one of several potential approaches from which the complete risk identification can be assembled.
Aligned Elements users can kick start their risk identification process by downloading and importing our ISO 14971:2012 Annex C Extension, assessing them and start generating risks and mitigation.
The ISO 14971:2012 Annex C Extension contains:
- RVT file for an ISO 14971 Annex C Question and a corresponding DOCX Reporting style template
- 37 importable questions built on Annex C in ISO 14971 to assess and integrate in your Risk Assessment
This Extension facilitates the assessment of the questions, the creation of both an automated assessment report of the Annex C questions as well as a starting point for generating new risks and mitigation.
It gives medical device manufacturers a predefined starting point when setting up their technical file with the intention of accelerating the documentation effort.
The user is of course welcome to expand this question list with questions that are particular for his/her device and the conditions under which it needs to operate.
The ISO 14971:2012 package can be combined with other risk identification packages from Aligned or in-house developed approaches by the manufacturer.
The ISO 14971:2012 Annex C package is free to Aligned Elements users.
Performing Medical Device Cybersecurity Risk Assessments is something we Medical Device Manufacturers must get used to. And the sooner the better, During 2016 and 2017 a mounting number of health associated cybersecurity incidents have been reported. Cybersecurity breaches may well become THE main safety concern in our industry within the next few years. Increased regulation on this matter is to be expected.
FDA has already published guidelines on its view on how medical device manufacturers ought to address cybersecurity in Medical Devices. The guidance outlines the documentation FDA expect to see in the premarket submissions as well as what is expected to be conducted for SOUPs and during postmarket activities.
At the core of this documentation lies the Cybersecurity Risk assessment. As already discussed, this type of risk assessment is slightly different to the typical Design Risk Assessment conducted during development.
To address this task, which many manufacturers will have to perform, we have developed a risk assessment template set specifically for documenting Cybersecurity risks and mitigations.
This template package is free to download and use for all Aligned Elements customers.
Are you interested in how the Cybersecurity Risk Assessment can be conducted and integrated with the rest of your Design Controls?
The Aligned Elements Cybersecurity Risk Assessment package contains:
- Risk assessment templates based on AAMI TIR 32, modelling Assets, Threats, Vulnerabilities and Risk Controls as Measures
- More than 30 Best Practice Cybersecurity Risk Mitigations ready to use
If you are looking for a Cybersecurity Risk Assessment Word Template, you can download an example here:
Cyber Security Risk Assessment Word Template
A smoother and faster IEC/ 60606-1 experience with Aligned Elements IEC/ISO 60601-1 Risk Assessment Checklist
The IEC/ISO 60601-1 "Medical electrical equipment" is the cornerstone document addressing many of the risks associated with electrical medical equipment.
The standard covers safety and performance requirements of medical electrical equipment and public health authorities in many countries recognize it as a pre-requisite for the market access. The standard is notorious for its depth and complexity and many manufacturers experience the task of ensuring compliance as challenging.
The safety testing, certification and global market access approvals done for IEC 60601-1 shall be conducted by an accredited Testing Lab. The manufacturer's collaboration with the Testing Lab is essential for a smooth and swift approval.
As of the 3rd edition of IEC 60601-1, a large number of risk management references were introduced in the standard. The Test Laboratory will request the manufacturer to demonstrate how the product's risk assessment covers the risks items stipulated in IEC 60601-1.
Poor preparation of this step can result in can delay the certification process, requiring an inordinate amount of time during the initial testing phase to correct the risk management files.
To facilitate this step, Aligned has developed an integrated assessment method in Aligned Elements, that assists the manufacturer in demonstrating compliance with these risks.
By assessing and connecting the IEC 60601-1 risk requirements with the product risk assessment already existing in Aligned Elements, a compliance assessment document can be automatically generated and presented to the Testing Lab.
Aligned Elements IEC/ISO 60601-1 Risk Assessment Checklist - How is it done
- The checklist contains approx. 80 risk checklist items from IEC 60601-1 which are imported into Aligned Elements.
- Each IEC 60601-1 risk checklist item contains the clause reference, the demonstration requirement, explanations and examples of what the risk intends to cover, all to facilitate the identification of the corresponding risk in the manufacturers risk assessment, already located in Aligned Elements.
- The manufacturer addresses each risk requirement, deeming it either as "not applicable" for his product (including providing a qualification for the answer), or applicable and then tracing the risk requirement to the corresponding existing risks in his own risk assessment.
- When completed, a Compliance Assessment Word Report is generated and can be presented to the Test Laboratory.
- With this compliance report, the Test Laboratory representative can quickly assess your IEC 60601-1 risk related complicance level
The benefit of the Aligned Elements IEC/ISO 60601-1 Risk Assessment Checklist is a massive reduction of time spent at the Testing Lab by leveraging your existing documentation!
How we developed the Aligned Elements IEC/ISO 60601-1 Risk Assessment Checklist
The Aligned Element IEC 60601-1 Risk Assessment Checklist has been developed in collaboration with former Eurofins Electrosuisse Test Laboratory Manager Karim Bader, currently serving in the swiss national working group CES/TK 62 for "Elektrische Apparate in medizinischer Anwendung" contributing to the development of the international standard IEC 60601-1.
Experts from Aligned will assist you in integrating the checklist into your current configuration and demonstrate its use.
If there is a need to further explain the IEC 60601-1 risk management requirements and identify findings that can be fixed, Mr. Karim Bader, an expert in this field is available to deliver the knowledge and confidence to ensure that your product will be certified without delay.
Whenever we come across risk management SOPs from different companies, we always keep an eye open for how that company solved the requirement 7.4 in ISO 14971 to perform a Risk vs Benefit Analysis as well as the handling of Residual Risks.
In excel-based approaches it is not uncommon to see something similar to the table below:
The QMS expects that each risk (representing one row) should be addressed in the spreadsheet. When I see this, I personally feel inclined to ask:
- What happens if someone in the team does not agree that the risk is acceptable, claims that the risk does not outweigh the benefits or simply cannot decide?
In many cases these extra columns made it into the template after an audit of the QMS to quickly fix any audit findings, a somewhat unfortunate result of increasing regulations.
Let’s look at these requirements in a bit more detail.
Is the risk acceptable?
Answering this question is very much like shooting from the hip. Either the answer is trivial, e.g. the risk is frequent and sever, in which case we should definitely control the risk as per procedure. The other complex scenario is that it very much depends on a lot of different factors. The complex scenario is only possible to answer in the scope of the benefit of the device. Still we often come across it as an individual item in the risk analysis.
The solution is straightforward
The principle of lowering risks as much as possible should be applied and when all possibilities for applying risk measures have been looked at, the company needs to do a proper Risk vs Benefit Assessment for the complete device.
Hint: The risk management report is a good location for this.
One could imagine that looking at an isolated function and weighing the risk towards the benefit of that function may give us an insight to if the function is acceptable for the device. In most cases, a function cannot be handled as an isolated part of the system, nor can the benefits of that function be easily compared to the risks it may impose to a user, patient or operator. E.g. Trying to argument that a power unit may impose risks to a patient but having the benefit that the device needs electricity to work is not a meaningful exercise. ISO 14971 (2019) is exceptionally clear in this matter:
Good reasoning and sensible pros and cons are asked for
Here we recommend to look at the Device as a whole. Will the discovered residual risks still make it beneficial in the scope of the intended use of the device? In case of doubt, look at trying to apply additional Risk Control Measures to open risks. Remember that all identified risks acceptable or not are considered residual risks for the device. See ISO 14971 (2019) Section 6:
Once again:The risk management report is a good location for this.
Claiming that there are no residual risks involved is in our opinion not possible to handle in a spreadsheet column. The questions cannot simply be answered with a yes or no. Let me be a bit provocative and suggest that this question alone could replace any risk analysis method altogether. Something like this:
Although I’ve seen similar approaches at very large established medical device manufacturers, I do not recommend this approach!
Here is what ISO 14971 (2019) has to say about it:
Bring the analysis to completion
Here we need to use our toolset properly and link the risk control measures to any implementing functions and continue with a new loop of risk analysis.
The outcome of that task will answer if there are still any residual risks present. This exercise may need to be repeated for any suggested risk control measures.
Finally, summarize all your findings in the risk management report and do not forget to remove these columns from your risk analysis templates!
Have you ever struggled to describe Hazardous Situations so it was clear to all stakeholders what you intended to say?
Did you spend a lot of time to come up with concisely written Sequence of Events and then the first person to review your document claims to not understand what you intended to convey?
When describing your harms, have you ever wished that someone had put together a list of all possible harms, so you could just pick the one, which is applicable for this particular situation?
And then after your product release, did a Risk occur that you did not foresee?
Common Terminology by curtesy of the IMDRF
If you have ever experienced one or more of the above, there might be some help out there. The International Medical Device Regulators Forum (http://www.imdrf.org) has created a document called IMDRF terminologies for categorized Adverse Event Reporting (AER): terms, terminology structure and codes.
Although that is quite a mouthful, this document can make your life a lot easier. It provides an extensive list of possible medical device problems, possible harms and related causes. Each term is assigned a code, which have to used when creating a Manufacturer Incident Report as required by the MDR (https://ec.europa.eu/docsroom/documents/41681).
These codes can also be used when reporting Adverse Events to the FDA by means of a Medical Device Report (https://www.fda.gov/medical-devices/mandatory-reporting-requirements-manufacturers-importers-and-device-user-facilities/mdr-adverse-event-codes).
Is the terminology only applicable for post-market events?
Although the terms compiled by the IMDRF have a strong focus on Post Market incidents, they are also useful in your pre-market design risk assessments. When performing your ISO 14971 compliant Risk Analysis during the development phase, a lot of time is (and should be) spent on the risk identification process to make sure all potential risks have been assessed and addressed.
In practice, this requires writing down and assess the hazardous situations, what causes and subsequent harms that could possibly arise by using your product. However, these are essentially the same as in a post market scenario. Using the lists provided by IMDRF can speed up this process significantly.
So how does this make things easier for me?
The IMDRF lists act as an acceleration vehicle for your Risk Analysis. By using and analysing these established terms, you will save a significant amount of time when documenting all possible hazardous situations, causes and harms. At the same time the likelihood of overlooking a particular hazardous situation, cause or harm is greatly reduced.
Furthermore, ambiguities are reduced by using and referring to an established set of risk terminologies. Thus, you reduce the risk that other stakeholders, not just your colleagues, but also the auditors, will not misunderstand your carefully constructed Risk Analysis.
Using the IMDRF terminology in Aligned Elements
The lists are applicable to Aligned Elements projects using Risk Assessments using the Preliminary Hazard Analysis method.
It is possible to import the IMDRF items directly into Aligned Elements by using four import packages which you can download here.
The extension consists of lists containing a Design Control type called “IMDRF Item”, which have the attributes “Code” and “Definition”.
When importing them, you will need to map the types to types that exist in your configuration.
Note that the lists contains a large number of items which may not all be applicable to your particular device.
A pre-assessment step of the list content is therefore recommended before applying them into production projects.
The following mappings should be done.
- “Annex A, Medical Device Problems” (469 items) should be mapped to a type which represents “Potential Hazards” in your configuration.
- “Annex D, Investigation Conclusions” (35 items) should be mapped to a type which represents “Causes” in your configuration.
- “Annex E, Health Effects - Clinical Signs and Symptoms or Conditions” (797 items) should be mapped to a type which represents “Harms” in your configuration.
- “Annex F, Health Effects - Health Impacts” (64 items) should be mapped to a type which represents “Harms” in your configuration.