In the beginning, there was the User Need.
According to FDA, the User Needs is the starting point for building a safe and efficient medical device. However, User Needs can be elusive to an engineering team used to rigid techniques and accustomed to having "full information" when approaching a problem.
It is rather rare that the developer has the real and deep experience a User has and it can therefore be precarious to leave the User Need elicitation process in the hands of an engineering team. Here are some tips for eliciting better User Needs.
Consider the Intended Use
It has been said 100 times before, but I say it again: start out by considering the Intended Use and Indication for Use.
These two items will tell you:
- who is the device for (or rather, who do you say the device is for)
- why (or for what) is the user using this device (or rather, for what you say the device is to be used for)
The whole idea here is to get as close as possible to the user and the situation in which the user applies the device. What medical condition is the device intended to address? Where and under which circumstances is device used? In a hospital or an ambulance? During the night? In the rain? By a child? By a blind person?
Answering these questions will put you in the shoes of the user and will let you describe the user's needs in his or her own words. Imagining being the user in the situation where the need for using the device arise will also give you an idea of the constraints facing the user at the time of application.
Thus, considering the "who" and the "why" is generally a more fruitful starting point than elaborating on the "how" (which is what engineers tend to do).
Who is the user?
In a majority of cases, a medical device is handled by more than one type of User during the products lify cycle. This becomes clear when considering non-core usage tasks such as:
How did the device arrive at the usage location? How and by whom was it deployed? Was it carried in a pocket? Was it transported by air?
Apparently, it is important to consider all people involved with the handling of the device since these actions may have safety implications. Again, do not second guess the needs of these users but involve them in the process.
User Needs can be vague
Using fuzzy language (such as adverbs) when documenting requirements is a known bad-practice. However, User Needs may be written in a less prescriptive way if it capture important aspects of the User and the Usage. We know that, in the end, the Medical Device will end up being very concrete and void of any fuzziness.
The challenge thus lies in translating the potentially fuzzy language used in User Needs into concrete specifications in the Design Input Requirements and concrete Validation Tests. Make sure that the User Needs are well understood by the team deriving the Design Input Requirements and Validation Tests from the User Needs.
Not all input is User Needs
Costs, brand colors, production constraints are examples of important design input that are not necessarilly User Needs. I usually recommend my clients to add an additional Design Control type for Stakeholder Needs in order to pick up this input which definitely influences the design, although not always being required to be verified or validated.
When setting up the validation activities, you must be explicit about what you intend to validate and explicitly define the criteria applied to make this selection. By separating the input in the two mentioned Design Control Types makes it easy to explain which Design Controls are intended for validation (User Needs) and which are not (Stakeholder Needs).
Write User Needs with Validation in mind
The way User Needs are written will heavily influence the Validation activities. Since Validation is a resource intensive (and therefore expensive) activity, it makes sense to keep a close eye on the prognosed validation work that will be derived by a User Need when writing it.
If some of the validation activities are already known at an early stage, the team can use this knowledge to cleverly formulate the User Needs in a way to maximize the coverage of the known validation activities. By this, I do not mean that important User Needs should be left out but rather that formulating and structuring the combined User Needs can have a positive or negative impact on the validation effort.
By following these simple guidelines, you should be able to get more bang for the buck next time you elicitate User Needs.
The medical device as a stand-alone product is a waning concept.
The desire to make use of the information collected in a medical device in other health systems, coupled with recent advancements in networks and interconnectivity has resulted in more devices being connected to the Internet. As a consequence, they have become more vulnerable to hackers.
A 2015 KPMG survey found that 81 percent of health care organizations had their data compromised within the previous two years.
FBI reports (FBI Cyber Division PIN (Private Industry Notification) #140408-010) that due to the transition of paper to electronic health records, lax cybersecurity standards and higher financial payout for medical records in the black market, the "cyber actors will likely increase cyber intrusions against health care systems".
During 2016, several ransom-attacks were launched against health institutions in the United States with wide ranging consequences.
FDA has shown heightened interest in cybersecurity issues and released three guidelines during the last two years. Medical Device manufacturers are likely to focus more on cybersecurity risk management activities in the near future and assign additional resources accordingly.
One integral part of the cyber security risk management process constitutes the risk assessment.
Performing risk assessments is a core activity in the medical device industry and many of the available techniques are well-known and well-used by industry professionals. Having long and thorough experience of risk management might lead the same professionals to believe that cybersecurity issues can be harnessed by the tools already at hand.
There are, however, key aspects where cybersecurity risks differ from traditional medical device risks.
Risk identification and the building blocks of a cybersecurity risk
The fundamental challenge during risk identification is to ensure that all relevant risks have been identified.
Verifying that this criterion has been fulfilled is often difficult and therefore structural help and practical advice is very welcome.
Due to the wide variety of possible medical device types, ISO 14971, the standard for medical device risk management, understandably has a hard time to define concrete identification techniques that are relevant enough to provide value for every kind of medical device.
IT risk management, operating in a narrower technical scope, provides a host of techniques, tailored to this domain. Many of these techniques are based around the asset-vulnerability-threat model.
These components can be described as followed.
- Assets are the entities a cyber-intruder attempts to access and control. An asset has a value to the patient and therefore also to an intruder. In a safety related context, we can exemplify assets as device configurations, health data, medical device functions or battery power.
- Vulnerabilities represents weaknesses in the medical device that can, when exploited, give an intruder access to an asset. Software bugs, application design flaws and insufficient input validation are examples of software vulnerabilities. But vulnerabilities can also be found in hardware, business processes, organizational structures and interpersonal communication.
- A threat is defined as an event with the potential to an adverse impact on an asset. The threat is executed by a threat agent (i.e. an intruder) exploiting a vulnerability in order to access an asset.
A combination of these building blocks describes a cybersecurity risk.
As a starting point, the manufacturer should be able to enumerate assets for the given device. By analyzing how these assets can be targeted, e.g. by performing a “Threat modelling analysis”, using a data flow analysis of the application, identifying "data-at-rest" (data storage) and "data-in-motion" (data transfer) will further help the manufacturer to identify vulnerabilities and threats, particular to the medical device.
Bottom-line: Identifying assets, threats and vulnerabilities will support the manufacturer when enumerating potential cybersecurity risks. The identified items should be listed in the risk management documentation.
What is the probability of a cyber attack?
The Medical Device industry takes its risk assessment cues from ISO 14971, which defines risk as the combination of the probability of the occurrence of harm and the severity of that harm.
The computer security industry, on the other hand, has used several kinds of assessment methods to estimate the "riskiness" of cybersecurity risks. None of them rely solely on probability and severity of an event. Instead these techniques estimate and/or quantify aspects of the assets, threats and vulnerabilities that in combination say something about the computer security risk.
So how can the medical device risk assessment methods concerned with the safety of patients be connect with techniques whose primary concern is information security?
In the AAMI paper "TIR 57 - Principles for medical device security - Risk Management", the authors address this discrepancy and attempt to transpose the cybersecurity line of thinking into the domain of ISO 14971.
It is easy to see how the "potential adverse effect" can be regarded as analogue to the “Harm” in ISO 14971 and be quantified with a corresponding severity.
The probability factor in ISO 14971 does not have a direct equivalent in the cybersecurity risk domain. A composite factor called "exploitability" combining characteristics of the vulnerability, the threat agent and the medical device itself is mentioned in the FDA guidelines. This factor intends to indicate the amount of work involved in order to invoke a successful attack.
The AAMI TIR57 group suggests a two pronged approach to establish something similar, combining two likelihood factors.
The first factor, the “Threat Likelihood”, defines the likelihood of the threat agent having the motivation, skills and resources to exploit a given vulnerability.
The second factor estimates the likelihood of harm being the effect of an exploited vulnerability. These two factors in combination makes out the probability of the cyber security risk.
Note that this approach is similar to the P1 (probability of the hazardous situation occurring) / P2 (probability of the hazardous situation causing harm) frequently used in the medical device community.
Information security theory recognizes that aspects and characteristics of the threat agent, the vulnerability and the device itself drive these factors.
Bottom-line: caution shall be taken when estimating probability/likelihood/exploitability for cyber security risks. The manufacturers Risk Management SOP shall address this concern accordingly and be adapted if necessary.
The drivers of cybersecurity risks
During classic medical device risk assessments, the identified causes leading to hazardous situations and potentially to harm are in most cases accidental. Although malicious use should be considered, this area often receives considerable less attention than harms caused by accidental events.
For cybersecurity risks, the opposite is true. Here, a significant factor in the causal chain constitutes the intentional behavior of a threat agent causing harm by exploiting a vulnerability.
Whereas the software flaw might have been created accidentally, exploiting it is an intentional act.
A malicious agent may come in many shapes and forms, such as a criminal organization, a competitor or disgruntled employee. Each of these has their own motivation, skill set and reach when it comes to detecting and exploiting vulnerabilities, which affects the likelihood of a vulnerability being exploited as we have seen above.
Therefore, not only needs the focus to be shifted from accidentally caused risks to intentionally caused risks. The manufacturer benefits from analyzing the characteristics of the malicious intruder in order to do a proper risk assessment.
Bottom line: Manufacturers will need to shift perspective from accidentally caused risks to explicit maliciously caused risks during the identification and classification of the risk assessment.
The poisoned SOUP
Just like in the rest of the software industry, medical device companies use third party libraries to increase productivity when developing medical devices.
These third party libraries (or frameworks or applications), sometimes referred to as "SOUP" components (Software of Unknown Provenance) are of course also subject to cybersecurity scrutiny.
It is debatable if third party components, and in particular open source components, are inherently more unsafe than proprietary produced applications (there are several arguments against such claims).
Regardless of this claim, it can be safely assumed that many of these libraries where not developed with a medical device cybersecurity context in mind. The security firm “Contrast Security” reports that 26% of downloaded SOUP:s contained known vulnerabilities and were still applied.
It shall be noted that SOUP:s themselves often rely on and integrate third party software, which increases the scope of the problem accordingly.
The medical device manufacturer is responsible for any vulnerabilities caused by SOUP:s in his device and must therefore analyses his SOUP:s accordingly.
This shall include a systematic inventory of SOUP:s, actively collecting information on current vulnerabilities in SOUP:s as well as applying timely updates of the SOUP:s when available.
Bottom-line: The manufacturer must have a clear plan for how to handle potential cybersecurity risks in SOUP:s.
Skills required to assess cybersecurity risks
Cybersecurity vulnerabilities are largely made up of software flaws. Understanding the technical details of how such vulnerabilities come into being, the technically influence they have and how they can be mitigated requires precise software knowledge. Including software developers into the assessment group is therefore a highly recommended measure.
It is a misconception that all software professionals are software security professionals. The majority of today’s information security problems can be traced to flaws in code. Many software developer lacks basic training and understanding of cyber security.
As already mentioned, the security scope for an interconnected medical device is much larger than the device itself. The network infrastructure in which connected medical devices operate are often outside the control of the medical device manufacturer but still has a great impact on the overall risk exposure and needs to be included in the risk assessment.
It is likewise a misconception that cybersecurity is a strictly technical field. Cybersecurity vulnerabilities are not exclusively found in hardware and software but also business process, organizational structures, human behavior and the environment in which the device operates. Thorough understanding of where, how and by whom the medical device will be operated is therefore important input for the risk assessment.
Last but not least, the clinical experts are required for estimating the potential harm of compromised availability, confidentiality and integrity of the associated data.
All these competences are required in order to perform a comprehensive cybersecurity risk assessment and it is clear that it stretches beyond being an internal R&D activity.
The risk assessment benefits from involving stakeholders beyond the immediate software development team. If specific cybersecurity expertise is lacking in the organization, the manufacturer should consider employing or train their own experts or outsource these functions to a competent partner.
Bottom-line: the knowledge required to perform a medical device cybersecurity risk assessment is both broader and deeper than what is often immediately available in many medical device companies.
The medical device industry has extensive experience with risk management and risk assessment techniques.
The cybersecurity dimension of medical device development is an additional aspect where risk assessments can enhance safety.
However, traditional medical device risk assessment techniques need adaptation in order to successfully be applied on cybersecurity risks.
Cybersecurity risks consist of other aspects and other drivers than "classic" medical device risks and are best mitigated by recognizing these differences.
That medical device development entails a lot of documentation should not be a surprise to anyone. Hundreds of documents are created, reviewed, released, then modified, reviewed and released again. The majority of these documents needs to be signed, often by two persons or more. Collecting signatures, although it seems like a trivial task, becomes a significant nuisance when the number of documents and releases increase.
One of our customers insisted on having 3 people sign each test case before release. Their 18 000 test cases yielded a combined signature collection effort of 5 man-years (estimated 30 mins to collect a single signature, their estimation).
It is rare to find medical device manufacturers that enjoy writing medical device development documents, but it is even rarer to find those who gladly spend their days collecting signatures for the said documents.
The obvious question is hence: how can we spend less time on document signatures?
For many medical device manufacturers, the equally obvious answer seems to be electronic (or digital) signatures.
So why is it so hard to get a signature?
There are several potential reasons for this. Maybe the Signer is a very busy person and simply has no time for this task. Maybe the Signer is located somewhere else through work, travelling, vacation or other reasons. Or perhaps the previous Signer did not pass on the document to the next Signer in line. Or maybe a formal signature sequence (order) is forced by the document process in question, where a Signer that actually is available, is prevented to carry out the task since some Signer further up the signing sequence has not fulfilled hers. Or it might be that the document in question cannot be signed before some other related document has been released (i.e. signed).
Thus, there can be formal reasons but also trivial reasons why a signature does not get timely collected.
The most trivial is of course that it is just hard to physically get the document in front of the Signer (or the other way around) for some reason or another. Once you get that far, the literary "stroke of the pen" is usually a quick affair. This is the perceived major efficiency benefit of Electronic Signatures. You do not have to physically get the document in front of the Signer. The document does not need to get passed around. The Signer can pull it up (from an E-Signing System) whenever he wants, from wherever he is. This allows a quasi-parallel execution of signatures. Two people on different sides of the planet can sign the same document at the same time (almost)! Costs associated with printing, sending, scanning and storing the paper copy are eliminated. E-Signatures also entail increased measures of security, enhanced authenticity, resisting tampering and also provide accurate signature audit trails.
Before explaining how to introduce an E-Signing System, let me say a few words about Digital and Electronic Signatures.
Digital and Electronic Signatures
Even though the terms are often used interchangeably, there are some notable differences between the two concepts.
According to FDA, Electronic Signatures are "Compilation of data (user name / password, dongles, biometric)", which is unique for a person. This can be used to sign documents and is as legally binding as a “wet signature”. The signature and the association with the signed entity (document) is stored in a database of the Signature System. Furthermore, not all E-Signing Systems leave a visual mark on the signed document that indicates that it has actually been signed.
Digital Signatures on the other hand, require a Digital Certificate that ensures the identity of the signer. A part of that Digital Certificate gets embedded in the signed document during the signing process. As a result, the validity of the signature can be checked independently of the E-Signing System.
So, someone needs to guarantee the identity of the signer.
For Electronic Signatures, the organisation (the manufacturer) does this by using the validates E-Signing System.
For Digital Signatures, it is the issuer of the Digital Certificate that ensures the identity of the signer. Digitally signed documents often also contain a visible signature.
Obviously, there seem to be several advantages using Digital Signatures. The validity of the signed document can be inspected independently of the E-Signing System, which is an advantage if the E-Signing System goes down, is corrupted or the system vendor goes bankrupt.
Yes, a few. Obtaining a Digital Certificate from a third-party vendor is expensive and requires an administrative effort. There is the obvious question of where and how to store these certificates as well as associate them with the user. They also have the nuisance to expire after a while and therefore need to get regularly renewed. People also have a tendency to marry and change their names etc. which also leads to renewals. Furthermore, it is not guaranteed that the validity shows up correctly in third party viewers (like Acrobat PDF Reader), for technical reasons having to do with root certificates.
An organisation can circumvent all this by issuing their own Digital Certificates. This is somewhat of an IT "adventure" but it can be done. Costs can then be lowered somewhat but there is still a significant administrative effort. Moreover, internally generated Digital Certificates can of course not be validated by third party viewers (like Acrobat PDF Reader).
So, there are pros and cons with both options.
However, they have several similarities and most important of all, both methods are recognized by the FDA.
Let's do E-SIgning!
Let's say we want to engage in eSigning (Electronic or Digital). What kind of effort can we expect to get this up and running?
Here is a short list of some of the steps:
- Assess the E-Signing System for Part 11 / Annex 11 compliance
- Qualify the E-Signing System Vendor as Supplier according to your QMS
- Assign responsible roles and people for the E-Signing System
- Install and configure the E-Signing System
- Make or buy the Digital Certificates (if used)
- Adapt your QMS to recognize E-Signatures and describe how they are intended to work
- Prepare all the Document Templates to be used for eSigning (the system needs to know where in the document the signature shall be placed. Page nr, location on page, margins and spaces etc).
- Validate the E-Signing System
- Create E-Signature User Guidelines and train all users in how to use it
- Notify the FDA (which is compulsory)
There is thus a non-neglectable initial effort to set up the E-Signing System, and also an effort to keep it maintained, both from a process as well as from an IT perspective.
There are also several other things to consider before you decide to go down the Electronic Signature path:
Document Life Cycle
All documents have a life cycle and the signing is only a very small part of this process. You need to consider how the document gets into the E-Signing System, how it interfaces with other systems such as Document Management Systems, workflow engines or e-Forms of which the document may be a part.
You also need to pay attention on how you plan to archive the electronic documents. This might seem like a trivial question but it is more depth here than you think.
If external users (as in external to your organisation) are going to use the system, you need to prepare a process where they get access to the E-Signing System, including setting up a corresponding user in the system with the appropriate Digital Certificate if applicable. These external users also need to get trained in how to use the system.
Hybrid Signature Situations
Are you going to end up with documents that are partly signed electronically and partly with traditional "wet signatures"? If so, you need a described process for this as well.
Last but not least you need to establish who has the ownership of the E-Signing System. Is it the IT Department that usually acquire and maintain IT systems? Or is it the R&D department that probably is the most frequent user of the system? Or is it the HR department that is concerned with the identity of the people working in the organisation? This needs to be clarified before you start.
As mentioned, an E-Signing System will decrease the effort of placing the document in front of the Signer. It will reduce costs associated with transporting the paper copy of the document. It will also potentially increase security and authenticity of the documents.
But there are things an E-Signing System cannot do. Regardless of how deep you entrench E-Signatures as a paradigm in your organisation, you will almost inevitable have a residual number of documents that are signed with ink. Thus, no matter how much you push E-Signatures you will end up with a hybrid system, composed by documents signed electronically and documents signed with ink. Be prepared for this.
Then, a Signature System is per se an IT system with all the work it entails. It needs to be validated and maintained, people will repeatedly forget their credentials if they do not use the system frequently and there will be the ubiquitous bugs and errors. This all means increased costs that need to be compared and contrasted with the costs by using a manual system.
Finally, an E-Signature System does not make bad processes good just by digitizing them. Overloaded employees will still remain overloaded regardless.
For which situations does E-Signatures make sense?
E-Signatures make sense when signing is a routine operation i.e. when a user makes several signings per week. E-signing for the occasional (or maybe even singular) CEO signature on a Product Requirements Document does not warrant the effort.
Document types that are well suited for E-Signatures are those that exist in many instances and that are comparably small amount of actual content (as in quick to read). Examples of such document types are time reports, expense reports, purchasing approvals and test case documents.
Last but not least, maintenance is of course made easier if all Signers are part of the organisation (as opposed to involving multiple external users).
Efficient with and without E-signatures
If you find it cumbersome to collect signatures today, there are several ways you can scrutinize your organisation for efficiency improvements.
Analyse current signing process
Are all these signatures really necessary? Ask yourself why they were added ("It is required by our process" is not a valid answer) and most importantly, what does the particular signature mean? In what way does a particular signature make the document "better"?
Don’t get dependent on busy Signers
The overloaded Project Manager or CTO that never has time for signing is a common bottleneck in many organisations. Appoint deputies to all signing functions (the deputies shall also have deputies). Try to avoid sequentially forced signature sequences. They cost more than they bring. Finally, simply planning the signing occasion like a regular meeting (set up a meeting in the calendar) might yield you some good results.
I hope this post has highlighted some of the pros and cons of employing Electronic Signatures. If there is anything I want you to take home it is probably this:
- Signature efficiency stands and falls with the process, not the system
- Analyse and improve the process first!!
- E-signatures can be very beneficial in specific situations
- E-signatures gains (i.e. speed gains) must be weighed against costs
Aligned Elements supports electronic as well as digital signatures of documents with automatic relaying to external Document Management Systems.
If you would like to get a demonstration of e-Signatures in Aligned Elements, just let us know.
A few years ago we could read articles claiming "Exoskeletons will one day replace wheelchairs". That the FDA regulates these kind of devices is a clear sign that they are about to enter the mass market.
Miniaturization of electronics, a steep drop in manufacturing costs of mechatronic parts, increased computational power, more powerful batteries. A number of recent technical developments converge into the realization of exoskeletons growing nearer and nearer.
In this rapidly expanding field, competitors are scrambling to put their new models onto the market.
In the beginning of 2015 FDA announced that power exoskeletons will be classified as Class II devices with Special Controls, which clarified the regulatory situation for these kinds of wearable devices. The special controls have been put in place to address a number of risk factors identified by FDA to be associated with the usage of powered exoskeletons.
For those used to medical device development, both the risks as well as the suggested requirements are straight forward. They include among other things, thermal and electric safety, biocompatibility and the obvious risk of falling to the ground.
To help exoskeleton manufacturers getting up and running with their documentation, we provided the Aligned Elements Exoskeleton Extension Package containing:
- 9 Potential Hazards (including mitigation strategies) outlined as exoskeleton associated risk factors by the FDA
- 38 exoskeleton Special Control Requirements deduced from 21 CFR Part 890 Docket No. FDA-2014-N-1903
This will give manufacturers a predefined starting point of setting up their Design History File with the intention of accelerating the documentation effort.
The Aligned Elements Exoskeleton Extension Package is downloadable here: Aligned Elements Exoskeleton Extension Package
On March 3rd, the 2016 revision of ISO 13485 was finally released. The new revision is essentially an evolution of the 2003 revision and includes a number of changes and clarifications.
For Aligned Elements users, a change in section 4.1.6 might have some important implications. Whereas ISO 13485:2003 did not explicitly require computer systems used in the quality management system to be validated, the 2016 edition certainly does. ISO 13485 is now finally on par with FDA 21 CFR 820 on this matter.
Computer Software Validation is used to ensure that each computer systems fulfills their intended purpose. It prevents problems with the software to reach the production environment. CSV is today used in many regulated industries and is today regarded as a good manufacturing practice.
Aligned Elements certainly fall into the category of Computer systems that must be validated according to ISO 13485:2016 and FDA 21 CFR 820. If you do not have CSV process in place, we do have some things that may help you.
Why is Aligned Elements not validated by Aligned AG?
Aligned Elements falls into the GAMP 5 Software category 4 - "Configurable Software". AE is a highly configurable software with the purpose of mapping the customers QMS, as opposed to force the customer to change his processes and templates to match Aligned Elements.
When validating a software of Category 4, it is of course the particular configuration of the software that is validated. Since each Aligned Elements customer is using a different configuration (each customer has its individual QMS), we cannot forsees which configuration our customers will use.
As mentioned, we do supply a number of useful tools to make the validation process faster.
What do I need in order to validate Aligned Elements?
Even though ISO 13485:2016 and FDA 21 CFR 820 require Computer Systems used in the Quality Management System to be validated, they do not explicitly described how to do it. This is up to each organisation to decide.
With no intention to be a complete listing, you need at least the following things:
- A Standard Operating Proceedure (SOP) that describes how Computer Software is validated in your organisation
- A validation plan, that describes:
- INTENDED USE of the computer system in question
- WHAT are you validating, i.e. which name, version and configuration of the software including manuals and supplier information and target environment
- WHY are you validating this software to this particular extent (hint: the GMP Software Categories are a good starting place. A risk-based approach is also viable starting point.)
- WHO is responsible for the software, for the maintenance of the software and who is responsible for the validation
- HOW do you intend to validated the software, what the acceptance criteria is and in particular how do you intend to handle software errors detected during the validation
- DELIVERABLES i.e. the documentation you intend to provide
- Requirements and/or use cases describing how you intend to use the Computer Software.
- Tests verifying that the use cases and documenting any deviations found.
- A Validation Summary or Report stating the result of the validation, preferably with some kind of traceability and weather the Computer Software should be cleared for use in the production environment
To make this process a bit simpler, we provide our customers with a pre-filled example validation kit of Aligned Elements that can be adapted to the individual needs of each organization.
Can Aligned Elements be used to document the validation of other systems?
Absolutely. Aligned Elements is very well equipped for documenting the validation of other systems (provided AE has been validated, of course). AE supports all steps of the CSV process, from evaluating the validation extent via checklists or using a risk based approach, documenting requirements, use cases and test cases, executing test cases, analysing the test results and finally supplying the necessary traceability reports.
Learn more about how Aligned Elements can help with achieving regulatory compliance
Request a live demo and let us show you how Aligned Elements can manage your documentation
Performing Medical Device Cybersecurity Risk Assessments is something we Medical Device Manufacturers must get used to. And the sooner the better, During 2016 and 2017 a mounting number of health associated cybersecurity incidents have been reported. Cybersecurity breaches may well become THE main safety concern in our industry within the next few years. Increased regulation on this matter is to be expected.
FDA has already published guidelines on its view on how medical device manufacturers ought to address cybersecurity in Medical Devices. The guidance outlines the documentation FDA expect to see in the premarket submissions as well as what is expected to be conducted for SOUPs and during postmarket activities.
At the core of this documentation lies the Cybersecurity Risk assessment. As already discussed, this type of risk assessment is slightly different to the typical Design Risk Assessment conducted during development.
To address this task, which many manufacturers will have to perform, we have developed a risk assessment template set specifically for documenting Cybersecurity risks and mitigations.
This template package is free to download and use for all Aligned Elements customers.
Are you interested in how the Cybersecurity Risk Assessment can be conducted and integrated with the rest of your Design Controls?
The Aligned Elements Cybersecurity Risk Assessment package contains:
- Risk assessment templates based on AAMI TIR 32, modelling Assets, Threats, Vulnerabilities and Risk Controls as Measures
- More than 30 Best Practice Cybersecurity Risk Mitigations ready to use
If you are looking for a Cybersecurity Risk Assessment Word Template, you can download an example here:
Cyber Security Risk Assessment Word Template
Not long ago I sat down and had a really good talk with Ilari Henrik Aegerter from House Of Test. Ilari has an extensive background in high-quality testing and is one of the leading opinion makers in the area of context-driven testing. Ilari is also a strong advocate of Exploratory Testing and I challenged him to explain how Exploratory Testing could fit into the world of strict and regulated medical device development.
"One way of thinking about testing is value creation. Value is created by increasing the knowledge about the system.” starts Ilari, “Through testing, you gather information about your system’s desirable and undesirable behavior. Now, some testing methodologies accomplish this more efficiently than others.”
"In traditional, scripted testing, the task is to verify that the system behaves according to a formally specified input, a.k.a "confirmatory checking". I see this approach a lot in the medical device industry, often accompanied by extensive documentation, meticulously describing the input and output.”
“The drawback of this approach is that it only traverses a very narrow path of the system. The generated results is equally meager when it comes to unveiling previously unknown knowledge about the system. It also abstains from tapping into the creativity and system expertise of the tester.
Let me make this clearer with an illustration.”
“This diagram describes a medical device. As you can see, we have three ways of looking at the product:
- The User Needs: this area represents the features set the customer actually wants.
- The formal Specifications: this area is made up of the Design Controls, where the manufacturer attempts to formally define the system he is planning to build.
- The actual Implementation: this final area represents the feature set the instrument actually supports
In the ideal case, a medical device is characterized by perfect intersection between all three areas.
Now, let's evaluate what happens when this is not the case.
"Specified but not implemented" - This area translates to turn up as failed tests in the medical device documentation, something that becomes accentuated in companies that focus on verifying formal specifications. Not uncommon in the medical device industry.
"Implemented and specified but not covered by User Needs" - Sometimes called "Gold Plating". From a scripted test perspective, this area is going to transform into passed test cases. One might question the value of implementing things that the Users don't need, but there might exist business reasons that explains this area.
"Implementation and User Needs intersect but are not covered by Specifications" - This is the result of an experienced developer who knows what the User Needs even though it is not specified. Unfortunately, this part is not going to be covered by the formal verification so we can only hope that the implementation actually works.
Finally, the most important area is represented by the "unmet User needs". This area is going to come back as bugs and change requests in the post-market phase. Despite having done the things right, we have apparently failed to do the right things. Some of these user needs might have been explicitly left out for cost reasons or with the intention to be implemented later. However, the critical part consists of the "things we didn't think about".
And voila, here is where exploratory testing can make a big difference. By applying a broader testing mindset, not being constrained by the narrow path of formal specifications, a wider test coverage is reached and more knowledge is obtained about the system. More knowledge is more value. The best part is that studies have shown that exploratory testing is much more efficient in finding bugs per time unit compared to traditional testing."
At this point, I asked Ilari if these unmet User Needs would not be uncovered during the Design Validation? Wasn't this exactly the objective of validation, to check that we have done the right things?
"Partly", says Ilari, "the notions are related but not the same."
“In recent years, we have seen an increased focus on usability and human factor engineering in medical device development. We have also seen an increased regulatory focus on performing proper validation for each new release. The whole point of these disciplines is to engage the user perspective, since the user probably knows more about the final usage than the manufacturer. Usability and human factor engineering are valuable tools to emphasize, charter and corroborate user needs in the design process.
Exploratory testing is focusing on similar issues. It leverages the creative and critical thinking of the tester, mimicking the thought process of a user. It challenges the tester as a domain expert and explicitly attempts to uncover tacit and unspecified needs."
I was still skeptical. "But, Ilari, we still need to be compliant with the FDA. We still have to show that the specifications are met and we still need to have documented proof. Exploratory testing makes a big point how traditional, scripted testing is obsessing about repeatability and documentation, indicating that these are not important steps."
"That is not true.", Ilari responds, "I think this is a misconception about exploratory testing, i.e. that it does not test specifications and does not produce documented evidence. Of course exploratory tests are documented. It's just that it might not use the same level of detail and that it better highlights undesired behavior, instead of only focusing on meticulously writing down everything that works as specified.”
“My opinion is that exploratory testing generates more knowledge about the system. This is particularly valuable during the early development stages. If you have worked in the industry, you know exactly what I am talking about. At this stage, the specs are constantly changing, the device is continuously modified, a lot of factors have not yet stabilized and it is simply not efficient to start writing formalized test scripts at this point. During these phase, exploratory testing is the superior testing method for generating knowledge about the current system state.”
“This idea is partly reflected in the FDA document “Design Considerations for Pivotal Clinical Investigations for Medical Devices - Guidance for Industry, Clinical Investigators, Institutional Review Boards and Food and Drug Administration Staff”. The document states that an exploratory studies in the early, and even pivotal stage of a clinical investigations, is advantegous, simply because it generates more knowledge about the system. In this line of thought, exploratory testing does not replace scripted testing, but certainly precedes and complements it.”
I must say that Ilari is building a case here. We leave the discussion at this point, both excited about the prospects and challenges of exploratory testing in a medical device context.
The Mobile Health Application market is glowing hot at the moment. New players are entering the market from left and right. Traditional blue chip players are moving in and new start-up's are popping up by the day. However, the medical device market is regulated market and finding out to which laws that apply for your mobile health app might be obvious. Start-up companies may not even realize that their apps are subjected by FDA regulations.
For those unsure about the legal status of their mobile health apps, the US Free Trade Comission has provided an excellent interactive online tool.
Take it for a spin right here!
Learn more about how Aligned Elements can help with achieving regulatory compliance for your app
Request a live demo and let us show you how Aligned Elements can manage your documentation for your app
You spend too much time and money on documentation.
So what is your plan?
The regulations of the medical device industry requires us to produce a pretty hefty chunk of documentation to show that the device is safe and efficient. If the documentation is not compliant, then it does not matter how safe, secure and performant the device itself it. Therefore, the documentation aspect that receives the largest share of attention is compliance. The most common path to a compliant stack of documents is to throw heaps of man-hours at the problem. Quantity seems to be the weapon of choice in many firms.
As a consequence, a large number of people gets involved in the documentation creation and maintenance, especially people residing the R&D part of the organization as most companies do not have Document Officers or documentation experts in their organization.
However, engineers and scientists are not necessarily the best writers (And they probably have no ambition to be.) Staff who are necessary for other tasks struggle to find the time to write tehse documents, and so the documents they do produce may be of lower quality than their usual work.
The effect on these people is often a suffocating feeling of inefficiency and a frustration of spending an un-proportionally large part of the working day on menial documentation tasks, deciphering SOP:S and unpractical standards to compile documents that no-one reads (apart from the auditor).
Our studies show that up to 30% of the total project effort is spent on documentation required by regulations. There is also an overwhelming consensus in the industry that this is a far too much. Money and time is inefficiently spent and morale buckles as the workload increases.
The good news is that these problems can be fixed. The bad news is that you are going to lose time, money and people until you fix them.
Excelling at documentation efficiency is not intuitive to many R&D-centric organization. However, considering the situation described above, good documentation practices is an investment. In the medical device industry, it is even a competitive dimension.
Your company can be efficient!!
Some good starting points are:
- Make documentation efficiency a prioritized objective using measurable goals
- Assign a responsible Manager
- Set up a tight collaboration between the people writing templates and SOPs (Quality people) and the people using the templates and SOPs (R&D people)
- Analyze your documentation processes
- Apply the right software tools to automate documentation tasks
You can start right away!
Download our Medical Device Documentation Self-assessment paper and take a few minutes to complete it. We assure you that you will have started the road to a more efficient documentation within the next 15 minutes!
With the 2015 update of the IEC 62366-1:2015 and the issuing of the FDA Human Factor Guidance, usability engineering in medical device development has received increased attention recently.
Human factor analysis gets more important with the rising trend of offering patient-centric solutions via new mobile health applications and wearables.
When patients, rather than specialists become the medical device user, increased focus needs to be placed on the patients capabilities and the environment in which the device is used. IEC 62366-1 is applied in an effort to increase patient and user safety by identifying, assessing and mitigating Use Errors, by paying attention to the usability of the device design and harness existing usability verification and validation methods to make sure that usability requirements are met and use errors are avoided.
For medical device manufacturers with limited previous experience in usability engineering, the task of implementing IEC 62366-1 might seem intimidating. However, the updated 2015 version of the standard has simplified and clarified the required process steps and tasks and Aligned Element now features a preconfigured setup that integrates the inputs, outputs and risk relevant elements of the usability process into the oveall Design Control traceability.
The configuration includes:
- 8 Design Control templates for capturing the input and output elements of the usability process
- Pre-configured content validation checks assessing the consistency of the project in real-time
- ISO 14971 compliant risk management to identify, assess and mitigate Use Error-driven risks
- Interactive Checklist for reviewing the Use Specifications Document against IEC 62366-1:2015
- 20 usability example risks for Use Errors during the Transport, Storage, Installation and Decommissioning
- 2 preconfigured Traceability Tables
- Integrated test protocols for established test methods such as:
- Cognitive Walkthrough
- Heuristic Evaluation (based on the Nielsen-Schneidermann heuristics)
- Simulated-Use Testing for Usability Validation
On top of all that, the Aligned Elements IEC 62366 includes all the standard features of Aligned Elements, including:
- Completed change control of all Design Control Items
- Complete audit trail for all changes
- Keep your company specific look and feel of all report outputs
- End-to-end tracability with real-time impact analysis
- Easy and fast Word reporting using drag and drop
- FDA QSR 21 CFR Part 11 compliant User Management
- Efficient decision-making process using workflows and E-signatures
By applying selected parts of the Aligned Elements IEC 62366-1 configuration to your Aligned Elements setup, you can efficiently leverage Aligned Elements in your IEC 62366-1 compliance effort.
The Aligned Elements IEC 62366 configuration is available for download in our extension library.