Christian Kaestner at Qadvis, is an expert company offering quality and regulatory services for medtech companies, and co-author of the upcoming standard "IEC 82304-1: Health Software - General Requirements for product safety." We were fortunate to get a few minutes of his time to answer five quick questions about the standard and its implications.
What is IEC 82304 all about?
"The purpose of the standard is to establish product requirements for standalone software, i.e. software products not running on a dedicated hardware. IEC 82304-1 could be seen as an equivalent to IEC 60601-1 with the difference that the later includes a dedicated hardware for the software to run on. (And of course have a lot of hardware requirements!)"
But isn't this already covered by IEC 62304?
"No, IEC 62304 is a process standard and as such “only” defines expectations on activities and their corresponding outputs. The key scope of IEC62304 is the lower part of the traditional V-model while IEC 82304-1 takes the complete product lifecycle into account. This means that IEC 82304-1 establish product requirements which isn’t addressed by IEC 62304, examples:
- Product requirement, like intended use and accompanying documents.
- Product validation
- Decommissioning of software (and its data!)
IEC 62304 is of course referenced by IEC 82304-1 for the development of the actual software."
Which companies are affected by IEC 82304? How can I find out if my company is affected?
"The scope of the standard is health software which is a broader term than medical device software. So, I would suggest any company working with medical device software (standalone) or software in the surrounding of medical device products to have a look at the standard. As a rule of thumb; you could say if your company develops software which might have an impact on the health/wellbeing of a human, I would say you are affected.
Below some examples which I believe should be included in the scope of health software:
- Prescription management systems - If the data is mixed up in anyway it may result wrong doses or even wrong medication to a patient.
- App keeping track of your medication (as one example) - If you use the app to avoid forgetting you medication or by mistake take it twice. This would of course not be good follow the instructions and the instructions are wrong.
- Software tracking “safe periods” to avoid pregnancy - Assuming the software counts wrong and gives green light when it shouldn’t… I can imagine the surprise in a month or so isn’t very popular!"
What is your advice to companies that needs to implement IEC 82304?
"To be fair; all standards are voluntary so there is no requirement or need to comply with IEC 82304-1. But as for all other standards it is a good praxis and gives your company a quality mark otherwise hard to claim.
My advice to companies with regards to IEC 82304-1 is to take advantage of it; it will support you in what is required to make standalone software to a product."
When will the standard be released and where can I get a copy?
"So far, the standard isn’t published, it is currently out for review (DIS, draft international standard). Depending on the result of the voting, a FDIS may be ready Q2 2016. Usually the difference between a FDIS and the final version should be marginal. The final version can probably be expected late Q3 2016. (But, it all depends on the results from the voting process.)
Should you be interested already today, you can purchase a copy of the draft standard at www.iso.org."
Learn more about how Aligned Elements can help you support IEC 62304
Request a live demo and let us show you how Aligned Elements can help you to comply to IEC 62304
In the beginning, there was the User Need.
According to FDA, the User Needs is the starting point for building a safe and efficient medical device. However, User Needs can be elusive to an engineering team used to rigid techniques and accustomed to having "full information" when approaching a problem.
It is rather rare that the developer has the real and deep experience a User has and it can therefore be precarious to leave the User Need elicitation process in the hands of an engineering team. Here are some tips for eliciting better User Needs.
Consider the Intended Use
It has been said 100 times before, but I say it again: start out by considering the Intended Use and Indication for Use.
These two items will tell you:
- who is the device for (or rather, who do you say the device is for)
- why (or for what) is the user using this device (or rather, for what you say the device is to be used for)
The whole idea here is to get as close as possible to the user and the situation in which the user applies the device. What medical condition is the device intended to address? Where and under which circumstances is device used? In a hospital or an ambulance? During the night? In the rain? By a child? By a blind person?
Answering these questions will put you in the shoes of the user and will let you describe the user's needs in his or her own words. Imagining being the user in the situation where the need for using the device arise will also give you an idea of the constraints facing the user at the time of application.
Thus, considering the "who" and the "why" is generally a more fruitful starting point than elaborating on the "how" (which is what engineers tend to do).
Who is the user?
In a majority of cases, a medical device is handled by more than one type of User during the products lify cycle. This becomes clear when considering non-core usage tasks such as:
How did the device arrive at the usage location? How and by whom was it deployed? Was it carried in a pocket? Was it transported by air?
Apparently, it is important to consider all people involved with the handling of the device since these actions may have safety implications. Again, do not second guess the needs of these users but involve them in the process.
User Needs can be vague
Using fuzzy language (such as adverbs) when documenting requirements is a known bad-practice. However, User Needs may be written in a less prescriptive way if it capture important aspects of the User and the Usage. We know that, in the end, the Medical Device will end up being very concrete and void of any fuzziness.
The challenge thus lies in translating the potentially fuzzy language used in User Needs into concrete specifications in the Design Input Requirements and concrete Validation Tests. Make sure that the User Needs are well understood by the team deriving the Design Input Requirements and Validation Tests from the User Needs.
Not all input is User Needs
Costs, brand colors, production constraints are examples of important design input that are not necessarilly User Needs. I usually recommend my clients to add an additional Design Control type for Stakeholder Needs in order to pick up this input which definitely influences the design, although not always being required to be verified or validated.
When setting up the validation activities, you must be explicit about what you intend to validate and explicitly define the criteria applied to make this selection. By separating the input in the two mentioned Design Control Types makes it easy to explain which Design Controls are intended for validation (User Needs) and which are not (Stakeholder Needs).
Write User Needs with Validation in mind
The way User Needs are written will heavily influence the Validation activities. Since Validation is a resource intensive (and therefore expensive) activity, it makes sense to keep a close eye on the prognosed validation work that will be derived by a User Need when writing it.
If some of the validation activities are already known at an early stage, the team can use this knowledge to cleverly formulate the User Needs in a way to maximize the coverage of the known validation activities. By this, I do not mean that important User Needs should be left out but rather that formulating and structuring the combined User Needs can have a positive or negative impact on the validation effort.
By following these simple guidelines, you should be able to get more bang for the buck next time you elicitate User Needs.
On March 3rd, the 2016 revision of ISO 13485 was finally released. The new revision is essentially an evolution of the 2003 revision and includes a number of changes and clarifications.
For Aligned Elements users, a change in section 4.1.6 might have some important implications. Whereas ISO 13485:2003 did not explicitly require computer systems used in the quality management system to be validated, the 2016 edition certainly does. ISO 13485 is now finally on par with FDA 21 CFR 820 on this matter.
Computer Software Validation is used to ensure that each computer systems fulfills their intended purpose. It prevents problems with the software to reach the production environment. CSV is today used in many regulated industries and is today regarded as a good manufacturing practice.
Aligned Elements certainly fall into the category of Computer systems that must be validated according to ISO 13485:2016 and FDA 21 CFR 820. If you do not have CSV process in place, we do have some things that may help you.
Why is Aligned Elements not validated by Aligned AG?
Aligned Elements falls into the GAMP 5 Software category 4 - "Configurable Software". AE is a highly configurable software with the purpose of mapping the customers QMS, as opposed to force the customer to change his processes and templates to match Aligned Elements.
When validating a software of Category 4, it is of course the particular configuration of the software that is validated. Since each Aligned Elements customer is using a different configuration (each customer has its individual QMS), we cannot forsees which configuration our customers will use.
As mentioned, we do supply a number of useful tools to make the validation process faster.
What do I need in order to validate Aligned Elements?
Even though ISO 13485:2016 and FDA 21 CFR 820 require Computer Systems used in the Quality Management System to be validated, they do not explicitly described how to do it. This is up to each organisation to decide.
With no intention to be a complete listing, you need at least the following things:
- A Standard Operating Proceedure (SOP) that describes how Computer Software is validated in your organisation
- A validation plan, that describes:
- INTENDED USE of the computer system in question
- WHAT are you validating, i.e. which name, version and configuration of the software including manuals and supplier information and target environment
- WHY are you validating this software to this particular extent (hint: the GMP Software Categories are a good starting place. A risk-based approach is also viable starting point.)
- WHO is responsible for the software, for the maintenance of the software and who is responsible for the validation
- HOW do you intend to validated the software, what the acceptance criteria is and in particular how do you intend to handle software errors detected during the validation
- DELIVERABLES i.e. the documentation you intend to provide
- Requirements and/or use cases describing how you intend to use the Computer Software.
- Tests verifying that the use cases and documenting any deviations found.
- A Validation Summary or Report stating the result of the validation, preferably with some kind of traceability and weather the Computer Software should be cleared for use in the production environment
To make this process a bit simpler, we provide our customers with a pre-filled example validation kit of Aligned Elements that can be adapted to the individual needs of each organization.
Can Aligned Elements be used to document the validation of other systems?
Absolutely. Aligned Elements is very well equipped for documenting the validation of other systems (provided AE has been validated, of course). AE supports all steps of the CSV process, from evaluating the validation extent via checklists or using a risk based approach, documenting requirements, use cases and test cases, executing test cases, analysing the test results and finally supplying the necessary traceability reports.
Learn more about how Aligned Elements can help with achieving regulatory compliance
Request a live demo and let us show you how Aligned Elements can manage your documentation
"Validation" is one of those terms that can be confusing for people new to medical device development.
This word, "validation", is used in (at least) three different contexts that are not necessarily related.
- Design Validation
- Process Validation
- Computer Software / System Validation
Let's take a look at each one of these in turn.
What is it?
The regulatory lingo (in this example from FDA 21 CFR 820.3) for "Design Validation" declares that it is about “establishing by objective evidence that device specifications conform with user needs and intended use(s).”
It is a mandatory step in the medical device development process and if you are developing a medical device you will have to perform it at some point.
Why is it done?
Design Validation is about proving that we have "built the right thing". This is often contrasted with Design Verification which is meant to prove that we "built the thing right".
The difference lies in that the latter checks that the device was designed/built according to specifications (which is very important), whereas the former checks that the device works as it is intended to work for the end user, where the intention is documented in the User Needs and to some extent in the intended use.
As you can see, there regulations have foreseen that there might be a discrepancy between the intention and the specifications derived from the intention when developing the device.
It might be the case that the designers have not solved the user needs in a way that makes the device usable.
"Anecdata" provides endless examples of devices that are built "correctly" according to specs but turns out to be unusable / dangerous when used by the end users in a real environment e.g. a glucose monitor alarm is not loud enough to be heard when travelling on a noisy bus.
Does this all sound like "usability" and "human factor analysis" activities to you? Then you are exactly right. Design Validation bears many similarities with usability activities. However, Design Validation also includes clinical tests in order to establish that clinical results corresponds to expected levels are under intended use.
What is the input for the activity?
How do you know what to validate? Simply put, your User Needs. Therefore, it is important to define the User Needs in a manner that makes them possible to validate in a practical way.
Hence, the next time you write down your User Need, think hard about how you plan to validate them.
When is it done?
Design Validation is usually performed at the end of the design process, often after the Design Verification has been completed but before the medical device is transferred to production.
The instances of your medical device used in the validation should ideally exist in production-equivalent state.
Who is involved?
The Design Validation tests are generally performed by real users (doctors, nurses, patients etc.), preferably in the actual intended environment (in the hospital, in the lab, in the ambulance, at home etc.). The idea is to get as close as possible to "real use by real users".
Engaging this type of personnel can be both complicated and costly. It is therefore important to plan ahead and secure necessary resources (location, personnel etc.) well in advance.
If the validation reveals serious flaws in the device, which requires re-design, sever consequences on development time-lines and costs are at stake.
It is therefore important to involve real users throughout the design process and not only at the end.
How is it documented?
Like all other Design Control stages, the Design Validation needs to be planned, which is documented in a Design Validation Plan, and then executed, which is often documented in validation test cases and results. The summary and outcome is documented in the Design Validation Report.
Traceability, consistency and completion are of course important steps at this stage. Therefore, Aligned Elements is a great tool to use for these activities.
Process Validation / Production Validation
What is it?
Process Validation most often means validation of your manufacturing process or parts thereof (although many other processes can be subject of this validation as well).
Or in FDA lingo, "process validation means establishing by objective evidence that a process consistently produces a result or product meeting its predetermined specifications."
Why is it done?
So why is this important? FDA requires (21 CFR 820.75) the "results of the process" ("results" = your medical device, "process" = your manufacturing process) to be "fully verified by subsequent inspection and test".
This is a way to say that each unit of your produced medical device must undergo "full" final testing and inspection and thereby we ensure that this particular instance of the medical device is safe for the user to use.
Now, there are cases when it is not possible to verify a safety relevant characteristic of the device using testing and inspections, with the typical example of sterile devices as prime contender.
To verify a device being sterile, one would need to break the sterilization barrier (open the packaging) which would of course make the device non-sterile and therefore unusable/unsellable.
So when it is not possible to "fully" inspect and test the device, you can resort to process validation.
Thus, you employ process validation to check that the production process gives your medical device that particular, safety relevant characteristic every time for every device.
What is the input for the activity?
Although ISO 13485 and FDA QSR 820:75 both lists process validation as something very important, they are both a bit elusive on exactly which parts to validate. Basically, all the characteristics that cannot be fully covered in the final inspection are eligible to process validation.
For more information, the go-to document for Process Validation, although published in 2004, is the GHTF SG 3 NB 99:10:2004.
When is it done?
Generally, process validation is a pre-production activity, meaning that the production line is fully operational but you have not yet started production "for real".
Who is involved?
Just like for Computer Software Validation, the legal manufacturer is the responsible party. However, it is not uncommon to outsource parts of the work 3rd party supplier or OEM manufacturers.
How is it documented?
The "GHTF SG 3 NB 99:10:2004" goes a long way in describing the recommended documents. Very broadly speaking, there ought to be a Process Validation Plan, sections describing IQ, OQ and PQ activities, and finally a Process Validation Report.
FDA explicitly requires the following from the validation, which ought to be properly documented:
- All activities which have been carried out must be recorded, including date and signature.
- Procedures, with which process parameters are shrivelled, must be established.
- Only qualified personnel may validate a process.
- Methods and data used for controlling and monitoring processes, the date of execution, persons carrying out the validation, as well as relevant equipment must be documented.
- In case of changes, the manufacturer must assess whether re-validation is necessary and must carry it out if needed.
Computer Software / System Validation
What is it?
This type of validation, often shortened as "CSV", concerns "data processing systems are used as part of production or the quality system" (from FDA QSR 820.70) or, as freely interpreted from ISO 13485:2016, all computerized systems being used in any of the processes regulated by the QM system.
Thus, it does not concern the software in your device. Rather it refers to computer systems used to design and produce your medical device. Aligned Elements is an example of such a system. Other examples include Excel Spreadsheets, e-QMS systems, measuring equipment in production and so forth.
Why is it done?
A multitude of regulations (FDA 820, FDA CFR 20 Part 11, ISO 13485, GAMP5) recognizes that patients/users are potentially exposed to safety risks if the computer systems employed in the design and production process of the device does not behave as intended.
Computer Software Validation is the suggested remedy to this perceived risk.
What is the input for the activity?
The input for the CSV is the User Requirements that should, as one might thing, specify what the software under test can do, but rather what the organization plans/intends/want to do with it.
If you plan to use Excel for calculating a tolerance, then you the User Requirement should state exactly that.
It is recommended to take a risk-based approach to the testing and focus on the User Requirements that render the most severe risk impact if not adequately fulfilled.
So what about the validation of the software used in your device? Since you are then validating the design of your device, go check the Design Validation section.
When is it done?
CSV should be performed on the target computer system before it is put in use.
Who is involved?
The responsible party getting the CSV activities planned and executed is clearly the organization that uses the computer system. However, the actual leg work is sometimes outsourced to 3rd party suppliers. Note that it is not mandatory for end users to be involved in the actual testing.
How is it documented?
How the organization performs CSV shall be defined in an a company SOP. The SOP shall define which computer system types that falls under CSV, how to plan and execute the CSV activities, and under which criteria a re-validation is applicable.
In its most basic form, a CSV plan should state the User Requirements which captures the intended use of the systems. The User Requirements are then tested in IQ, OQ and PQs and the results of these tests and, if applicable, deviations are summarized in the Validation Report.
As we can see, although these three types of Validation do not have a lot to do with each other, they are very important to Medical Device manufacturers.
A common trait for all activity groups is that they need be properly documented, often following the same type of scheme, starting out with a plan document stipulating the "things to check for" (requirements), a number of test protocols to be executed, collection of objective evidence, traceability and report generation.
Aligned Elements is a perfect tool to structure and manage these type of validation activities and manufacturers can therefore benefit from Aligned Elements in all tree areas.
For a free trial of Aligned Elements, start here.
Computer Software Validation is something we discuss a lot where I work. Since Aligned Elements is a Medical Device Quality Management System relevant software, our customers make sure that their Aligned Elements installations and configurations are validated.
It is a necessary activity, although not always perceived as bringing an awful lot of value. Sometimes the cost of CSV activities actually supersedes the cost of acquiring the software itself.
I sometimes get the question if I know any best practices when it comes to Computer Software Validation and I have made a few observations from my CSV experiences.
Let me tell you about the top 3 things that has an impact on the Computer Software Validation effort:
1) How many test cases do you perform?
It is dead simple. More test cases (note: not necessarily "more requirements") is more work. But do YOU have to perform all those test cases? What if the supplier has already verified them?
You are very much allowed to leverage existing supplier documentation and testing records. A properly conducted Supplier / Vendor assessment can lead you to the conclusion that the suppliers verification documentation suffice. Remember that the risk we are trying to mitigate with a Computer Software Validation effort is primarily about patient safety. A software like Aligned Elements is not directly involved in the patient safety and this fact can be leveraged when assessing and deciding on the validation scope.
2) How do you record the test results?
The bulk of the CSV work resides in performing and recording the test results. The way you record those results can have a significant impact on the overall effort. In order to record that the actual behavior of a test step corresponds to the expected behavior, is it enough to tick a box (passed/failed)? Or do you need to write a text? Or do you need to make screenshots? As you can imagine, there is a huge difference between the former and the latter.
"But are we not required to take screen shots?". The short answer is "No, you don't". Not if you do not think it proves anything more than the tester checking a box. FDA requires you to select your own methods and tools for quality assurance. If you have a good case for not making screen shots (which I think you have), you do not have to.
3) Who (and how many) has to sign all these CSV documents?
This might sound a bit odd but more than once I have run into cases where the validation is completed and everything that is missing is a signature from some top management figure. And now we run into a buy-in problem. If this guy has not been involved CSV approach and suddenly disagrees with how it was conducted ("BUT THERE ARE NO SCREENSHOTS?!?"), it can have a significant impact (significant as in "redo the validation").
So the lesson here is to get early buy-in from the people that sign the document. On a general level, reducing the number of signatures will speed up any documentation process. And you might want to contemplate on the necessity of having the IQ, OQ and PQ plans / reports in different documents (more documents to sign) or if you can combine them.
Validating Aligned Elements
What's up ahead regarding Computer Software Validation?
FDA announced that their much anticipated "Computer Software Assurance for Manufacturing, Operations, and Quality System Software" draft guidance should be out in 2019 but now it seems like it has been postponed to 2020. The new guidance is supposed to use a more agile approach, including a risk-based and value creating perspective on CSV activities.
"Dexcom has announced a massive and serious continuous glucose monitors recall, involving more than 260,000 devices with alarms that may fail to sound."
Not hearing the alarm of a medical device is not a good thing. When should the manufacturer have picked up on this? During the usability validation at the latest.
Underestimating usability engineering and proper usability validation can have serious consequences (obviously). Still, usability engineering is still in many places regarded as an outsider practice by many "regular" engineers, considering it being a discipline about "taste". I assure you that this is not the case.
There is a tremendous potential in applying usabaility engineering during product development. Standards covering usability and human factors design such as the IEC 62366-1:2015 address the potential risks of poor usability design. However, being compliant to theses standards will not only make the medical device safer. They also makes the product easier to use and thereby creates more value to the users and patients.
To make the documentation of the usability engineering tasks faster and easier, we have added a preconfigured IEC 62366-1:2015 setup to Aligned Elements.
Test managers often have to deal with superhuman juggling of timelines, resource allocations and continuously changing specification while facing increasing pressure from management as the shipping-date draws closer. Furthermore, the test manager is responsible for making sure that the traceability is completed, that test data integrity remains intact and that change management procedures are followed correctly, even under situations of extreme stress.
Efficient change management, planning, tracking and reuse of test executions are therefore much appreciated tools in the V&V Managers tool box. The Aligned Elements Test Runs aims to address these challenges. The Test Runs manages the planning, allocation, execution and tracking of test activities. Let's dive into the details.
Planning the Test Run
The Test Run Planning section is the place to define the Test Run context, much as you would write a Verification Plan.
The Test Run Planning information includes:
- What to test i.e. information the Object Under Test and the addressed configurations
- When to perform the tests i.e. the planned start and end date for the Test Run
- Who participates in the test effort i.e. the team of testers and their allocated work
- How to test the device i.e. which test strategy to use by selecting among the existing test types
- Why the test is being done i.e. the purpose or reason for performing this particular set of tests on this Object Under Test
Allocate existing Test Cases from your Test Case library to the Test Run by using simple drag-and-drop. You can use any number of tests of any available types from any number of projects. If needed, add Test Run specific information to the Test Cases and designate the Test Cases to the members of the Test Team.
Once the planning phase is completed, the Test Execution can begin.
The Test Execution phase
Test Case data is kept separate from the Test Execution result data, permitting a high degree of test case reuse and a clear separation between Test Instructions and Test Results.
Consistency checks are optionally carried out on the Test Case before execution in order to ensure that tests cannot be performed until all foreseen process steps are completed.
During execution, the test input data is optionally kept read-only, preventing testers to modify a reviewed and released Test Case during execution.
All Test Team Members can access the Test Run and simultaneously Execute Tests as well as continuously monitor how the Execution phase progresses.
Real-Time Test Execution data is presented through:
- Individual Test Execution results including any found defects as well as colour coded feedback on states
- Colour coded test progression statistics, with possibility to drill down on e.g. individual Testers or Test Types
- Burn down charts, showing how planned Test progress over time corresponds to the actual progression
During Test Execution, Defects and Anomalies can be created and traced on-the-fly without having to leave the Test Execution context. The Defects can be tracked in either Aligned Elements internal Issue Management system or already existing integrated Issue Trackers such as Jira, TFS, GitHub, Trac or Countersoft Gemini or any mix of these systems. Created Defects and their corresponding status are displayed in the Test Run.
Test Case Change Management
When developing medical devices, it is of paramount importance to keep your Design Control under tight change control.
The Test Run assist the testers and test managers in several ways to accomplish this goal, including optional actions:
- Preventing inconsistent tests from being executed
- Preventing input data to get modified during test execution
- Allowing Test Managers to lock/freeze Design Input and Tests during execution
- Alert testers when attempting to modify tests for which valid results already exist
- Signalize weather a Test Case has been reviewed and released or not
- Allowing the user to explicitly invalidate existing test results when a test is updated with major changes
Testing Variants of the Object Under Test
If several variants of the Object Under Test exist, it is sometimes desirable to create variant specific test results for common test cases and subsequently create separate traceabilities for the variants. The Test Run uses a concept called "Configurations" to achieve this behaviour. A Test Case is executed for one or more Configurations to make sure that variant specific test results are kept separate.
The exact data composition of a Configuration is customizable to fit the needs of each customer.
Complete the Test Run
Once all Test Cases has been completed, the Test Run and all its content is set to read-only. Optionally, a Snapshot of all Test Run relevant items is created as part of the completion procedure. A Test Run report containing all the necessary Test Run information can be inserted into any Word Document using Aligned Elements Word Integration with a single drag and drop action.
The Test Run is a Test Managers best friend, providing the flexibility needed during test planning and full transparency during test execution, making it possible to quickly react as real time test events unfold.
Note: Burn Down Charts is under development at the point of writing and planned to be released in the next service pack.