October 18 2020

 

Are you developing software for a medical device?

Then you probably know about IEC 62304 -“Medical device software – Software life cycle processes”. This standard describes what is expected of you when building software for your medical device. As it covers the entire device life-cycle, it specifies not only the development requirements but also covers maintenance, risk management, configuration and problem resolution activities necessary for compliance.

This article will not analyze the IEC 62304 activities in detail (should you be looking for a more general overview of software development according to IEC 62304, I can highly recommend this presentation.).

Instead, we shall focus on a specific situation: you have existing piece of software. It was not developed according IEC 62304, but you want to use it in a device.

What are your options? Can you use that software at all? Is complete re-development the best, or even the only option?

Before answering these questions, it is necessary to step back and consider why this software was not developed according to the IEC 62304.

Is it a SOUP?

Let’s say the software was developed by a 3rd party, commercially or as open source. If so, this case is covered by treating the software as SOUP (Software of Unknown Provenance) according to IEC 62304.

“Unknown” in this context is not about the identity of the maker, but rather the lack of knowledge about the procedures used to develop the SOUP. As a consequence, there is an issue of trust regarding the software’s quality, its level of safety and potential risk of failure.

How does IEC 62304 let you get to grips with this lack of trust?

Start out by documenting all the functional / technical requirements that your use of the software results in. This gives you a good basis for writing the corresponding testcases that verifies correct functionality. It also serves as a starting point for identifying and assessing the risks associated with the functions of the SOUP.

After having analyzed the functional risks, you should proceed with identifying and assessing any risks from an integration the software into your device. As a further risk identification source, you should, if available, get hold of a list of all the known bugs of the SOUP.

For all the identified and assessed risks, derived from the requirements, the integration and the list of known animalities, you need to reduce the risks by applying appropriate measures.

Now this might get a bit tricky. In many cases, modifying the SOUP itself might not be a viable option.

One known way forward is to provide dedicated error-handling and comprehensive, over-arching software risk mitigations in the software that uses the SOUP, which itself must be developed according to IEC 62304.

If you can foresee reuse of this SOUP in other devices, you should consider designing a “insulation layer” around the SOUP. As a result, you get a wrapped SOUP which, in the context of your products, is safe to use.

However, note that all risk analysis activities described above to create a valid, safe SOUP, are conducted with a particular device, a particular intended use and a particular risk acceptance profile in mind.

Thus, re-use of the SOUP, leveraging the SOUP documentation described above, is only valid if the target devices have similar intended uses and near-identical risk acceptance profiles. Using the SOUP in an HIV detecting device (Class C) requires a different risk assessment than when used in an allergy detecting device (Class A).

Is it a Legacy Software?

Another scenario is that the software was developed in-house and put on the market before IEC 62304 was released.

If you now want to modify the software or use it in a different context, say with an updated hardware, you can consider the software as “legacy software”. Amendment I to IEC 62304, released in 2015, describes the necessary steps.

Start out with a risk assessment. As the software is already on the market, you will have Post Market Data to include. Leverage this data as best you can. If your identified risks do not show up in years of carefully collected Post Market Data, you have a strong case for declaring your identified risks as acceptable! Truly a nice short cut!

On the other hand, if you find reported risks which are not sufficiently addressed, you need perform the necessary risk mitigation process steps described in 4.4.3 of the standard.

If you plan to modify the software before the next release, the changes must be part of the risk analysis. These modifications need to be made according to your IEC 62304 maintenance process, described in chapter 6 of the IEC 62304.

Amendment I to IEC 62304 can be your friend, especially if your risk assessment concludes that the risks originating from the use of this software are acceptable.

Is it a Prototype?

The most common scenario for having to bring an existing software into the folds of IEC 62304 is this: you developed a software prototype and, let’s be honest, you cut some quality corners during the rapid proof of concept phase.

Now it turns out that management loves your work! They are truly impressed by its dazzling features and, moreover, how fast you were able to achieve such great results! Just quickly make it IEC 62304 compliant and get it to the market!

Do you really have to start all over again? If your prototype is really just a prototype, then you should. But what if it turns out to have a clean architecture, modules separated by interfaces and even have some unit tests in place? Wouldn’t that be good enough for IEC 62304?

No, not strictly speaking. IEC 62304 is a process standard. It is concerned with _how_ you develop your software, not the software itself. Since it is not possible to alter the past, “retrofitting the software into IEC 62304 compliance”, is formally not a way forward.

BUT. Let’s be a bit more pragmatic here.

It is rarely the case that software development is performed in the exact prescribed order defined by a software development process. Therefore, more often than not, there will be a set of remaining development and documentation tasks to finalize after your software is up and running. It could be those additional tests to write, a code review to add or updating that detail design document.

This is not specific to the IEC 62304, but applies to any software development or more general software lifecycle process. At times, the actual development and the process are simply slightly out of synch. As long as everything runs well in the end, all tests are passed, the process is largely followed and the documentation is complete and consistent, there is a good case for claiming compliance.

It would therefore be possible to argue that a prototype “catching up” with IEC 62304 is just an extreme case of this “getting back in synch” scenario.

Are there any downsides with this “catching up” approach?

Yes, potentially. IEC 62304 recognizes that the standard requires an extensive testing and documentation effort by the developers. It also recognizes that some parts of the software is less risky than others. As a result. IEC 62304 provides an opportunity to do a little less testing and documentation for the lower risk parts, as long as you are rigorous about the high risk parts. If this separation can be made, lot of work can be avoided.

Why is this a problem for an existing software? Can’t I just do a risk analysis on existing parts and then divide them into “safe” and “risky” ones?

Well, not quite. You will discover that a software architecture being efficient and easy to maintain, which is what most software developer intuitively implement and strive for, is not the same as an architecture which separates high and low risk parts from a patient safety perspective.

If grouping of safety-relevant functionality into dedicated parts is not done from the start, the riskiness tend to spread out and “contaminate” all parts of the software. An architecture with many high risk software items (parts) and very few low risk items will ensue.

You can of course accept this situation and do the additional required steps. The IEC 62304 will allow it. But it is a foregone opportunity compared to designing your software with the IEC 62304 in mind from the very start.


Conclusion

Bottom line, is using an existing software not developed according to IEC 62304 worth the trouble?

As described above, depending on the situation, there are definitely options. It all boils down to comparing the inherited risks of the software to the benefit of using it. A very complex software, being hard to rewrite, but does not introduce significant risk is a good candidate for re-use.

However if the number of inherited risks are high and the offered functionality could also be a realized by developing and documenting a new software according to a IEC 62304 compliant process, you should strongly consider to do just that.

 

October 01 2020

"Validation" is one of those terms that can be confusing for people new to medical device development.

This word, "validation",  is used in (at least) three different contexts that are not necessarily related.

These are:

  • Design Validation
  • Process Validation
  • Computer Software / System Validation

Let's take a look at each one of these in turn.

Design Validation

What is it?

The regulatory lingo (in this example from FDA 21 CFR 820.3) for "Design Validation" declares that it is about “establishing by objective evidence that device specifications conform with user needs and intended use(s).”

It is a mandatory step in the medical device development process and if you are developing a medical device you will have to perform it at some point.

Why is it done?

Design Validation is about proving that we have "built the right thing". This is often contrasted with Design Verification which is meant to prove that we "built the thing right".

The difference lies in that the latter checks that the device was designed/built according to specifications (which is very important), whereas the former checks that the device works as it is intended to work for the end user, where the intention is documented in the User Needs and to some extent in the intended use.

As you can see, there regulations have foreseen that there might be a discrepancy between the intention and the specifications derived from the intention when developing the device.

It might be the case that the designers have not solved the user needs in a way that makes the device usable.

"Anecdata" provides endless examples of devices that are built "correctly" according to specs but turns out to be unusable / dangerous when used by the end users in a real environment e.g. a glucose monitor alarm is not loud enough to be heard when travelling on a noisy bus.

Does this all sound like "usability" and "human factor analysis" activities to you? Then you are exactly right. Design Validation bears many similarities with usability activities. However, Design Validation also includes clinical tests in order to establish that clinical results corresponds to expected levels are under intended use.

What is the input for the activity?

How do you know what to validate? Simply put, your User Needs. Therefore, it is important to define the User Needs in a manner that makes them possible to validate in a practical way.

Hence, the next time you write down your User Need, think hard about how you plan to validate them.

When is it done?

Design Validation is usually performed at the end of the design process, often after the Design Verification has been completed but before the medical device is transferred to production.

The instances of your medical device used in the validation should ideally exist in production-equivalent state.

Who is involved?

The Design Validation tests are generally performed by real users (doctors, nurses, patients etc.), preferably in the actual intended environment (in the hospital, in the lab, in the ambulance, at home etc.). The idea is to get as close as possible to "real use by real users".

Engaging this type of personnel can be both complicated and costly. It is therefore important to plan ahead and secure necessary resources (location, personnel etc.) well in advance.

If the validation reveals serious flaws in the device, which requires re-design, sever consequences on development time-lines and costs are at stake.

It is therefore important to involve real users throughout the design process and not only at the end.

How is it documented?

Like all other Design Control stages, the Design Validation needs to be planned, which is documented in a Design Validation Plan, and then executed, which is often documented in validation test cases and results. The summary and outcome is documented in the Design Validation Report.

Traceability, consistency and completion are of course important steps at this stage. Therefore, Aligned Elements is a great tool to use for these activities.

Process Validation / Production Validation

What is it?

Process Validation most often means validation of your manufacturing process or parts thereof (although many other processes can be subject of this validation as well).

Or in FDA lingo, "process validation means establishing by objective evidence that a process consistently produces a result or product meeting its predetermined specifications."

Why is it done?

So why is this important? FDA requires (21 CFR 820.75) the "results of the process" ("results" = your medical device, "process" = your manufacturing process) to be "fully verified by subsequent inspection and test".

This is a way to say that each unit of your produced medical device must undergo "full" final testing and inspection and thereby we ensure that this particular instance of the medical device is safe for the user to use.

Now, there are cases when it is not possible to verify a safety relevant characteristic of the device using testing and inspections, with the typical example of sterile devices as prime contender.

To verify a device being sterile, one would need to break the sterilization barrier (open the packaging) which would of course make the device non-sterile and therefore unusable/unsellable.

So when it is not possible to "fully" inspect and test the device, you can resort to process validation.

Thus, you employ process validation to check that the production process gives your medical device that particular, safety relevant characteristic every time for every device.

What is the input for the activity?

Although ISO 13485 and FDA QSR 820:75 both lists process validation as something very important, they are both a bit elusive on exactly which parts to validate. Basically, all the characteristics that cannot be fully covered in the final inspection are eligible to process validation.

For more information, the go-to document for Process Validation, although published in 2004, is the GHTF SG 3 NB 99:10:2004.

When is it done?

Generally, process validation is a pre-production activity, meaning that the production line is fully operational but you have not yet started production "for real".

Who is involved?

Just like for Computer Software Validation, the legal manufacturer is the responsible party. However, it is not uncommon to outsource parts of the work 3rd party supplier or OEM manufacturers. 

How is it documented?

The "GHTF SG 3 NB 99:10:2004" goes a long way in describing the recommended documents. Very broadly speaking, there ought to be a Process Validation Plan, sections describing IQ, OQ and PQ activities, and finally a Process Validation Report.

 FDA explicitly requires the following from the validation, which ought to be properly documented:

  • All activities which have been carried out must be recorded, including date and signature.
  • Procedures, with which process parameters are shrivelled, must be established.
  • Only qualified personnel may validate a process.
  • Methods and data used for controlling and monitoring processes, the date of execution, persons carrying out the validation, as well as relevant equipment must be documented.
  • In case of changes, the manufacturer must assess whether re-validation is necessary and must carry it out if needed.

Computer Software / System Validation

What is it?

This type of validation, often shortened as "CSV", concerns "data processing systems are used as part of production or the quality system" (from FDA QSR 820.70) or, as freely interpreted from ISO 13485:2016, all computerized systems being used in any of the processes regulated by the QM system.

Thus, it does not concern the software in your device. Rather it refers to computer systems used to design and produce your medical device. Aligned Elements is an example of such a system. Other examples include Excel Spreadsheets, e-QMS systems, measuring equipment in production and so forth.

Why is it done?

A multitude of regulations (FDA 820, FDA CFR 20 Part 11, ISO 13485, GAMP5) recognizes that patients/users are potentially exposed to safety risks if the computer systems employed in the design and production process of the device does not behave as intended.

Computer Software Validation is the suggested remedy to this perceived risk.

What is the input for the activity?

The input for the CSV is the User Requirements that should, as one might thing, specify what the software under test can do, but rather what the organization plans/intends/want to do with it.

If you plan to use Excel for calculating a tolerance, then you the User Requirement should state exactly that.

It is recommended to take a risk-based approach to the testing and focus on the User Requirements that render the most severe risk impact if not adequately fulfilled.

So what about the validation of the software used in your device? Since you are then validating the design of your device, go check the Design Validation section. 

When is it done?

CSV should be performed on the target computer system before it is put in use.

Who is involved?

The responsible party getting the CSV activities planned and executed is clearly the organization that uses the computer system. However, the actual leg work is sometimes outsourced to 3rd party suppliers. Note that it is not mandatory for end users to be involved in the actual testing.

How is it documented?

How the organization performs CSV shall be defined in an a company SOP. The SOP shall define which computer system types that falls under CSV, how to plan and execute the CSV activities, and under which criteria a re-validation is applicable.

In its most basic form, a CSV plan should state the User Requirements which captures the intended use of the systems. The User Requirements are then tested in IQ, OQ and PQs and the results of these tests and, if applicable, deviations are summarized in the Validation Report.

Conclusion

As we can see, although these three types of Validation do not have a lot to do with each other, they are very important to Medical Device manufacturers.

A common trait for all activity groups is that they need be properly documented, often following the same type of scheme, starting out with a plan document stipulating the "things to check for" (requirements), a number of test protocols to be executed, collection of objective evidence, traceability and report generation.

Aligned Elements is a perfect tool to structure and manage these type of validation activities and manufacturers can therefore benefit from Aligned Elements in all tree areas.

For a free trial of Aligned Elements, start here.   

April 23 2020

Traditionally, many medical device manufacturers have chosen the Waterfall process as the best way of managing their development with the assumption that regulatory bodies preferred this method.

While the regulations do not prescribe a method for designing and developing a product, some FDA QSR related guidance, such as the FDA’s “Design Control Guidance for Medical Device Manufacturers”, use a language that point in this particular direction.

 

FDAProcess

 

The medical device manufacturers’ focus, due to the uncompromising effects of not being compliant, has always been on creating documents and reviewing those documents - with organization, compliance and quality being more important than end-user focused and efficient development. Testing processes have also stayed true to this path. As most of the development in the medical device industry is subjected to these forces, we often see the following “testing truths”

  • Testing starts on completion of components at the end of the development cycle
  • Since formal testing is documentation heavy, most testing is one-off.
  • Involving the end user tends to begin at design validation, after the product is either completed or nearly completed
  • The rush to decrease the time-to-market causes hasty testing without nearly enough meaningful test coverage
  • Manual testing is done really manually – meaning printed tests, checkoff sheets, and then feedback and results are input manually back into the Word test plan.
  • Regression testing is also done manually which causes burned out testers who are more prone to make errors over time.

With time, these drawbacks have become more and more obvious to the industry as they result in increased cost, lower quality and lower end-user acceptance.

Although Agile development appears to be a promising alternative, its (unwarranted) reputation of being unstructured and informal, focusing on speed rather than quality, has made the traditional medical device manufacturer shun it.

After all, does it not seem wiser to use a tried and tested, compliant development process, known and accepted by notified bodies, than to gamble with an unknown alternative, even though the cost might be a bit higher?

On the contrary. The truth is that many of the industry’s major players, including GE Healthcare, Medtronic, Roche, St. Jude Medical, Cochlear, Boston Scientific and Toshiba have adopted the agile, iterative development approach. However, going agile is not always an all-or-nothing proposition.

It is up to the manufacturer to pick and choose the parts from the agile methodology that makes most sense for their business.

agile4

Using an iterative approach will allow you to enjoy earlier and more frequent testing. One of the clear benefits of the Agile process is that testing is introduced earlier in the development process. With shorter development cycles, testers have the advantage of finding issues earlier, which should provide an improved overall quality further down-stream. This is also the case for formative usability testing, in line with IEC 62366-1:2015. The earlier we receive feedback from our target customer, the better we can steer development towards producing a viable and accepted product upon release.

Test early, fix early

Introducing test activities early in the development process allows early fixing of detected quality issues. It is a long proven fact that it is much easier and less expensive to fix issues and solve problems earlier in the development cycle when there are fewer dependencies and design is fresh in the mind.

Once you development has progressed, it is natural to forget why something was done and the difficulty to avoid serious impact when introducing a change increases dramatically.

By having frequent release cycles, you have an opportunity to adjust as you go so that your path fits your testing – both for errors as well as for acceptance. Requirements never stop changing and evolving - your testing should mirror this and your development should be in line with the results.

Involve the end user on a regular basis

The idea of Agile, or the iterative development approach, is to release working prototypes often, review the work done, improve on that work in the next iteration or sprint.

While prototypes may not always fit the development needs of medical devices, the iterative approach focuses on feature-driven iterations which can be tested. Either way, the focus is customer based – accepting and implementing requirement changes as they come in to better fit the customers’ expectations.

Shorter release schedules allow for more frequent reviews, which helps the development team stay on track while improving both quality and compliance adherence over time.

Testing – especially usability testing, a prescribed activity by MDR and FDA QSR 820 – will be frequent and timely, enabling teams to build quality into the iterations at a much faster speed. This will in turn shorten time-to-market and deliver a higher quality product as you will see an increase in the test coverage capabilities with a more frequent and early testing process.

Working features will be produced in every iteration and verification will be achieved on these frequent builds through unit tests, manual verification & regression testing. By finding the issues early, there is a significant lowering of development risks for the project and ultimately the product.

Addressing high risks with early testing

Using a risk-based approach, an established best practice in medical device development and prescribed by standards like ISO 13485, implies prioritizing design areas of high risk, address these early and direct efforts and resources to tasks targeted to minimize said risks.

This includes the implementation of the corresponding risk mitigating features but also on the verification of the effectiveness of these features, as stated in ISO 14971.

Only by verifying that an implemented feature effectively reduces the identified risk, can it be proven that the initial risk has been reduced. If it cannot be proven that a feature actually reduces a risk, the risk is considered to be unmitigated which can jeopardize the entire project.

An early proof of risk reduction effectiveness through verification, will in turn lower the business risk of the project.

Automated testing in medical device development

To increase efficiency and continue to lower time-to-market, automating testing wherever possible is a great way to increase test coverage, decrease cost and lower overall project risk. Regression testing in particular is very labor intensive and can lead to mistakes due to the stress that it inherently causes.

By automating these and other tests you will see reliability and predictability of your test plan increase. Testing visibility and transparency will enable you to better budget for future projects in terms of labor, finance, and effort.  

The key to avoiding delays and lowering development risk, is to shorten development iterations, which enables you to test early, test frequently and to adjust development to fit the user needs sooner rather than later.

Early identification of risks, issues and product validation problems can help overcome them before they become project or product killers. Usability testing, when possible, should be done throughout development to maintain the validity of the project and keep development on track.

Automate regression and other labor intensive testing wherever possible.

Happy testers tend to be more accurate testers.

A well tested, compliant product with early usability acceptance should be everybody’s goal – especially when it arrives on schedule.

March 10 2020

If you are developing medical devices, there are an awful lot of Design Control documents that need to be created.

There are several different ways of approaching that task.

Some use office products like MS Word and Excel.

Most of us are familliar with at least two of these products so I can understand that familiarity with them can cause a warm and fuzzy feeling.

However, these products were not built to handle the complexity of Design Control Management and the resulting documentation.

In fact, if you are an Office Cowboy, you are spending a lot more time and effort with design control documents than you actually have to.

esc

And it does not have to stay that way. You can be free. 

Take 10 minutes and watch our new webinar, Escape from Office.

October 31 2019

Computer Software Validation is something we discuss a lot where I work. Since Aligned Elements is a Medical Device Quality Management System relevant software, our customers make sure that their Aligned Elements installations and configurations are validated.

It is a necessary activity, although not always perceived as bringing an awful lot of value. Sometimes the cost of CSV activities actually supersedes the cost of acquiring the software itself. 

I sometimes get the question if I know any best practices when it comes to Computer Software Validation and I have made a few observations from my CSV experiences. 

Let me tell you about the top 3 things that has an impact on the Computer Software Validation effort:

1) How many test cases do you perform?

It is dead simple. More test cases (note: not necessarily "more requirements") is more work. But do YOU have to perform all those test cases? What if the supplier has already verified them? 

You are very much allowed to leverage existing supplier documentation and testing records. A properly conducted Supplier / Vendor assessment can lead you to the conclusion that the suppliers verification documentation suffice. Remember that the risk we are trying to mitigate with a Computer Software Validation effort is primarily about patient safety. A software like Aligned Elements is not directly involved in the patient safety and this fact can be leveraged when assessing and deciding on the validation scope. 

2) How do you record the test results?

The bulk of the CSV work resides in performing and recording the test results. The way you record those results can have a significant impact on the overall effort. In order to record that the actual behavior of a test step corresponds to the expected behavior, is it enough to tick a box (passed/failed)? Or do you need to write a text? Or do you need to make screenshots? As you can imagine, there is a huge difference between the former and the latter.

"But are we not required to take screen shots?".  The short answer is "No, you don't". Not if you do not think it proves anything more than the tester checking a box. FDA requires you to select your own methods and tools for quality assurance. If you have a good case for not making screen shots (which I think you have), you do not have to.

3) Who (and how many) has to sign all these CSV documents?

This might sound a bit odd but more than once I have run into cases where the validation is completed and everything that is missing is a signature from some top management figure. And now we run into a buy-in problem. If this guy has not been involved CSV approach and suddenly disagrees with how it was conducted ("BUT THERE ARE NO SCREENSHOTS?!?"), it can have a significant impact (significant as in "redo the validation").

So the lesson here is to get early buy-in from the people that sign the document. On a general level, reducing the number of signatures will speed up any documentation process. And you might want to contemplate on the necessity of having the IQ, OQ and PQ plans / reports in different documents (more documents to sign) or if you can combine them.

Validating Aligned Elements

When you acquire Aligned Elements, you get free access to our internal verification documents to use in your vendor assessments as well as pre-filled Validation documents to kick-start your validation. Contact us for more information on This email address is being protected from spambots. You need JavaScript enabled to view it.

What's up ahead regarding Computer Software Validation?

FDA announced that their much anticipated "Computer Software Assurance for Manufacturing, Operations, and Quality System Software" draft guidance should be out in 2019 but now it seems like it has been postponed to 2020. The new guidance is supposed to use a more agile approach, including a risk-based and value creating perspective on CSV activities.

 

googleplus facebook