The AI Act was formally adopted by the full European Parliament on 13 March, and we have reviewed the current text to form our initial impressions of the law’s impact on the MedTech industry.
The AI Act will be crucial for any AI-containing product or device used in our industry. It represents the world’s first regulatory framework for AI and aims to ensure trustworthy AI in Europe by mandating that AI systems respect fundamental rights, safety, and ethical principles, and by addressing the risks posed by AI models.
Failure to comply with the AI Act could result in fines of up to €35,000,000 or up to 7% of a company’s global annual turnover, whichever is higher.
In this blog post, we will explore:
- the different AI categories mentioned in the AI Act,
- outline their requirements,
- and discuss the implications of the AI Act for the MedTech industry.
Prohibited AI Systems
The AI Act outlines a number of prohibited AI practices. These include AI models that:
- use behavioral manipulation techniques.
- exploit vulnerabilities based on demographics or biometrics.
- employ facial recognition databases obtained through untargeted scraping, among others.
These prohibitions are designed to protect against manipulative or exploitative practices and to uphold ethical standards in AI usage.
General Purpose AI
For general-purpose AI models that can perform a wide range of tasks competently, and for AI systems designed to directly interact with people, the AI Act recognizes and addresses emerging challenges such as deepfakes and emotion recognition technologies.
The AI Act enforces transparency requirements, especially concerning the disclosure of content that has been artificially generated or manipulated.
Provisions for watermarking are intended to increase transparency, with exceptions allowed for certain use cases that are legally authorized or deemed artistic or satirical in nature.
In our industry, relevant examples might include AI models that aid in the creation of high-quality records, such as those for change control or root cause analysis.
High-Risk AI Systems
Criteria for high-risk AI systems are designed to mitigate potential harms to health, safety, or fundamental rights.
These criteria include:
- strict compliance with documentation and registration requirements,
- adherence to robust data governance processes,
- and a significant focus on cybersecurity, underscoring the importance of thorough risk assessment and compliance.
While manufacturers of In Vitro Diagnostics (IVD) and Medical Devices (MD) are familiar with such requirements, these additional requirements represent new challenges for certain equipment and software suppliers to the life sciences industry.
Suppliers impacted by the AI Act must now navigate these requirements in addition to the Good Manufacturing Practice (GMP) standards demanded by their customers.
This heightened emphasis on data governance and cybersecurity aligns with the latest developments in Software as a Medical Device (SaMD) guidelines and regulations.
Examples falling under this category include AI-containing MDs or IVDs.
Human Oversight and Employer Obligations
Robust human oversight emerges as another cornerstone of the AI Act, requiring competent personnel with the necessary training and authority.
In particular, employers deploying high-risk AI systems will be required to inform workers of their integration, promoting transparency and accountability in the workplace.
Impact on the MedTech industry
While the AI Act does not impact the entire MedTech and Life Sciences industry, software developers and device manufacturers planning to integrate AI models into their products — whether as industry support tools or as software as a medical device — should consider several requirements during the design and development phases. Given the recent surge in AI-enabled software, the influence of the AI Act will be a pivotal factor in the market introduction of these products.
In response to these developments, organizations should adopt a proactive stance by first defining how the AI Act may affect their products. This approach will equip them with a clear strategy for navigating the regulatory landscape once the AI Act is enforced.
Â
Â
Need help with AI Act Implementation?
If you’re seeking a partner to assist in crafting your action plan or navigating the AI Act within the Life Sciences or MedTech industries, we warmly invite you to contact us for an initial discussion.