AI in Regulated Industries: Data Security, AI Act, Strategy

AI in regulated industries strikes many as a contradiction: on one side, a fast-moving, experimental technology – on the other, strict requirements on data protection, liability, and supervision. The result is often paralysis. Companies wait, fearing the legal risk. Yet waiting is itself a risk: law firms, hospitals, financial institutions, and public administrations that fail to act lose efficiency and fall behind.

The way out is not to ignore the rules or let them paralyze you, but to understand them. One important point up front: most companies neither want nor need to operate high-risk AI at all. The first and often most effective step is to verify exactly that – and to design use cases so they never fall into the high-risk class in the first place. This guide explains what the EU AI Act requires of high-risk systems and why implementing them is so demanding (and thus why avoidance pays off), what a risk assessment actually involves – from the data protection impact assessment to the fundamental rights impact assessment – and how large organizations in particular can get distributed responsibilities under control with a RACI matrix.

Note: This article offers practical guidance and does not constitute legal advice. 6Rocks is not a law firm. For a binding legal assessment of your individual case – for instance regarding the EU AI Act, GDPR, or liability – please consult a lawyer.

Why Regulated Industries Are a Different Playing Field

Regulated industries work with particularly sensitive data and are subject to additional obligations. In law firms it is attorney-client privilege, in hospitals medical confidentiality under Section 203 of the German Criminal Code (StGB), in the financial sector supervision by BaFin, in public administration the special relationship of trust with citizens. A data breach here is not just an IT incident but a breach of law and trust.

Then there is liability. Case law is clear: whoever deploys AI is responsible for its output. In 2026, the Higher Regional Court of Hamm ruled that a chatbot operator is fully liable for false AI statements – a hallucination is no excuse.¹ For regulated industries, this means the structure must be in place before deployment, not after.

The EU AI Act: The Four Risk Classes

The EU AI Act is the EU's first comprehensive AI law. At its core is a risk-based approach – AI is sorted into four classes according to its risk:

Prohibited systems (unacceptable risk, such as social scoring) are banned; these prohibitions already apply.²
High-risk systems significantly affect fundamental rights or safety – such as AI in recruitment, credit scoring, insurance, or as a medical device. The strictest obligations apply here. Following the "Digital Omnibus," they take effect for standalone high-risk systems on December 2, 2027, and for AI in regulated products such as medical devices on August 2, 2028.²
Limited risk means a transparency obligation – a chatbot must identify itself as AI. The general transparency obligations apply from August 2, 2026.²
Minimal risk covers the vast majority of applications and remains largely unregulated.

The fines should be taken seriously: up to 35 million euros or 7 percent of global annual revenue.³ For regulated industries, the high-risk class is decisive – because that is exactly where many of their most valuable use cases sit, and exactly where implementation becomes demanding.

Why High-Risk AI Is So Hard to Implement

Introducing a high-risk system does not mean licensing a tool. It means fulfilling an extensive catalog of obligations that accompanies the system throughout its entire lifetime. Among other things, the AI Act requires the following for high-risk AI:

a risk management system that runs continuously over the entire lifecycle (more on this below),
data governance – training, validation, and test data must be relevant, representative, and as error-free as possible,
technical documentation that makes the system's design, purpose, and functioning traceable,
logging of system operations for traceability,
transparency toward deployers and users,
human oversight – the system must be designed so that a human can intervene,
and accuracy, robustness, and cybersecurity at an appropriate level.

Before such a system reaches the market, a conformity assessment is required; the system is registered in an EU database, receives a CE marking, and is then subject to ongoing post-market monitoring.⁵

This is why high-risk AI takes months rather than days: technical standards must be met, data prepared, processes documented, and responsibilities clarified – often across several departments. Making matters harder, many of the harmonized standards are still being developed; that is precisely why the deadlines were postponed. Anyone planning a high-risk system is planning a project, not a setup.

For most companies, the real lesson is this: the smartest first step is to avoid high-risk classification wherever possible. Much of what pushes a system into the high-risk class comes down to the AI making or materially influencing decisions about people. Those who deliberately design their use cases so that a human retains final decision-making authority and none of the high-risk purposes named in the law are touched stay in the lower risk classes – with far lighter obligations. That is not circumvention, it is good design: AI supports, but does not decide alone.

What a Risk Assessment Really Involves

"Risk assessment" sounds like a document you fill out once. In reality, it covers three distinct, partly mandatory assessments that should not be confused.

1. The risk management system under the AI Act (Art. 9). For high-risk AI, this is not a one-time check but a continuous, iterative process over the entire lifecycle. You identify and analyze risks to health, safety, and fundamental rights, evaluate them, take countermeasures, and test the system deliberately – and repeat this regularly.⁵ If the system or its use changes, the loop starts over.

2. The data protection impact assessment (DPIA) under Art. 35 GDPR. As soon as a processing operation is likely to pose a high risk to the rights of affected persons – the norm when sensitive data is involved – a DPIA is mandatory.⁴ It is the most demanding part because it gets concrete: Which data flows into the system? On what legal basis? Is it used for training? Who has access? Does it leave the premises? Which technical and organizational measures mitigate the risk? A rigorous DPIA forces you to lay open the entire data flow – and exposes exactly the gaps that would otherwise turn into incidents during live operation.

3. The fundamental rights impact assessment (FRIA) under Art. 27 AI Act. For certain deployers of high-risk AI – public bodies, private providers of public services, and deployers in the areas of credit scoring and insurance – a dedicated fundamental rights impact assessment is added. In it, the deployer describes the deployment processes, the duration and frequency of use, the affected groups of people, the specific risks of harm, the measures for human oversight, and the procedure in the event of harm – and reports the result to the market surveillance authority.⁶

These three layers overlap, but they do not replace one another. Those who think them through together early save duplicate work; those who ignore them risk rework in the middle of the project.

Responsibility in Large Organizations: Clarifying Roles with RACI

The larger the organization, the bigger the question of responsibility. The AI Act distinguishes roles with different obligations: the provider develops a system or has it developed and places it on the market; the deployer puts it to use. Deployers must, among other things, ensure human oversight, retain logs for at least six months, and – where applicable – carry out the FRIA.⁶ In a large organization, you are often both at once: deployer of purchased systems and provider of in-house developments. That multiplies the obligations.

This is exactly where many large organizations fail – not because of the technology, but because of unclear responsibilities. The proven answer is a RACI matrix that defines four roles for every task:

R (Responsible) – who does the work, e.g. the business unit using the system.
A (Accountable) – who signs off and decides; this role is assigned exactly once per task, usually at leadership level.
C (Consulted) – who is involved for expertise: data protection officer, IT security/CISO, legal department, works council.
I (Informed) – who is informed of the results, e.g. executive management or supervisory bodies.

For an AI project, this means concretely: Who is responsible for the DPIA? Who approves the system? Who monitors it in operation? Who reports an incident? Without unambiguous assignment, the question "Who approved this?" goes unanswered when it matters most – and liability lands, unplanned, with executive management.

In addition, the three lines of defense model has proven itself: the first line is the business unit that uses AI and carries the operational controls. The second line consists of the risk, compliance, and data protection functions that set and monitor requirements. The third line is internal audit, which reviews independently. Many large organizations additionally consolidate governance in an AI governance board that prioritizes use cases, grants approvals, and monitors compliance. That may sound like overhead – but it is the precondition for scaling AI across the organization at all, and doing so in a legally sound way.

Our two deep dives show what this looks like in practice: for law firms, focused on confidentiality and liability, and for hospitals, focused on patient data and on-premise architecture.

Why Strategy Makes AI Efficient Despite Regulation

The common misconception goes: regulation makes AI slow. In reality, unplanned deployment makes it slow – because it leads to dead ends, rework, and liability risks. A risk assessment answers up front exactly the questions that otherwise bring a project to a halt midway: Are we allowed to use this data? Is this tool permissible? Who is liable? Once these questions are settled and the roles assigned, the team can work freely and quickly within the defined boundaries. Regulation shifts from being a brake to being a guardrail.

That is precisely what 6Rocks is about. We structure AI transformation along six dimensions – the 6 Rocks: Strategy, Governance, Organization, Data, Technology, and Iteration. In regulated industries, Governance and Data come first so that the other four can carry weight. Those who clarify their strategy first – which use cases, which data, which security level – deploy AI more efficiently afterward than anyone who starts without a plan.

What You Should Do Specifically

Take inventory: Record all AI systems in use – including quietly embedded features.
Classify – and avoid high risk: Assign each system to an AI Act risk class. For high-risk candidates, first check whether the use case can be designed – for instance with a human making the final decision – so that it stays in a lower class.
Set up the assessments: Conduct a data protection impact assessment where sensitive data is involved, check whether a FRIA is required, and establish ongoing risk management.
Clarify roles: Use a RACI matrix to define who is responsible, who decides, who is consulted, and who is informed.
Architecture & strategy: Make a deliberate decision on data hosting (on-premise vs. secured cloud) and concentrate resources on the use cases with the greatest value.

Guiding questions: Do we know which AI is running in our organization and which risk class it falls into? Is it clear who is responsible for each assessment? And do we have a strategy – or just tools?

Conclusion

AI in regulated industries is not a contradiction but a question of sequence: structure first, speed second. The EU AI Act and data protection law set the framework; risk management, the DPIA, and – where required – the FRIA translate it into manageable steps; a RACI matrix clarifies responsibility; and a clear strategy ensures that AI works quickly and effectively within that framework. Those who know what they want do not have to choose between security and efficiency.

If you want to know where your organization stands between regulation and value creation, talk to us – a structured look at your starting position, no sales pitch, no slides.

6Rocks – Your Path to AI Sovereignty.

Sources & References

Higher Regional Court of Hamm (case no. 4 UKl 3/25) on the full liability of chatbot operators for AI errors, reported by Handelsblatt, 2026: handelsblatt.com
Gibson Dunn: "EU AI Act Omnibus Agreement — Postponed High-Risk Deadlines and Other Key Changes", 2026 (deadlines Dec 2, 2027 / Aug 2, 2028, transparency obligations from Aug 2, 2026): gibsondunn.com
Deloitte Deutschland: „EU AI Act – Änderungen durch den ‚Digital Omnibus on AI'" (risk classes, fines of up to €35 million / 7% of annual revenue): deloitte.com
DSN Group / datenschutz notizen: „KI-Systeme im Krankenhaus – datenschutzrechtliche und sicherheitstechnische Pflichten", 2026 (data protection impact assessment, Section 203 StGB, on-premise): dsn-group.de
EU AI Act, Article 9 "Risk Management System" (continuous, iterative process) and high-risk requirements Art. 9–15: artificialintelligenceact.eu/article/9
EU AI Act, Article 26 "Obligations of Deployers" and Article 27 "Fundamental Rights Impact Assessment": artificialintelligenceact.eu/article/27