AI consulting for hospitals differs fundamentally from AI consulting in any other industry. The reason has a name: patient data. It ranks among the most sensitive data there is, subject to medical confidentiality under Section 203 of the German Criminal Code (StGB) and to the GDPR – and a mistake here is not just a compliance issue, it can affect people.
At the same time, the benefits are substantial and the pace is fast. According to a study by Bitkom, Germany's digital industry association, and the Hartmannbund physicians' association, AI use in German hospitals has doubled since 2022: 18 percent of hospital physicians now work with AI, for example to analyze medical imaging – up from just 9 percent in 2022.1 78 percent of physicians see AI as a major opportunity for medicine.1
This article takes a detailed look at what matters in AI consulting for hospitals: why security and data protection must come first, which legal obligations apply – and why a local AI operated in-house (on-premise) is in many cases the safest solution.
The Starting Point: High Benefit, High Sensitivity
The use cases are real and growing. AI supports the analysis of radiological images, structures findings, automates documentation, and thereby relieves scarce medical staff. 60 percent of the physicians surveyed expect that in certain cases AI will make better diagnoses than a human.1
Yet the same physicians remain vigilant. 76 percent demand strict regulation of AI in medicine.1 This stance is justified. AI in clinical use regularly falls under the high-risk category of the EU AI Act, with the corresponding operator obligations2 (what that means in detail is covered in our article AI in Regulated Industries). Anyone introducing AI in a hospital therefore operates in a regulated space from day one – unlike in many other industries, where experimentation comes first and regulation later.
For consulting, this means the sequence is reversed. Not "use first, secure later" but "secure first, then use".
Why Data Security in Hospital AI Is Non-Negotiable
The critical point is the path the data takes. As soon as patient data reaches an AI system, three questions arise that any serious AI consulting for hospitals must answer.
First: Is the data used for training? Patient data must not be used to optimize or train an AI model – that would be use for an unrelated purpose without a legal basis.2 An AI system for clinical use must technically ensure that entered data does not "feed" the model.
Second: Who can access the data? Medical confidentiality under Section 203 StGB requires that external service providers and their subcontractors are also contractually bound to protect professional secrets.2 A standard data processing agreement under Art. 28 GDPR is not enough – the additional Section 203 clause is required. With US providers, the CLOUD Act risk comes on top: US authorities could in theory access data even in European data centers.2
Third: Does the provider meet the certification requirements? For cloud services processing health data, Section 393 of the German Social Code Book V (SGB V) requires a C5 Type 2 attestation and an establishment in Germany.2 Plain consumer chatbots do not meet this standard – health data must not be entered there.
These three questions are not bureaucratic box-ticking. They determine whether an AI deployment in a hospital is permissible at all.
On-Premise: The Local Solution for Maximum Security
One architectural approach resolves many of these risks at the root: on-premise AI – an AI model running on the hospital's own servers within its own network. The advantage follows directly from the architecture: the data never leaves the building.2 From this follows a chain of security benefits – no transfer to third parties, no CLOUD Act risk, no training on patient data, and full control over every access.2
This is the technical foundation of the medical AI solutions our sister company Admed develops for clinical use: AI that computes where the data resides. The trade-offs – model quality, infrastructure, and effort – as well as the question of when on-premise and when a secured cloud instance with a Section 203 clause and C5 attestation is the right path, are covered in depth in our dedicated article: On-Premise AI: How Companies Host Their Own AI – Benefits and Limits.
What AI Consulting for Hospitals Actually Delivers
Effective consulting places AI within a framework instead of recommending a single tool. 6Rocks works along six dimensions – the 6 Rocks. For hospitals, four are particularly relevant:
Governance and law. EU AI Act classification, GDPR, Section 203 StGB, and Section 393 SGB V belong at the start of every project, not at the end. A data protection impact assessment is mandatory for health data, not optional.
Data and architecture. The central decision: where is the data processed? On-premise, dedicated instance, or cloud – this choice determines the security level of the entire system.
Technology selection. Not every model is suited to medical use. Selection is based on suitability, verifiability, and security – not on brand recognition.
Organization and iteration. Clear responsibility, staff training, and a process that reviews new applications before they reach the patient. Physicians are open to AI – provided it is under medical responsibility and technically reliable.1
What You Should Do Specifically
- First: For every planned AI deployment, clarify the data question – which patient data flows in, and where does it go?
- Then: Determine the EU AI Act risk class and conduct a data protection impact assessment.
- Architecture decision: Compare on-premise against a dedicated cloud instance with an open mind – measured by sensitivity, performance, and effort.
- Contractually: Secure the Section 203 obligation, C5 attestation, and German location wherever an external provider is involved.
- Organizationally: Appoint responsible staff, train your teams, and establish an approval process for new AI applications.
Guiding questions: Does our patient data leave the building – and if so, with what protection? Who is liable if the system makes a mistake? And is our architecture chosen so that security is built into the design rather than depending on individual contract clauses?
Conclusion
In hardly any industry is the phrase "legally sound from day one" to be taken as literally as in the hospital. AI can improve care and relieve staff – but only if patient data stays protected. The safest answer is often the simplest: let the AI compute where the data resides. On-premise is not a step backward – for sensitive core processes, it is often the most sovereign path.
If you want to know which AI architecture fits your hospital – from risk classification to on-premise implementation – talk to us. A structured look at your starting position, no sales pitch and no slide decks.
Sources & References
- Bitkom / Hartmannbund: „KI in fast jeder siebten Praxis und vielen Kliniken im Einsatz" (survey of 616 physicians), 2025: bitkom.org
- DSN Group / datenschutz notizen: „KI-Systeme im Krankenhaus – datenschutzrechtliche und sicherheitstechnische Pflichten", 2026 (Section 203 StGB, Section 393 SGB V / C5 attestation, CLOUD Act, on-premise/open source): dsn-group.de