HIPAA and AI use cases

Article 6(1)(a) and (b) and Annex I: a system is high risk when it is a safety component of, or itself is, a product covered by EU harmonisation legislation listed in Annex I, and that product is required to undergo a third party conformity assessment under that legislation. Software that predicts sepsis or deterioration to guide treatment typically qualifies as a Class IIa or higher medical device under the EU Medical Device Regulation, Rule 11, which brings it into the high risk tier, though MDR Article 5(5) provides an in house exemption that can apply to a tool a health institution builds and uses only within its own organization.

Article 6(1) and Annex I: software that analyzes a medical image to detect or prioritize a disease finding is itself, or is a safety component of, a device in scope of the EU Medical Device Regulation, and typically needs a notified body conformity assessment as software as a medical device (the FDA's AI Enabled Medical Device List shows US market authorization for devices in this category, listing authorized stroke triage devices from Viz.ai and Aidoc's BriefCase triage devices), which makes it high risk under the EU AI Act regardless of Annex III. The radiologist's own diagnostic read stays a human decision; the AI narrows and reorders the queue. Annex I high risk classification under Article 6(1) applies from 2 August 2028 (Article 113(c)); until then, Article 4 (AI literacy obligations) and Article 5 (prohibited practices), which bind the hospital as a deployer, already apply.

Annex III point 5(c): AI intended for risk assessment and pricing in relation to natural persons in life and health insurance. Article 6(3) exempts some purely preparatory tasks, but never a system that profiles natural persons. Extracting an applicant's health conditions and mapping them to the underwriting manual evaluates their health, which is profiling, so treat the system as high risk. Under the timeline as amended, the obligations for Annex III high risk systems apply from 2 December 2027, and Article 27 requires deployers of point 5(c) systems to assess the impact on fundamental rights before first use.

People must be told they are interacting with an AI system, and synthetic voice or text must be identifiable as such (Article 50). Reminding people of existing bookings and disclosure alone are limited risk. A missed appointment score used by or for a public authority to grant, reduce, revoke or reclaim access to healthcare or other essential public assistance and services, for example deciding who is offered funded transport, can fall within Annex III point 5(a), and profiling of natural persons within Annex III rules out the Article 6(3) exemption. Using the score only to decide who gets extra reminders or support does not by itself place it outside Annex III when that support is itself the assistance being granted.

Booking, rescheduling and reminders carry transparency duties: patients must be told they are dealing with AI (Article 50(1)). It becomes high risk if a public authority, or a provider acting on its behalf, uses it to evaluate eligibility for healthcare services (Annex III point 5(a)), or if it acts as an emergency healthcare patient triage system (Annex III point 5(d)). Clinical triage may also make it a medical device, which is high risk under Article 6(1) when the device needs a notified body assessment. Keep the agent to scheduling and use risk scores only to offer support.

A scribe that only transcribes and summarises for a clinician to review is not listed in Annex III and is usually minimal risk, although the provider of a system that generates text can still owe the Article 50(2) duty to mark output as AI generated, unless an exception such as an assistive function for standard editing applies. If the product qualifies as medical device software under the EU Medical Device Regulation and needs a notified body assessment, for example because it suggests diagnoses or treatment, it becomes high risk under Article 6(1) and Annex I. Health data in audio and notes falls under GDPR Article 9 in every case.

Prescreening for research that staff verify is not listed in Annex III and is usually minimal risk. The Article 2(6) exclusion covers only systems developed and put into service for the sole purpose of scientific research and development, so an operational recruitment tool used across a health system usually falls inside the Act. If the software recommends trials to a clinician as a treatment option for an individual patient, it may qualify as medical device software under the Medical Device Regulation; where that needs a notified body assessment, it is high risk under Article 6(1). Processing health records for research falls under GDPR Article 9 and national research rules.

The tier depends on what the system is scoped to do. A design limited to occupancy and discharge forecasting and to sequencing bed assignments for patients already admitted is operational decision support for hospital logistics, outside Annex III. Annex III point 5(d) covers AI used "to dispatch, or to establish priority in the dispatching of, emergency first response services", including medical aid and emergency healthcare patient triage systems. On a plain reading, that point can apply when a system dispatches, or sets the priority of dispatching, ambulance or critical care transport itself (work similar to what the Johns Hopkins center's Lifeline transport staff do for helicopter and ambulance transfers), or when it assesses the clinical urgency of an emergency patient, that is, triage. Sequencing which already admitted ED patient gets the next ward bed, and deciding whether to accept an inter hospital transfer request on capacity grounds, are not listed activities under 5(d) as written; whether either counts as dispatching or triage in a given deployment is a case by case legal question, not a settled fact, and should be assessed with counsel before relying on this tier. For public hospitals, Annex III point 5(a) (access to essential public services, including healthcare) can also be relevant. Scoping the system to bed sequencing and transfer acceptance only, and keeping every ambulance dispatch and ED triage decision with clinical staff outside the AI's recommendation, is what keeps a deployment in the lower tier.

The patient never interacts with the AI system directly, only with the clinician who reviews, edits and sends the reply, so Article 50(1), which covers systems intended to interact directly with natural persons, does not apply here; UC San Diego Health's patient facing disclosure is a voluntary practice, not an Article 50(1) duty. A copilot that only drafts for a clinician to review is not listed in Annex III and is usually minimal or limited risk, although the provider can still owe the Article 50(2) duty to mark the drafted text as AI generated, unless the exception for an assistive function with standard editing where the deploying clinician retains final responsibility applies. It would move toward high risk under Article 6(1) and Annex I only if the software qualifies as a medical device under the EU Medical Device Regulation and needs a notified body assessment, for example because it suggests a diagnosis or treatment rather than only drafting correspondence.

Annex III point 5(a) makes AI high risk when it is used by or on behalf of public authorities to evaluate eligibility for essential public assistance benefits and services, including healthcare services, or to grant, reduce or revoke them, which can cover statutory health schemes run by or for public bodies. Point 5(c) covers risk assessment and pricing in life and health insurance, not claim review. A copilot for a private insurer's claim review, where people decide, is usually outside Annex III; for public schemes, Article 6(3) may exempt a system that only performs a preparatory task, unless it profiles natural persons. GDPR rules on health data (Article 9) and on solely automated decisions (Article 22) apply in every case.

Claims fraud, waste and abuse detection is not itself listed in Annex III. Annex III point 5(a) only covers AI used by or on behalf of public authorities to evaluate a natural person's eligibility for essential public assistance benefits and services, or to grant, reduce or revoke them, which can cover statutory health schemes run by or for public bodies such as Medicare; it does not cover a private health insurer denying a member's own claim. For a private insurer, only Annex III point 5(c), risk assessment and pricing in life and health insurance, can make a system high risk, and claims fraud, waste and abuse detection by itself is not covered by it. Provider level detection that never decides a patient's own entitlement to care stays outside Annex III either way.

Triage of phishing, endpoint, network and cloud alerts for an organization's own cyber defence is not listed in Annex III. Recital 55 of the AI Act says that components intended to be used solely for cybersecurity purposes should not qualify as safety components, so the agent does not fall under Annex III point 2 (critical infrastructure), and for this scope the tier is minimal. The design changes that when the agent triages identity, data loss prevention, insider risk or user behaviour alerts in a way that scores or monitors individual employees: monitoring and evaluating the behaviour of persons in a work relationship falls under Annex III point 4(b), so that scope needs its own high risk assessment before it goes live. The Article 50(1) duty to disclose AI interaction does not apply because it is obvious to a reasonably well informed analyst that they are working with an AI agent. An operator that lets AI act autonomously on network or operational technology controls should assess that design separately, and reading employees' emails and sign in data remains subject to data protection law.

A generator of synthetic tabular test data is not listed in Annex III and does not interact with people, so it is minimal risk with only the AI literacy duty of Article 4. When the system generates synthetic text, images, audio or video, such as documents or conversation transcripts, Article 50(2) requires its provider to mark the output in a machine readable format as artificially generated. When synthetic data is used to train, validate or test a high risk system, such as credit scoring, it falls under that system's data governance duties in Article 10.

Bedside voice charting only transcribes and maps a nurse's own observations for the nurse to confirm, which is not listed in Annex III and is usually minimal risk. The end of shift note drafter generates a structured document from the patient's chart, so the provider of that generative text can owe the Article 50(2) duty to mark the output as AI generated, unless an exception such as an assistive function for standard editing applies. Neither deployment makes a clinical decision or profiles the patient today, but a version that summarized or flagged clinical risk, rather than only structuring what already happened, could need assessment as medical device software under Article 6(1) and Annex I of the EU Medical Device Regulation. Health data captured by voice or generated in a note falls under GDPR Article 9 in every case.

Drafting letters for human approval is not listed in Annex III. The decision the letter communicates may come from a separate high risk system, such as credit scoring (Annex III point 5(b)) or a public body's eligibility decision on benefits (point 5(a)); the drafting tool does not make that decision. Article 50(2) requires the provider of an AI system that generates text to mark the output as artificially generated, which puts this on the limited risk (transparency) tier; this includes an organization that builds its own drafting tool. Article 50(2) does not apply where the AI has only an assistive function for standard editing and does not substantially alter the input data or the semantics of the output.

Assigning billing and statistical codes from clinical documentation is not listed in Annex III and does not decide on a person's access to care, so no specific AI Act obligations apply beyond AI literacy. Health data processing falls under GDPR Article 9, and in the United States under HIPAA and the payment integrity rules of public payers. Minimal under the AI Act does not mean low stakes: the Veterans Health Administration classifies its computer assisted coding deployment on this page as high impact in the 2025 US federal AI use case inventory, even though coders select every code.