Regulation explainerEnterprise AI

AI data sovereignty: turning GDPR, HIPAA and SOC 2 into architecture decisions

Data sovereignty for an AI system is not only about where the database sits. Prompts, retrieved documents, outputs, logs and training examples each travel and persist differently, and GDPR, HIPAA and SOC 2 attach different obligations to them. This guide maps those obligations to the decisions architects make: where inference runs, what is logged and for how long, which sub-processors are allowed and who can reach the data.

Reviewed 8 min read

On this page
  1. Five dimensions of sovereignty in an AI system
  2. Where data travels in a single AI request
  3. What GDPR, HIPAA, SOC 2 and the EU AI Act ask of an AI deployment
  4. Mapping obligations to architecture decisions
  5. Logging without creating a new sensitive data store
  6. Due diligence on a model provider
  7. Scope and limits of this mapping
  8. Questions and answers
  9. Sources

Five dimensions of sovereignty in an AI system

Contracts often say data residency when they mean only one of these. Ask about each.

Storage location
Where prompts, outputs, logs and indexes are written, including backups and disaster-recovery copies.
Processing location
Where inference computes. A provider can store data in one region and process requests in another, or overflow to other regions under load.
Key control
Who holds the encryption keys and whether the provider can decrypt without you. Customer-managed keys protect stored data, not data being processed.
Operational access
Which people, in which countries, can reach the data for support, abuse monitoring or debugging, and with what approval.
Legal reach
Whose laws can compel disclosure. Under the US CLOUD Act, a provider must disclose data in its possession, custody or control whether it is stored inside or outside the United States9.

Where data travels in a single AI request

01User prompt02Retrieved context03Model inference04Generated output05Logs and traces06Evaluation and tuningsets
  1. User prompt

    Free text that can carry personal data or PHI even when the workflow was not designed for it.

  2. Retrieved context

    Documents and records pulled from internal systems to ground the answer.

  3. Model inference

    Processing by the model, in your environment or a provider's.

  4. Generated output

    The answer, which may restate or infer sensitive facts.

  5. Logs and traces

    Copies kept for debugging, abuse monitoring or audit.

  6. Evaluation and tuning sets

    Examples saved to test or improve the system, often kept longest.

Conceptual data path for one AI request; each stage can create a copy with its own location, retention and access list.

What GDPR, HIPAA, SOC 2 and the EU AI Act ask of an AI deployment

The provisions that most often change an AI architecture. Sector rules, such as those on health records or financial services, add to them.

GDPR (Regulation (EU) 2016/679)

EU and EEA, plus organizations elsewhere that offer goods or services to, or monitor, people in the EU

Applies whenPersonal data appears in prompts, retrieved context, outputs or logs1.

  • Article 28: a provider processing on your behalf needs a processor contract, and its sub-processors need your prior authorization1.
  • Chapter V: transfers outside the EEA need an adequacy decision or safeguards such as standard contractual clauses, plus supplementary measures where local law undermines them2.
  • Article 35: a data protection impact assessment where processing with new technologies is likely to be high risk1.
  • Article 5: data minimization and storage limitation govern what prompts and logs may hold and for how long1.
  • Article 32: security appropriate to the risk, including access control and encryption1.

HIPAA Privacy and Security Rules (45 CFR Part 160 and Part 164)

United States: covered entities and their business associates

Applies whenProtected health information is typed into prompts, retrieved from clinical systems or stored in logs by or for a covered entity3.

  • A vendor that creates, receives, maintains or transmits PHI for a covered entity is a business associate3.
  • The business associate agreement must limit use and disclosure, require reporting of improper disclosures, bind subcontractors and provide for return or destruction of PHI4.
  • Security Rule technical safeguards: access control, audit controls, integrity, authentication and transmission security5.
  • Minimum necessary: limit PHI to what the purpose requires, including what goes into prompts and retrieved context8.

SOC 2 (AICPA Trust Services Criteria)

AICPA attestation framework used internationally; not a law

Applies whenYou rely on an AI or hosting provider as a service organization, or your own customers rely on you6.

  • The report is an auditor's examination of controls relevant to security, availability, processing integrity, confidentiality or privacy6.
  • Read the scope: services included, criteria examined and sub-service organizations carved out.
  • Operate the complementary user entity controls the report assigns to you, such as your own access reviews.

EU AI Act (Regulation (EU) 2024/1689)

European Union

Applies whenYou deploy an AI system in the EU, with the heaviest duties for systems classed as high-risk7.

  • Deployers of high-risk systems must follow the instructions for use, assign competent human oversight and keep the logs the system generates where they control them7.
  • See EU AI Act deployer obligations for the full set; this page stays on data location and handling.

Mapping obligations to architecture decisions

Read across a row to see where frameworks agree; where they differ, the strictest requirement that applies to you usually sets the design.

DecisionWhat GDPR pushes you towardsWhat HIPAA pushes you towardsWhat SOC 2 evidence should show
Where inference runsProcessing in the EEA or an adequate country, or documented transfer safeguardsAny location, provided every vendor touching PHI has signed a business associate agreementWhich regions and facilities are in scope, including failover regions
Prompt and output loggingOnly what the purpose needs; content logging justified in the impact assessmentAudit controls over system activity without copying PHI into unprotected log storesLogging and monitoring controls examined over the report period
RetentionDefined periods per log type, with deletion that also reaches backups and indexesRetention aligned with your policies, and return or destruction of PHI when a vendor relationship endsData disposal controls and evidence that they operated
Model providers and sub-processorsA processor contract, an authorized sub-processor list and notice of changesAgreements with each vendor and subcontractor that handles PHIVendor management controls and the sub-service organizations carved out of scope
Staff and vendor accessAccess limited to need, with a transfer analysis for support staff outside the EEAUnique user identification, access control and authentication for ePHILogical access controls, access reviews and monitoring of privileged users

Logging without creating a new sensitive data store

Prompts copied into a general log platform can end up with wider access and longer retention than the systems they came from.

0 of 7 checked

Due diligence on a model provider

Run these for every provider that receives prompts, context or outputs, including AI features inside SaaS you already license.

  1. Draw the data flow

    Show every component that receives prompts, context, outputs or logs, including the provider's abuse monitoring and support tooling.

    Output
    Data flow diagram

  2. Collect the contract set

    Data processing agreement, sub-processor list with locations, a business associate agreement where PHI is involved, and each document's change-notification terms.

    Output
    Contract register

  3. Confirm retention and training terms

    Get in writing how long prompts and outputs are kept, for what purposes and whether they are ever used to train or improve models.

    Output
    Retention and use statement

  4. Pin down processing regions

    Ask where inference runs normally and during failover or overflow, not only where data is stored.

    Output
    Region map

  5. Read the SOC 2 Type 2 report properly

    Check the period covered, services in scope, carved-out sub-service organizations, exceptions noted and the user controls assigned to you.

    Output
    Assurance review note

  6. Complete transfer and impact assessments

    Document the transfer tool and supplementary measures for data leaving the EEA, and add AI-specific risks to the impact assessment.

    Output
    Updated DPIA and transfer assessment

  7. Set review triggers

    Repeat the review when the provider changes sub-processors, regions, retention terms or the model behind your endpoint.

    Output
    Review schedule

Scope and limits of this mapping

Questions and answers

Is a private cloud deployment enough for data sovereignty?

Not on its own. A private cloud tenancy settles storage and processing location, but sovereignty also depends on who holds the keys, which operator staff can reach the environment, which law binds the cloud operator and where logs and backups go. Treat it as one control among several and document the rest in your risk assessment.

Do we need to control the encryption keys?

Customer-managed keys reduce a provider's ability to read stored data and let you cut off access, which can strengthen a transfer assessment. They do not protect data during processing, because inference works on decrypted inputs. Where provider access during processing is unacceptable, the options are self-hosted inference or confidential computing, verified separately.

Does a SOC 2 report mean a vendor is GDPR or HIPAA compliant?

No. SOC 2 is an independent auditor's examination of controls over a defined system against the AICPA criteria. It is useful evidence for a GDPR security assessment or a HIPAA risk analysis, but it does not replace a processor contract, a business associate agreement or your own assessment of how the AI system uses personal data.

Is de-identified or pseudonymized data out of scope?

It depends on the framework. HIPAA treats health information de-identified by expert determination or by removing the listed identifiers as no longer individually identifiable8. Under GDPR, pseudonymized data remains personal data. In AI systems, free-text prompts and retrieved documents often reintroduce identifiers, so de-identification must be tested, not assumed.

Can provider support staff in another country see our prompts?

Possibly, so ask. Many providers keep limited prompt copies for abuse monitoring and let support engineers reach customer data under defined procedures. Find out where those staff sit, what approval they need and how long copies last; under GDPR, remote access from outside the EEA can be a transfer needing its own safeguard.

Sources

  1. Regulation (EU) 2016/679 (General Data Protection Regulation) — EUR-Lex · checked 10 October 2026
  2. Recommendations 01/2020 on measures that supplement transfer tools to ensure compliance with the EU level of protection of personal data — European Data Protection Board · checked 10 October 2026
  3. 45 CFR § 160.103 Definitions (business associate) — Legal Information Institute, Cornell Law School · checked 10 October 2026
  4. 45 CFR § 164.504 Uses and disclosures: organizational requirements (business associate contracts) — Legal Information Institute, Cornell Law School · checked 10 October 2026
  5. 45 CFR § 164.312 Technical safeguards — Legal Information Institute, Cornell Law School · checked 10 October 2026
  6. SOC 2: Controls at a Service Organization Relevant to Security, Availability, Processing Integrity, Confidentiality, or Privacy — AICPA & CIMA · checked 10 October 2026
  7. Regulation (EU) 2024/1689 (Artificial Intelligence Act) — EUR-Lex · checked 10 October 2026
  8. 45 CFR § 164.514 Other requirements relating to uses and disclosures of protected health information (de-identification, minimum necessary) — Legal Information Institute, Cornell Law School · checked 10 October 2026
  9. 18 U.S. Code § 2713 Required preservation and disclosure of communications and records — Legal Information Institute, Cornell Law School · checked 10 October 2026

More in Enterprise AI

Back to Enterprise AI

Next step

Map your AI data flows against your obligations

Describe the workflow, the data it touches and the providers involved. We will reply with the questions to settle first and whether a private or managed deployment fits your constraints.

Discuss AI data sovereignty