GDPR and LLM apps: what founders must get right before shipping AI features

Does the GDPR apply to LLM apps?

Yes, whenever the app processes personal data: names, emails, chat transcripts, customer documents, call recordings. The UK regulator puts it bluntly: “There is no ‘AI exemption’ to data protection law”. The GDPR reaches companies outside the EU too when they offer services to people in the Union, so a US startup with European users is in scope.

In the usual set-up you are the controller, because you decide what happens to the data, and the model provider is a processor acting on your behalf. Article 28 requires a contract governing that relationship, the data processing agreement or DPA; business API providers generally offer a standard one that you accept and keep on file.

Other duties, including data protection by design (Article 25), processor contracts (Article 28) and impact assessments (Article 35), sit in a lower tier capped at €10 million or 2% (Article 83(4)). Either way, these are numbers no startup can absorb.

Will customer data end up training the model?

With the major providers' paid APIs, not by default: data you send is not used to train their models. Free tiers are a different story, even at the same provider. Here is what the official pages say.

Sources: OpenAI's data controls documentation, Anthropic's privacy centre and data residency docs, and the Gemini API additional terms. These terms change, so reread them before connecting personal data and keep a copy of the version you accepted.

Which lawful basis covers AI features?

You need one of the six bases in Article 6; for AI features inside a product that is usually performance of the contract or legitimate interests, and less often consent. Purpose matters: summarising a customer's emails inside the service they pay for is not the same as using those emails to train a model.

On legitimate interests, the reference point is the European Data Protection Board's Opinion 28/2024, adopted in December 2024. It confirms there is no hierarchy between lawful bases and sets out a three-step test: the interest must be lawful, real and clearly articulated; the processing must be necessary, with no less intrusive alternative; and the balance against people's rights and reasonable expectations must hold.

Two points from the EDPB opinion that matter if you build on third-party models

• Whether a model trained on personal data is anonymous must be assessed case by case; it is not anonymous by default.
• A model developed through unlawful processing can affect the lawfulness of its later use, unless it has been properly anonymised. Deployers are expected to carry out their own checks as part of their accountability.

National regulators read it the same way. France's CNIL, finalising its AI recommendations in July 2025, echoed the EDPB's view that the GDPR often applies to models trained on personal data because they can memorise it.

What must the privacy notice say about AI?

Explain that you use AI systems, for which purposes, on which lawful basis, which providers receive the data and how long it is kept (Articles 13 and 14). If the system makes decisions based solely on automated processing that have legal or similarly significant effects, such as refusing a service or a loan, Article 22 applies: people have the right not to be subject to such decisions except in specific cases, to obtain human intervention and to receive meaningful information about the logic involved.

Do LLM apps need a DPIA?

You need one where processing is likely to result in a high risk to people's rights, particularly when new technologies are involved (Article 35). An app that uses AI to evaluate people systematically, handle health data or monitor behaviour almost certainly qualifies; an internal assistant summarising documents with no personal data probably does not. When in doubt, do it and document it before launch.

Can EU personal data go to a US model provider?

Yes, within the GDPR's transfer rules (Articles 44 onwards). The simplest route is a provider certified under the EU-US Data Privacy Framework, which the Commission found adequate in July 2023. On 3 September 2025 the EU General Court dismissed the action for annulment brought against that decision (case T-553/23), and an appeal has been lodged with the Court of Justice (C-703/25 P). Standard contractual clauses, usually built into the provider's DPA, are the fallback. Where a provider offers European processing, consider it.

What have regulators actually done about generative AI?

They have focused on lawful basis, transparency, security and children. In December 2024 Italy's data protection authority, the Garante, fined OpenAI €15 million, citing among other things training ChatGPT without an adequate legal basis, failure to notify a data breach, transparency gaps and no age verification. In March 2026 the Court of Rome (judgment 4153/2026) upheld OpenAI's challenge and the decision was taken down from the Garante's website. The issues raised remain a useful checklist for your own app.

Where do you start? A GDPR checklist for LLM apps

Start from the data, not the model: what goes in, where it goes, how long it stays. We built this checklist for products running on language models.

This guide does not replace a privacy professional or a lawyer, who remain essential for your specific case. Read it alongside our guide to the EU AI Act for startups; on the technical side, the LLM application development guide shows how customer documents enter a retrieval pipeline, and what AI agents are explains why an agent multiplies the data it touches.