Privacy

EU’s ChatGPT taskforce offers first look at detangling the AI chatbot’s privacy compliance

Comment

OpenAI and ChatGPT logos
Image Credits: Didem Mente/Anadolu Agency / Getty Images

A data protection taskforce that’s spent over a year considering how the European Union’s data protection rulebook applies to OpenAI’s viral chatbot, ChatGPT, reported preliminary conclusions Friday. The top-line takeaway is that the working group of privacy enforcers remains undecided on crux legal issues, such as the lawfulness and fairness of OpenAI’s processing.

The issue is important as penalties for confirmed violations of the bloc’s privacy regime can reach up to 4% of global annual turnover. Watchdogs can also order non-compliant processing to stop. So — in theory — OpenAI is facing considerable regulatory risk in the region at a time when dedicated laws for AI are thin on the ground (and, even in the EU’s case, years away from being fully operational).

But without clarity from EU data protection enforcers on how current data protection laws apply to ChatGPT, it’s a safe bet that OpenAI will feel empowered to continue business as usual — despite the existence of a growing number of complaints its technology violates various aspects of the bloc’s General Data Protection Regulation (GDPR).

For example, this investigation from Poland’s data protection authority (DPA) was opened following a complaint about the chatbot making up information about an individual and refusing to correct the errors. A similar complaint was recently lodged in Austria.

Lots of GDPR complaints, a lot less enforcement

On paper, the GDPR applies whenever personal data is collected and processed — something large language models (LLMs) like OpenAI’s GPT, the AI model behind ChatGPT, are demonstrably doing at vast scale when they scrape data off the public internet to train their models, including by syphoning people’s posts off social media platforms.

The EU regulation also empowers DPAs to order any non-compliant processing to stop. This could be a very powerful lever for shaping how the AI giant behind ChatGPT can operate in the region if GDPR enforcers choose to pull it.

Indeed, we saw a glimpse of this last year when Italy’s privacy watchdog hit OpenAI with a temporary ban on processing the data of local users of ChatGPT. The action, taken using emergency powers contained in the GDPR, led to the AI giant briefly shutting down the service in the country.

ChatGPT only resumed in Italy after OpenAI made changes to the information and controls it provides to users in response to a list of demands by the DPA. But the Italian investigation into the chatbot, including crux issues like the legal basis OpenAI claims for processing people’s data to train its AI models in the first place, continues. So the tool remains under a legal cloud in the EU.

Under the GDPR, any entity that wants to process data about people must have a legal basis for the operation. The regulation sets out six possible bases — though most are not available in OpenAI’s context. And the Italian DPA already instructed the AI giant it cannot rely on claiming a contractual necessity to process people’s data to train its AIs — leaving it with just two possible legal bases: either consent (i.e. asking users for permission to use their data); or a wide-ranging basis called legitimate interests (LI), which demands a balancing test and requires the controller to allow users to object to the processing.

Since Italy’s intervention, OpenAI appears to have switched to claiming it has a LI for processing personal data used for model training. However, in January, the DPA’s draft decision on its investigation found OpenAI had violated the GDPR. Although no details of the draft findings were published so we have yet to see the authority’s full assessment on the legal basis point. A final decision on the complaint remains pending.

A precision ‘fix’ for ChatGPT’s lawfulness?

The taskforce’s report discusses this knotty lawfulness issue, pointing out ChatGPT needs a valid legal basis for all stages of personal data processing — including collection of training data; pre-processing of the data (such as filtering); training itself; prompts and ChatGPT outputs; and any training on ChatGPT prompts.

The first three of the listed stages carry what the taskforce couches as “peculiar risks” for people’s fundamental rights — with the report highlighting how the scale and automation of web scraping can lead to large volumes of personal data being ingested, covering many aspects of people’s lives. It also notes scraped data may include the most sensitive types of personal data (which the GDPR refers to as “special category data”), such as health info, sexuality, political views etc, which requires an even higher legal bar for processing than general personal data.

On special category data, the taskforce also asserts that just because it’s public does not mean it can be considered to have been made “manifestly” public — which would trigger an exemption from the GDPR requirement for explicit consent to process this type of data. (“In order to rely on the exception laid down in Article 9(2)(e) GDPR, it is important to ascertain whether the data subject had intended, explicitly and by a clear affirmative action, to make the personal data in question accessible to the general public,” it writes on this.)

To rely on LI as its legal basis in general, OpenAI needs to demonstrate it needs to process the data; the processing should also be limited to what is necessary for this need; and it must undertake a balancing test, weighing its legitimate interests in the processing against the rights and freedoms of the data subjects (i.e. people the data is about).

Here, the taskforce has another suggestion, writing that “adequate safeguards” — such as “technical measures”, defining “precise collection criteria” and/or blocking out certain data categories or sources (like social media profiles), to allow for less data to be collected in the first place to reduce impacts on individuals — could “change the balancing test in favor of the controller”, as it puts it.

This approach could force AI companies to take more care about how and what data they collect to limit privacy risks.

“Furthermore, measures should be in place to delete or anonymise personal data that has been collected via web scraping before the training stage,” the taskforce also suggests.

OpenAI is also seeking to rely on LI for processing ChatGPT users’ prompt data for model training. On this, the report emphasizes the need for users to be “clearly and demonstrably informed” such content may be used for training purposes — noting this is one of the factors that would be considered in the balancing test for LI.

It will be up to the individual DPAs assessing complaints to decide if the AI giant has fulfilled the requirements to actually be able to rely on LI. If it can’t, ChatGPT’s maker would be left with only one legal option in the EU: asking citizens for consent. And given how many people’s data is likely contained in training data-sets it’s unclear how workable that would be. (Deals the AI giant is fast cutting with news publishers to license their journalism, meanwhile, wouldn’t translate into a template for licensing European’s personal data as the law doesn’t allow people to sell their consent; consent must be freely given.)

Fairness & transparency aren’t optional

Elsewhere, on the GDPR’s fairness principle, the taskforce’s report stresses that privacy risk cannot be transferred to the user, such as by embedding a clause in T&Cs that “data subjects are responsible for their chat inputs”.

“OpenAI remains responsible for complying with the GDPR and should not argue that the input of certain personal data was prohibited in first place,” it adds.

On transparency obligations, the taskforce appears to accept OpenAI could make use of an exemption (GDPR Article 14(5)(b)) to notify individuals about data collected about them, given the scale of the web scraping involved in acquiring data-sets to train LLMs. But its report reiterates the “particular importance” of informing users their inputs may be used for training purposes.

The report also touches on the issue of ChatGPT ‘hallucinating’ (making information up), warning that the GDPR “principle of data accuracy must be complied with” — and emphasizing the need for OpenAI to therefore provide “proper information” on the “probabilistic output” of the chatbot and its “limited level of reliability”.

The taskforce also suggests OpenAI provides users with an “explicit reference” that generated text “may be biased or made up”.

On data subject rights, such as the right to rectification of personal data — which has been the focus of a number of GDPR complaints about ChatGPT — the report describes it as “imperative” people are able to easily exercise their rights. It also observes limitations in OpenAI’s current approach, including the fact it does not let users have incorrect personal information generated about them corrected, but only offers to block the generation.

However the taskforce does not offer clear guidance on how OpenAI can improve the “modalities” it offers users to exercise their data rights — it just makes a generic recommendation the company applies “appropriate measures designed to implement data protection principles in an effective manner” and “necessary safeguards” to meet the requirements of the GDPR and protect the rights of data subjects”. Which sounds a lot like ‘we don’t know how to fix this either’.

ChatGPT GDPR enforcement on ice?

The ChatGPT taskforce was set up, back in April 2023, on the heels of Italy’s headline-grabbing intervention on OpenAI, with the aim of streamlining enforcement of the bloc’s privacy rules on the nascent technology. The taskforce operates within a regulatory body called the European Data Protection Board (EDPB), which steers application of EU law in this area. Although it’s important to note DPAs remain independent and are competent to enforce the law on their own patch where GDPR enforcement is decentralized.

Despite the indelible independence of DPAs to enforce locally, there is clearly some nervousness/risk aversion among watchdogs about how to respond to a nascent tech like ChatGPT.

Earlier this year, when the Italian DPA announced its draft decision, it made a point of noting its proceeding would “take into account” the work of the EDPB taskforce. And there other signs watchdogs may be more inclined to wait for the working group to weigh in with a final report — maybe in another year’s time — before wading in with their own enforcements. So the taskforce’s mere existence may already be influencing GDPR enforcements on OpenAI’s chatbot by delaying decisions and putting investigations of complaints into the slow lane.

For example, in a recent interview in local media, Poland’s data protection authority suggested its investigation into OpenAI would need to wait for the taskforce to complete its work.

The watchdog did not respond when we asked whether it’s delaying enforcement because of the ChatGPT taskforce’s parallel workstream. While a spokesperson for the EDPB told us the taskforce’s work “does not prejudge the analysis that will be made by each DPA in their respective, ongoing investigations”. But they added: “While DPAs are competent to enforce, the EDPB has an important role to play in promoting cooperation between DPAs on enforcement.”

As it stands, there looks to be a considerable spectrum of views among DPAs on how urgently they should act on concerns about ChatGPT. So, while Italy’s watchdog made headlines for its swift interventions last year, Ireland’s (now former) data protection commissioner, Helen Dixon, told a Bloomberg conference in 2023 that DPAs shouldn’t rush to ban ChatGPT — arguing they needed to take time to figure out “how to regulate it properly”.

It is likely no accident that OpenAI moved to set up an EU operation in Ireland last fall. The move was quietly followed, in December, by a change to its T&Cs — naming its new Irish entity, OpenAI Ireland Limited, as the regional provider of services such as ChatGPT — setting up a structure whereby the AI giant was able to apply for Ireland’s Data Protection Commission (DPC) to become its lead supervisor for GDPR oversight.

This regulatory-risk-focused legal restructuring appears to have paid off for OpenAI as the EDPB ChatGPT taskforce’s report suggests the company was granted main establishment status as of February 15 this year — allowing it to take advantage of a mechanism in the GDPR called the One-Stop Shop (OSS), which means any cross border complaints arising since then will get funnelled via a lead DPA in the country of main establishment (i.e., in OpenAI’s case, Ireland).

While all this may sound pretty wonky it basically means the AI company can now dodge the risk of further decentralized GDPR enforcement — like we’ve seen in Italy and Poland — as it will be Ireland’s DPC that gets to take decisions on which complaints get investigated, how and when going forward.

The Irish watchdog has gained a reputation for taking a business-friendly approach to enforcing the GDPR on Big Tech. In other words, ‘Big AI’ may be next in line to benefit from Dublin’s largess in interpreting the bloc’s data protection rulebook.

OpenAI was contacted for a response to the EDPB taskforce’s preliminary report but at press time it had not responded.

Responding to the EDPB’s report, after we queried the suggestion that OpenAI can now avail itself of the GDPR’s OSS, Maciej Gawronski of the law firm GP Partners — which is representing complainant behind the Polish ChatGPT GDPR investigation, told TechCrunch: “We have not been provided by anyone with any information which would suggest that OpenAI’s EU office has any powers to take ‘decisions on the purposes and means of the processing of personal data’ in the meaning of Article 4 point 16 letter a) of the GDPR.” 

“Given the centralised nature of ChatGPT service it is impossible to have headquarters in the US and personal data processing headquarters in the EU,” he added. “On top of that, I’ve just checked my May 24 invoice from OpenAI for using ChatGPT. It is issued by Open AI LLC, SF, CAL, US.”

In further remarks, Gawronski described the EDPB report as “enigmatic and shallow”, suggesting it reads as if it was “drafted by the Irish [DPC]”. “It seems like EDPB is trying hard to help OpenAI to look as compliant possible,” he added. “We are still of the opinion that UODO [Polish DPA] has competence and obligation to examine and decide our complaint.”

This report was updated with additional comment

More TechCrunch

Fisker is just a few days into its Chapter 11 bankruptcy, and the fight over its assets is already charged, with one lawyer claiming the startup has been liquidating assets…

The fight over Fisker’s assets is already heating up

A hacker is advertising customer data allegedly stolen from the Australia-based live events and ticketing company TEG on a well-known hacking forum. On Thursday, a hacker put up for sale…

Hacker claims to have 30 million customer records from Australian ticket seller giant TEG

Welcome to Startups Weekly — Haje‘s weekly recap of everything you can’t miss from the world of startups. Sign up here to get it in your inbox every Friday. Elon…

Tesla makes Musk best-paid CEO of all time and Fisker bites the dust

Dot is a new AI companion and chatbot that thrives on getting to know your innermost thoughts and feelings.

Dot’s AI really, really wants to get to know you

The e-fuels startup is working on producing fuel for aviation and maritime shipping using carbon dioxide and other waste carbon streams.

E-fuels startup Aether Fuels is raising $34.3 million, per filing

Fisker was facing “potential financial distress” as early as last August, according to a new filing in its Chapter 11 bankruptcy proceeding, which the EV startup initiated earlier this week.…

Fisker faced financial distress as early as last August

Cruise, the self-driving subsidiary of General Motors, has agreed to pay a $112,500 fine for failing to provide full information about an accident involving one of its robotaxis last year.…

Cruise clears key hurdle to getting robotaxis back on roads in California

Feel Therapeutics has a pretty original deck, with some twists we rarely see; the company did a great job telling the overall story.

Pitch Deck Teardown: Feel Therapeutics’ $3.5M seed deck

The Rockset buy fits into OpenAI’s broader recent strategy of investing heavily in its enterprise sales and tech orgs.

OpenAI buys Rockset to bolster its enterprise AI

The U.S. government announced sanctions against 12 executives and senior leaders of the Russia-based cybersecurity giant Kaspersky. In a press release, the Department of the Treasury’s Office of Foreign Assets…

US government sanctions Kaspersky executives

Style DNA, an AI-powered fashion stylist app, creates a personalized style profile from a single selfie. The app is particularly useful for people interested in seasonal color analysis, a process…

Style DNA gets a generative AI chatbot that suggests outfit ideas based on your color type

Rates of depression, anxiety and suicidal thoughts are surging among U.S. teens. A recent report from the Center of Disease Control found that nearly one in three girls have seriously…

Khosla-backed Marble, built by former Headway founders, offers affordable group therapy for teens

Cover says what sets it apart is the underlying technology it employs, which has been exclusively licensed from NASA’s Jet Propulsion Laboratory.

A new startup from Figure’s founder is licensing NASA tech in a bid to curb school shootings

Spotify is introducing a new “Basic” streaming plan in the United States, the company announced on Friday. The new plan costs $10.99 per month and includes all of the benefits…

Spotify launches a new Basic streaming plan in the US

Photographers say the social media giant is applying a ‘Made with AI’ label to photos they took, causing confusion for users.

Meta is tagging real photos as ‘Made with AI,’ say photographers

Website building platform Squarespace is selling Tock, its restaurant reservation service, to American Express in a deal worth $400 million — the exact figure that Squarespace paid for the service…

Squarespace sells restaurant reservation system Tock to American Express for $400M

Featured Article

Change Healthcare confirms ransomware hackers stole medical records on a ‘substantial proportion’ of Americans

The February ransomware attack on UHG-owned Change Healthcare stands as one of the largest-ever known digital thefts of U.S. medical records.

18 hours ago
Change Healthcare confirms ransomware hackers stole medical records on a ‘substantial proportion’ of Americans

Google said today that it globally paused its experiment that aimed to allow new kinds of real-money games on the Play Store, citing the challenges that come with the lack…

Google pauses its experiment to expand real-money games on the Play Store

Venture firms raised $9.3 billion in Q1 according to PitchBook data, which means this year likely won’t match or surpass 2023’s $81.8 billion total. While emerging managers are feeling the…

Kevin Hartz’s A* raises its second oversubscribed fund in three years

Google is making reviews of all your movies, TV shows, books, albums and games visible under one profile page starting June 24, according to an email sent to users last…

Google is making your movie and TV reviews visible under a new profile page

Zepto, an Indian quick commerce startup, has more than doubled its valuation to $3.6 billion in a new funding round of $665 million.

Zepto, a 10-minute delivery app, raises $665M at $3.6B valuation

Speak, the AI-powered language learning app, has raised new money from investors at double its previous valuation.

Language learning app Speak nets $20M, doubles valuation

SpaceX unveiled Starlink Mini, a more portable version of its satellite internet product that is small enough to fit inside a backpack.  Early Starlink customers were invited to purchase the…

SpaceX debuts portable Starlink Mini for $599

Ali Rathod-Papier has stepped down from her role as global head of compliance at corporate card expense management startup Brex to join venture firm Andreessen Horowitz (a16z) as a partner…

Brex’s compliance head has left the fintech startup to join Andreessen Horowitz as a partner

U.S. officials imposed the “first of its kind” ban arguing that Kaspersky threatens U.S. national security because of its links to Russia.

US bans sale of Kaspersky software citing security risk from Russia 

Apple has released Final Cut Pro for iPad 2 and Final Cut Camera, the company announced on Thursday. Both apps were previously announced during the company’s iPad event in May.…

Apple releases Final Cut Pro for iPad 2 and Final Cut Camera

Paris has quickly established itself as a major European center for AI startups, and now another big deal is in the works.

Poolside is raising $400M+ at a $2B valuation to build a supercharged coding co-pilot

The space industry is all abuzz about how SpaceX’s Starship, Blue Origin’s New Glenn, and other heavy-lift rockets will change just about everything. One likely consequence is that spacecraft will…

Gravitics prepares a testing gauntlet for a new generation of giant spacecraft

LTK (formerly LiketoKnow.it and RewardStyle), the influencer shopping app with 40 million monthly users, announced on Thursday the launch of a free direct message tool for creators to instantly share…

Influencer shopping app LTK gets an automatic direct message tool

YouTube appears to be taking a firm stance against Premium subscribers who attempt to use a VPN (virtual private network) to access cheaper subscription prices in other countries. This week,…

YouTube confirms crackdown on VPN users accessing cheaper Premium plans