News and lessons
Artificial intelligence can help a great deal in professional work: it summarises, organises and helps you prepare. Real cases show that most problems come from a few avoidable mistakes. We have collected them, each with what we can learn from it. Every article cites public sources.
Keep identifying data out of general chatbots.
Don't share client material by link.
Know the provider: what it keeps and what it uses it for.
Don't give the assistant access to everything.
Keep recordings with you.
Check machine-made text before it goes into a record.
Let the client know.
-
Identifying data
Nearly 3,200 patients' data in an unapproved AI tool
In September 2026 an employee at Johns Hopkins Bayview in Baltimore uploaded a spreadsheet of patient data to a public AI tool the hospital had not approved. Patients were notified in October.
-
Choosing a provider
Open storage at an AI aggregator: 144,000 chat logs
In October 2026 it emerged that 302.ai, a service offering ChatGPT, Claude, Gemini and other models in one place, had kept 17.6 million files, including more than 144,000 chat logs, in publicly accessible cloud storage.
-
Sharing
Shared Claude conversations and Artifacts on Google
In July 2026 users reported that shared Claude conversations and Artifacts were appearing in Google's search results. VentureBeat could verify the Artifacts; Anthropic said the links cannot be guessed.
-
Identifying data
UK tribunal: client documents in an open AI tool breach confidentiality
The UK Upper Tribunal has said that uploading confidential client documents to an open AI tool such as ChatGPT breaches confidentiality and waives legal privilege. The ruling became public in February 2026.
-
Choosing a provider
Court: defence documents made with Claude are not privileged
In February 2026 a federal court in New York ruled that documents a defendant created with Claude on his own initiative were not protected by legal privilege. Anthropic's privacy policy played a part in the decision.
-
Access
Copilot summarised emails labelled confidential despite the rules
In early 2026 a bug made Microsoft 365 Copilot Chat process and summarise emails with confidentiality labels, although organisations' data protection policies explicitly excluded them. The bug lasted for weeks.
-
Choosing a provider
“Chat & Ask AI”: 300 million messages could be read
In January 2026 it emerged that, because of a misconfiguration in a popular app that relays ChatGPT, Claude and Gemini, a researcher could access 300 million messages from more than 25 million users.
-
Choosing a provider
AI plush toy: 50,000 children's conversations could be read
In January 2026 two security researchers found that anyone with a Google account could log in to the web console of a talking AI plush toy maker and read around 50,000 conversations with children.
-
Recordings
A lawsuit over a California health system's AI scribe
A class action filed in late 2025 claims that California's Sharp HealthCare recorded appointments with an AI scribe without patients' knowledge, and that the notes falsely said they had consented. These are allegations.
-
Access
A law firm's files open without login on a legal AI platform
In October 2025 an independent researcher found that on the legal AI platform Filevine, full access to one law firm's document store could be obtained without logging in. A search for “confidential” returned nearly 100,000 results.
-
Identifying data
Youth care files in public chatbots in Eindhoven
In December 2025 the Dutch city of Eindhoven said its staff had uploaded many files containing personal data to public AI websites, including youth care documents. How many can no longer be established.
-
Identifying data
Flood victims' data uploaded to ChatGPT in Australia
A former contractor of an Australian state authority uploaded a spreadsheet of more than 12,000 rows to ChatGPT, holding flood victims' data including health information. The authority made it public more than six months later.
-
Choosing a provider
43 million messages on an open server of two AI companion apps
In autumn 2025 the server behind two AI companion chat apps turned out to be reachable without authentication: 43 million messages and 600,000 images and videos from more than 400,000 users could be read.
-
Informing clients
Therapists secretly used ChatGPT with their clients
In September 2025 MIT Technology Review spoke to clients who discovered by accident that their therapists were using ChatGPT. In one case, the client's own words had gone into the service without his knowledge.
-
Access
ShadowLeak: ChatGPT's research agent leaked data from Gmail
In September 2025 Radware showed how an email with hidden instructions could make ChatGPT's Deep Research agent send personal data from a mailbox to an outside address, straight from OpenAI's servers.
-
Informing clients
Illinois: artificial intelligence may not provide therapy
In August 2025 the US state of Illinois banned AI from making therapeutic decisions or taking part in therapeutic communication. It may still help with administration, but recording or transcribing a session needs written consent.
-
Identifying data
Italian regulator: don't ask a chatbot to read medical reports
In July 2025 Italy's data protection authority called it alarming that more and more people upload blood tests, X-rays and other medical reports to AI chatbots, asking for an interpretation or a diagnosis.
-
Sharing
Shared ChatGPT conversations in Google's search results
In July 2025 about 4,500 shared ChatGPT conversations could be found on Google, some with names, CVs and confidential work material. Within days, OpenAI removed the feature that made this possible.
-
Recordings
NHS England: stop using non-compliant AI scribes
In June 2025 NHS England wrote to NHS organisations telling them to pause or stop using AI scribing tools that did not meet its standards. The British Medical Association advised GPs to stop too if they were in any doubt.
-
Sharing
Meta AI: personal chats in a public feed
In June 2025 it emerged that the public feed of the Meta AI app contained many conversations users had evidently not meant for the public: legal and medical questions, home addresses, CVs.
-
Access
EchoLeak: one email was enough to make Copilot hand over data
In June 2025 Aim Security showed how an email carrying hidden instructions could make Microsoft 365 Copilot send a user's sensitive data to an outside server, without a single click.
-
Choosing a provider
Deleted ChatGPT chats preserved, 20 million logs ordered in a lawsuit
In The New York Times' copyright case, a US court ordered OpenAI in 2025 to preserve deleted ChatGPT chats, and then to hand over 20 million chat logs. Sam Altman himself said that ChatGPT conversations carry no legal confidentiality.
-
Choosing a provider
DeepSeek: a ban in Italy and findings in South Korea
In 2025 Italy's data protection authority restricted DeepSeek's data processing with immediate effect, and South Korea's regulator found that the company had passed users' prompts to a Chinese company without their consent.
-
Identifying data
UK law firm: 32,000 ChatGPT requests in a week
In February 2025 it emerged that more than 32,000 requests had gone from the law firm Hill Dickinson to ChatGPT in a single week. The firm made access to public AI tools subject to approval, and said this was not a ban.
-
Choosing a provider
DeepSeek's iPhone app sent data unencrypted
In February 2025 the security company NowSecure found that DeepSeek's iOS app switched off the iPhone's protection for encrypted connections and sent some device data over the internet unencrypted.
-
Choosing a provider
An open database with DeepSeek's chat logs
In January 2025 security researchers found a DeepSeek database that anyone could reach without a password. It held users' chat history and secret keys, and could be queried from a web browser.
-
Choosing a provider
ModeLeak: flaws in Google's Vertex AI cloud platform
In November 2024 researchers at Palo Alto Networks showed two flaws in Google Vertex AI: a malicious job or model could reach a project's storage, data tables and other fine-tuned models.
-
Checking
Invented sentences in medical transcripts: Whisper's flaw
In October 2024 the AP reported that OpenAI's Whisper speech recogniser sometimes makes up whole sentences, even treatments that do not exist. A tool built on it had already transcribed millions of medical visits.
-
Checking
A child protection report written with ChatGPT in Australia
A child protection worker in the Australian state of Victoria used ChatGPT to draft a report for the Children's Court. The text understated the risk to the child, and the regulator ordered generative AI tools to be blocked for two years.
-
Recordings
Otter.ai sent out investors' private discussion after a call
In September 2024 an AI note-taker kept recording after a video call ended, and a participant was emailed a transcript of the investors' confidential discussion afterwards. A class action against Otter.ai followed.
-
Recordings
Therapy notes in a database without a password
In September 2024 a US mental health and addiction treatment provider that markets itself as AI-powered turned out to have left a database open without a password, holding therapy notes and identity documents.
-
Identifying data
Patient data in a chatbot: a warning from the Dutch regulator
In August 2024 the Dutch data protection authority said it had received several breach reports about employees entering patients' and customers' data into AI chatbots. In one, a GP practice employee had entered patients' medical data.
-
Choosing a provider
“Repeat forever”: training data in ChatGPT's answers
In November 2023 researchers showed that a simple trick made ChatGPT return parts of its training data word for word, including a real email address and phone number. About 200 dollars bought more than 10,000 such passages.
-
Identifying data
Samsung: source code and a meeting transcript in ChatGPT
In March 2023, within 20 days, three employees in Samsung's semiconductor division gave ChatGPT confidential material: source code twice and a meeting transcript once. The company later restricted generative AI.
-
Identifying data
ChatGPT bug exposed chat titles and billing data; Italy stepped in
In March 2023 a bug let ChatGPT users see the titles of other people's chats, and let some subscribers' billing details be seen by others. Italy's data protection authority stopped the service; a court later annulled its fine.
-
Informing clients
Koko: emotional support messages written with GPT-3
A volunteer emotional support service sent replies partly written by GPT-3 to about 4,000 people, flagged only by a short note in brackets. In January 2023 it was widely criticised for the lack of informed consent.
-
Choosing a provider
Amazon: ChatGPT's answers resembled internal material
In January 2023 an Amazon lawyer asked staff not to give ChatGPT any confidential information, having already seen answers that closely resembled internal material. Where the resemblance came from was never established.
-
Informing clients
HUF 250 million fine for AI emotion analysis of calls
A Hungarian bank used artificial intelligence to analyse callers' emotional state in recorded customer service calls, without properly informing its customers. The Hungarian data protection authority fined it HUF 250 million.
-
Choosing a provider
A chatbot trained on private messages in South Korea
South Korea's Scatter Lab trained its public chatbot on users' private KakaoTalk conversations without removing names or phone numbers. In April 2021 the data protection authority fined the company.
-
Recordings
Siri contractors heard confidential medical details
In 2019 a whistleblower said contractors grading Siri recordings regularly heard confidential medical information and intimate moments, mostly from accidental activations. Apple apologised and changed its practice.
-
Recordings
Leaked Google Assistant recordings, many of them made by accident
In July 2019 Belgium's public broadcaster listened to over a thousand Google Assistant recordings, obtained through someone working for a Google contractor. Many were made without the wake word, and addresses and medical questions could be heard.
-
Recordings
A data request returned 1,700 of a stranger's Alexa recordings
In 2018 a German customer asked Amazon for his data under the GDPR. The download also held about 1,700 voice recordings from a stranger's home, including the bedroom and the shower.