Checking
A child protection report written with ChatGPT in Australia
A child protection worker in the Australian state of Victoria used ChatGPT to draft a report for the Children's Court. The text understated the risk to the child, and the regulator ordered generative AI tools to be blocked for two years.
In December 2023 Victoria's Department of Families, Fairness and Housing (DFFH) reported a privacy incident to the Office of the Victorian Information Commissioner (OVIC). A child protection worker had used ChatGPT to draft a report submitted to the Children's Court in a case concerning a young child. On the day of a further hearing, a departmental lawyer noticed unusual language and doubted the adequacy of the risk assessment. The report was withdrawn and replaced, and the people affected were notified.
OVIC's investigation, published in September 2024, found that the worker had entered a significant amount of personal and delicate information into ChatGPT, including names and details of risk assessments relating to the child. This disclosed it to OpenAI, an overseas company, outside the department's control. The generated text also contained inaccurate personal information that downplayed the risks to the child. Fortunately, OVIC said, the flaws did not ultimately change the decisions of child protection or the court.
A departmental review of a year of cases in the worker's wider unit found 100 with signs that ChatGPT may have been used to draft child protection documents. Between July and December 2023 nearly 900 employees, almost 13 per cent of the workforce, had visited the ChatGPT website. There were no specific rules, training or technical controls.
OVIC ordered the department to direct child protection staff not to use web-based generative text tools in their official duties, and to block them at network level from 5 November 2024 until 5 November 2026, including ChatGPT, Claude, Gemini and Copilot. The department accepted the findings.
What can we learn from it?
Two things went wrong at once: confidential data reached an overseas provider, and machine-written text distorted a professional assessment. AI writes fluent, plausible sentences even when their content is wrong, which makes it a good drafter but not an author. Confiella sends only reviewed, pseudonymised text to the cloud; checking the answer remains the professional's job. In an official report or expert opinion, take over only sentences you have checked and stand behind, and always write the risk assessment yourself.
Sources
- OVIC: Investigation into the use of ChatGPT by a Child Protection worker
- OVIC: OVIC finds department responsible for breaches of privacy through use of ChatGPT
- Cyber Daily: Victorian child protection worker uses ChatGPT for protection report
This article is based on the public sources listed. If you find a mistake, write to us: [email protected]