Choosing a provider
A chatbot trained on private messages in South Korea
South Korea's Scatter Lab trained its public chatbot on users' private KakaoTalk conversations without removing names or phone numbers. In April 2021 the data protection authority fined the company.
Scatter Lab's chatbot Lee Luda was presented as a 20-year-old female university student, and it gathered more than 750,000 users on Facebook Messenger within a month. In early January 2021 many users complained about its crude and discriminatory replies, and on 12 January South Korea's data protection authority, the PIPC, opened an investigation.
The investigation soon turned to the training data. Two of the company's other apps, Science of Love and Text At, had collected conversations from KakaoTalk, a popular Korean messaging app. The chatbot was trained on around 9.4 billion of these messages from some 600,000 users. It picked its replies from a database of 100 million messages taken from conversations of women in their twenties. Scatter Lab made no effort to delete or encrypt names, phone numbers or addresses.
On 28 April 2021 the authority found eight violations. A “new service development” clause in the apps' login terms did not amount to explicit consent: users could not have foreseen from it that their conversations would become a chatbot. Between October 2019 and January 2021 the company had also posted 1,431 KakaoTalk messages on the code-sharing site GitHub, including 22 names and 34 place references. The fine came to 103.3 million won in total (about 92,900 US dollars). It was the first time the PIPC had sanctioned an AI company for the indiscriminate use of personal data.
The reports on the decision do not claim that real personal data appeared in the chatbot's replies. What the authority objected to was the collection and the use beyond the original purpose.
What can we learn from it?
Text handed to a service can be put to other uses if the terms allow it broadly enough, and a conversation always includes the other party, who agreed to nothing. With many AI services, especially business plans, the terms rule out training on what users type, or it can be switched off in the settings. For client material, choose such a service: find the part of the terms about training, and leave out names, phone numbers and addresses even then.
Sources
- Future of Privacy Forum: South Korea: The First Case Where the Personal Information Protection Act was Applied to an AI System
- Byline Network: ‘이루다’ 개발사 과징금 등 1억원…AI 기업 무단 개인정보 처리 첫 제재
This article is based on the public sources listed. If you find a mistake, write to us: [email protected]