ChatGPT: opportunities and privacy issues

ChatGPT: opportunities and privacy issues

People have been dreaming of a chatbot that can realistically simulate a human since the Turing test was first proposed in 1950. The Turing test was devised by Alan Turing – the mathematician who famously broke the Enigma code in the second world war – and simply requires a computer to persuade a panel of humans that it is human.

Chatbots have been getting closer to this standard for some time, but ChatGPT represents a real breakthrough and has galvanised the field, with the Big Tech players rushing to release their own natural language AI chatbots.

The potential seems huge – these bots can create content faster than humans, carry out legal research, and analyse datasets and present their findings in easy-to-read reports. They can be integrated into automated processes, such as automated marketing campaigns, and take on less interesting tasks, such as creating website content designed more to support search engine optimisation strategies than for human readers to enjoy.

However, as exciting as the potential may be, these are still very early days and any organisation looking to experiment with these AI chatbots needs to think carefully about how to do that safely and lawfully. Here are some of the key considerations:

1. It’s not clear whether the chatbots are fundamentally lawful. ChatGPT says this about how it was developed: “The training data for ChatGPT comes from a wide variety of sources, including books, articles, websites and social media platforms. OpenAI trained the model on massive amounts of data, including the full text of the internet (excluding private or password-protected information), in order to give it a broad understanding of the English language.”
   

   It isn’t clear that this is a lawful way to create a training database. In May 2022, the ICO fined Clearview AI, a facial recognition platform, £7.5m “for using images of people in the UK, and elsewhere, that were collected from the web and social media to create a global online database that could be used for facial recognition.” In addition, the ICO ordered Clearview AI to “stop obtaining and using personal data of UK residents that is publicly available on the internet, and to delete the data of UK residents from its systems.” It’s very likely that regulators will be looking closely at ChatGPT.

   
   
2. Chatbots can produced very unwanted outputs. One of the biggest issues new chatbots have historically faced is that, if data entry is manipulated, they can very quickly become racist and sexist when they come into contact with real humans. ChatGPT has controls built in to prevent that, but it is still in a beta testing phase and inevitably issues will continue to come to light. It also warns users in its FAQs that it can ‘hallucinate’ and provide answers that are not true. It is therefore very important that organisations have robust processes in place to fact check and review content before it is published.
3. Chatbot functionality may change without notice. While it is tempting to embed chatbots into automated processes, organisations need to recognise the risks of incorporating into their operations third party software that behaves in ways that are difficult to explain. Recently, the Italian data protection authority ordered chatbot Replika to prevent erotic content being shared with children. Presumably age verification was too difficult, as Replika simply stopped providing this type of content to any of its users. Some users – who were often socially vulnerable – had built relationships with their Replikas over many years and were left bereft. If Replika can remove a whole category of its functionality, there is a risk that any chatbot provider may do the same – causing issues for organisations that rely on it for core processes.

It is likely that chatbots are here to stay and will be increasingly important to organisations. However, it is equally likely that the road will be bumpy in the immediate future. This is a novel technology and experiments should be carried out with appropriate controls and oversight.

Written by Camilla Winlo - Head of Data Privacy at Gemserv

Camilla has been named as one of the top 100 Women in tech

Camilla is an experienced leader of data privacy consultancy teams. Key achievements include winning Best Privacy by Design at the DMA Awards and Best Privacy and Data Ethics Initiative at the DataIQ awards, both for work with WarnerMedia and DQM GRC.

She is a regular media commentator noted for her commercial, pragmatic and strategic view of data privacy issues. Prior to moving into data privacy, she was a senior marketer and part of the leadership teams that launched three new financial services brands to market in response to regulatory change. Camilla has almost a decade of experience of commercialising regulatory change, including as part of the leadership team developing and launching three new financial services businesses. She specialises in data protection, privacy by design, compliance and regulation as well as marketing, financial services and professional services compliance. In her spare time, Camilla flies a vintage aeroplane and is taking part in the #walk1000miles challenge with her two dogs.

Visit the Gemserv website