OpenAI, the maker of ChatGPT, has banned several accounts from China that attempted to use the AI tool for creating sales pitches and debugging code for a suspected social media surveillance company. A report by OpenAI reveals that the banned accounts were leveraging ChatGPT to develop and enhance an AI assistant designed to gather real-time data on anti-China protests in the US, UK, and other Western countries. This information was later passed to Chinese authorities.
OpenAI’s disclosure of these cases aims to highlight how authoritarian regimes may attempt to exploit US-developed, democratic AI for purposes that contradict democratic values.
Ben Nimmo, a principal investigator at OpenAI, told Bloomberg, “This is a troubling insight into how one non-democratic actor tried to use democratic or US-based AI for non-democratic objectives, based on the materials they were producing.”
The Microsoft-backed AI startup highlighted that the accounts in question also utilized other AI models to develop their code, including a version of Meta’s Llama. Meta has since stated that if its service was involved, it was likely just one of many AI models used, including those from Chinese providers.
What was the activity of these banned accounts?
The banned accounts reportedly used software called “Qianyue Overseas Public Opinion AI Assistant” to send surveillance reports to Chinese authorities, intelligence agents and Chinese embassy staff. The software is said to be specifically tailored to identifying online conversations in Western countries related to human rights demonstrations in China.
According to the threat report published by OpenAI, it is against the company’s policy to use its AI for communications surveillance or unauthorised monitoring of individuals, including “on behalf of governments and authoritarian regimes that seek to suppress personal freedoms and rights”.
In a statement to Bloomberg, Meta addressed the issue, noting that the global availability of AI models is increasing, and restrictive models may not be a significant barrier for bad actors, especially with China heavily investing in its own AI program. The company added, “China is already investing over a trillion dollars to outpace the US technologically, and Chinese tech companies are releasing their own open AI models just as quickly as US companies.”
The US government has previously expressed concerns about China’s use of artificial intelligence to spread misinformation, suppress its population, and compromise the security of the US and its allies.
