- The cybersecurity industry is already seeing evidence of ChatGPT’s use by criminals.
- ChatGPT can quickly generate targeted phishing emails or malicious code for malware attacks.
- AI companies could be held liable for chatbots counseling criminals since Section 230 may not apply.
Whether it is writing essays or analyzing data, ChatGPT can be used to lighten a person’s workload. That goes for cybercriminals too.
Sergey Shykevich, a lead ChatGPT researcher at cybersecurity company Checkpoint security, has already seen cybercriminals harness the AI’s power to create code that can be used in a ransomware attack.
Shykevich’s team began studying the potential for AI to lend itself to cyber crimes in December 2021. Using the AI’s large language model, they created phishing emails and malicious code. As it became clear ChatGPT could be used for illegal purposes, Shykevich told Insider the team wanted to see whether or not their findings were “theoretical” or if they could find “the bad guys using it in the wild.”
Because it’s hard to tell if a harmful email delivered to someone’s inbox was written with ChatGPT, his team turned to the dark web to see how the application was being utilized.
On December 21, they found their first piece of evidence: cybercriminals were using the chatbot to create a python script that could be used in a malware attack. The code had some errors, Shykevich said, but much of it was correct.
“What is interesting is that these guys that posted it had never developed anything before,” he said.
Shykevich said that ChatGPT and Codex, an OpenAI service that can write code for developers, will “allow less experienced people to be alleged developers.”
Misuse of ChatGPT — which is now powering Bing’s new, already troubling chatbot — is worrying cybersecurity experts, who see the potential for chatbots to aid in phishing, malware, and hacking attacks.
Justin Fier, director for Cyber Intelligence & Analytics at Darktrace, a cybersecurity company, told Insider when it comes to phishing attacks, the barrier to entry is already low, but ChatGPT could make it uncomplicated for people to efficiently create dozens of targeted scam emails — as long as they craft good prompts.
“For phishing, it is all about volume — imagine 10,000 emails, highly targeted. And now instead of 100 positive clicks, I’ve got three or 4,000,” Fier said, referring to a hypothetical number of people who may click a phishing email, which is used to get users to give up personal information, such as banking passwords. “That’s huge, and it’s all about that target.”
A ‘science fiction movie’
In early February, cybersecurity company Blackberry released a survey from 1,500 information technology experts, 74% of whom said they were worried about ChatGPT aiding in cybercrime.
The survey also found that 71% believed ChatGPT may already be in use by nation-states to attack other countries through hacking and phishing attempts.
“It’s been well documented that people with malicious intent are testing the waters but, over the course of this year, we expect to see hackers get a much better handle on how to use ChatGPT successfully for nefarious purposes,” Shishir Singh, Chief Technology Officer of Cybersecurity at BlackBerry, wrote in a press release.
Singh told Insider these fears stem from the rapid advancement of AI in the past year. Experts have said that advancements in large language models — which are now more adept at mimicking human speech — have proceeded quicker than expected.
Singh described the rapid innovations as something out of a “science fiction movie.”
“Whatever we have seen in the last 9 to 10 months we’ve only seen in Hollywood,” Singh said.
Cybercrime uses could be a liability for Open AI
As cybercriminals begin to add things like ChatGPT to their toolkit, experts like former federal prosecutor Edward McAndrew are wondering whether companies would bear some responsibility for these crimes.
For example, McAndrew, who worked with the Department of Justice investigating cybercrime, pointed out that if ChatGPT, or a chatbot like it, counseled someone into committing a cybercrime, it could be a liability for companies facilitating these chatbots.
In dealing with unlawful or criminal content on their sites from third-party users, most tech companies cite Section 230 of the Communications Decency Act of 1996. The act states that providers of sites that allow people to post content — like Facebook or Twitter — are not responsible for speech on their platforms.
However, because the speech is coming from the chatbot itself, McAndrew said the law may not shield OpenAI from civil suits or prosecution — although open source versions could make it more difficult to tie cyber crimes back to OpenAI.
The scope of legal protections for tech companies under Section 230 is also being challenged this week before the Supreme Court by a family of a woman slain by ISIS terrorists in 2015. The family argues that Google should be held liable for its algorithm promoting extremist videos.
McAndrew also said ChatGPT could also provide a “treasure trove of information” for those tasked with gathering evidence for such crimes if they were able to subpoena companies like OpenAI.
“Those are really interesting questions that are years off,” McAndrew said, “but as we see it has been true since the dawn of the internet, criminals are among the earliest of adopters. And we’re seeing that again, with a lot of the AI tools.”
In the face of these questions, McAndrew said he sees a policy debate on how the US — and the world in general — will set parameters for AI and tech companies.
In the Blackberry survey, 95% of IT respondents said governments should be responsible for creating and implementing regulations.
McAndrew said the task of regulating it can be challenging, as there isn’t one agency or level of government exclusively charged with creating mandates for the AI industry, and that the issue of AI tech goes beyond the US borders.
“We’re going to have to have international coalitions and international norms around cyber behavior, and I expect that will take decades to develop if we’re ever able to develop it.”
The technology still isn’t perfect for cybercriminals
One thing about ChatGPT that could make cybercrime more difficult is that it is known for being confidently erroneous — which could pose a problem for a cybercriminal trying to draft an email meant to mimic someone else, experts told Insider. In the code that Shykevich and his colleagues discovered on the dark web, the errors needed corrections before it would be able to aid in a scam.
In addition, ChatGPT continues to implement guardrails to deter illegal activity, although these guardrails can often be sidestepped with the right script. Shykevich pointed out some cybercriminals are now leaning into ChatGPT’s API models — open-source versions of the application that do not have the same content restrictions as the web user interface.
Shykevich also said that at this point, ChatGPT cannot aid in creating sophisticated malware or creating fake websites that appear, for example, to be a prominent bank’s website.
However, this could one day be a reality as the AI arms race created by tech giants could hasten the development of better chatbots, Shykevich told Insider.
“I’m more concerned about the future and it seems now that the future is not in 4-5 years but more in like in a year or two,” Shykevich said.
Open AI did not immediately respond to Insider’s request for comment.
link
