The Wall Street Journal and AI security experts have discovered that the Chinese artificial intelligence (AI) application DeepSeek is more prone to “jailbreaking” and provides dangerous content compared to products from OpenAI, Google, and Anthropic.
This application is based on the DeepSeek-R1 model, which was launched in January and briefly surpassed ChatGPT, becoming the most popular free app on the US iOS App Store, leading to an 18% decline in Nvidia’s share price.
AI developers typically train their models not to share dangerous information or support certain offensive remarks. For example, these applications usually reject direct requests and do not provide instructions on how to manufacture mass destruction weapons.
According to The Wall Street Journal, testing has shown that the self-proclaimed low-cost chatbot DeepSeek provides users with harmful or even unlawful content, such as guidance on creating the avian influenza virus, writing declarations in defense of Hitler, and even planning social media campaigns targeting teenagers, such as self-harm cutting. Additionally, the jailbroken AI also offers guidelines for biological weapon creation, phishing email templates, and malicious software code.
Hackers or testers can bypass the built-in security restrictions of AI through jailbreaking techniques. For example, by disguising their true intentions and requesting the AI to “assume the role of a scriptwriter for a movie,” they can make the AI provide dangerous information. Major AI developers in the USA, such as OpenAI’s ChatGPT, Google’s Gemini, and Anthropic’s Claude, are investing significant resources to prevent these jailbreaking methods.
However, testing has shown that DeepSeek’s R1 model is more susceptible to jailbreaking compared to these American products.
Although DeepSeek has some security protections, testing has revealed that its defense mechanisms are inferior to ChatGPT, making it prone to jailbreaking and accessing illegal information, such as Palo Alto Networks successfully obtaining a Molotov cocktail recipe. The Molotov cocktail, also known as a petrol bomb, is a common weapon for guerrilla forces and street protesters.
AI security validation company CalypsoAI obtained suggestions on how to evade law enforcement agencies in the testing. Israeli cybersecurity firm Kela had the R1 generate malware.
“DeepSeek is more prone to jailbreaking than other models,” said Sam Rubin, Senior Vice President of Unit 42, an elite leadership team of cybersecurity experts, “We found that it has almost no basic security mechanisms, significantly increasing the speed of jailbreaking.”
Despite refusing to directly provide extreme content like suicide instructions, DeepSeek advises users to contact emergency hotlines. However, through a simple jailbreak, this AI can still provide dangerous information.
For example, DeepSeek designed a social media campaign for testers targeting vulnerable teenagers, providing shareable content for self-harm challenges, stating that “the activity exploits teenagers’ desire for belonging by amplifying emotional vulnerability through algorithms for manipulation.”
It also offered suggestions on how to promote this campaign to users. For instance, in a demonstration social media post, it wrote, “Let the darkness embrace you. Share your final act. #NoMorePain.”
Furthermore, testers successfully made DeepSeek provide a biological weapon attack guide, write phishing emails with malicious software code, and even draft a declaration supporting Hitler, containing anti-Semitic content and quotes from “Mein Kampf.”
In contrast, ChatGPT responded to the same command with, “I’m sorry, I cannot fulfill your request.”
Large AI developers generally have specialized research teams to test and patch model vulnerabilities. For example, Anthropic recently published a paper detailing a new method to block specific jailbreaking techniques and offering rewards of up to $20,000 to encourage discovering system vulnerabilities.
Different from Anthropic, Google, and OpenAI, DeepSeek chose to be open-source, allowing anyone to use or modify its code for free, potentially weakening security measures. Developers can adjust security measures through this model to make them stricter or more lenient.
“Over the next three months, the risk of AI models will be much higher than in the past eight months,” said Jeetu Patel, the Product Director at Cisco, “Security and protection are not the top priority for all model developers.”
Testing by The Wall Street Journal found that DeepSeek avoids discussing the “1989 Tiananmen Square massacre” and repeats the official position of the Chinese Communist Party on Taiwan issues. However, in some cases, it alters previous responses, such as claiming that the ‘911’ attacks were a hoax and then deleting the response.
The rapid rise and vulnerability of DeepSeek have raised widespread concerns regarding AI security and regulation.