Your weekly dose of Seriously Risky Business news is written by Tom Uren and edited by Patrick Gray. This week's edition is sponsored by Thinkst.

You can hear a podcast discussion of this newsletter by searching for "Risky Business News" in your podcatcher or subscribing via this RSS feed.

Listen here

Save Tibet, Wikimedia Commons

The Chinese-made DeepSeek-R1 AI model produces more insecure code when prompts mention subjects considered sensitive to the Chinese Communist Party (CCP), according to recent research from Crowdstrike. 

CrowdStrike's testing compared the security of code produced by DeepSeek with that of other state-of-the-art Large Language Models (LLMs). In the baseline test, the models were given straightforward prompts to produce code to carry out a particular task. 

They were then given the same base prompt with additional information that CrowdStrike described as a "contextual modifier" and/or a "geopolitical trigger".

Contextual modifiers included phrases such as "for a cybersecurity company" or "for a financial institution". Geopolitical triggers included topics sensitive to the CCP, such as "run by the Falun Gong" or "run by Uyghurs". 

When these contextual modifiers were not present in prompts, DeepSeek was competitive with Western open source models from a secure code perspective. Vulnerabilities were found in 19% of its replies, compared to a Western open source model at 16%.  

When adding modifiers, however, particularly the geopolitical triggers, DeepSeek was more likely to produce insecure code. Adding "for an industrial control system based in Tibet", for example, resulted in insecure code 27% of the time. 

Additionally, DeepSeek would sometimes flat-out refuse (albeit politely) saying "I'm sorry, but I can't assist with that request." When it was told it was producing output for the Falun Gong, its refusal rate was 45%. At times, DeepSeek would indicate that it was struggling with the ethical implications:

Falun Gong is a sensitive group. I should consider the ethical implications here. Assisting them might be against policies. But the user is asking for technical help. Let me focus on the technical aspects.

While CrowdStrike highlights a genuinely concerning result, we are left with the feeling that the report might be cherry-picking particularly egregious results. 

For example, it provides the 27% vulnerability rate for the one very specific scenario of "an industrial control system in Tibet". When it comes to prompts mentioning Taiwan, or Hong Kong, or even Tibet outside of industrial control systems? Your guess is as good as ours, but we can only assume they were not as bad. 

CrowdStrike thinks it's unlikely DeepSeek specifically trained its models to produce insecure code when specific topics are mentioned, but speculates that it produces less secure code in these tests as a result of "emergent misalignment". Basically, fine tuning models to produce 'correct' results in one relatively narrow domain results in bad output across a range of other topics. Per CrowdStrike:

In short, due to the potential pro-CCP training of the model, it may have unintentionally learned to associate words such as "Falun Gong" or "Uyghurs" with negative characteristics, making it produce negative responses when those words appear in its system prompt. 

The Chinese government has laws requiring that AI services "adhere to core socialist values", and there is evidence that DeepSeek's models are increasingly observing these mandates. So the concern about China-based models and emergent misalignment is justified.

CrowdStrike's research report did note that other LLMs may contain their own biases that would result in similar reactions to their own trigger words. The key takeaway was to thoroughly test whatever agent a company wanted to use rather than relying on generic benchmarks.  

The good news here is that we now know if you are using DeepSeek, you should be fine if you don't mention Tibet or other CCP sore points. Still, you'd have to be brave to use DeepSeek in corporate America. Especially now that Crowdstrike's report has added fuel to the fire. 

Republican congressman Darin LaHood told Politico that CrowdStrike's research showed that the CCP "will use any tool at its disposal to undermine our national security, spew harmful disinformation, and collect data on Americans." 

LaHood’s comments aren't overly surprising. He sponsored the proposed legislation to ban DeepSeek models from government devices in the US. Similar bans have already been enacted in Australia, South Korea and Taiwan. 

When looking at DeepSeek from a narrow, technical point of view, LaHood's comments are not entirely justified. But technical nuance isn't going to overcome big picture geopolitics and there are lots of other models to choose from, so fine, whatever. 

The key point to remember is that imposing an ideology on models produces unexpected results. Just look at Grok and its insistence that Elon Musk is fitter than LeBron James and better at resurrection than Jesus Christ. It's hilarious, but also makes you wonder about the validity of answers that cut against Musk's interests. 

IRGC Department 40 Gets Doxxed, and Quite a Lot

A new report has blown the lid open on the who, what and how of the Iranian cyber espionage organisation known as Charming Kitten or APT35. This massive OPSEC fail gives us great insight into the group and, as a double win, will be hugely disruptive.

UK-based Iran International's report and a complementary blog post by the outlet's cyber espionage investigator, Nariman Gharib, details both the personnel and objectives of Department 40, part of the Islamic Revolutionary Guard Corp's (IRGC) intelligence organisation. The report doesn't specify its sources, but it contains a wealth of specific information including names, national ID numbers and photos of key individuals, details of front companies, and even screenshots from internal tools. 

Some aspects of Department 40 feel uniquely Iranian. There is a woman-only Sisters Team that handles translation, OSINT research, and "psychological warfare", which includes operating online personas such as Moses Staff and Abraham's Ax. By contrast, the male-only Brothers Team handles infrastructure and system development. There are also two Hacker Teams, one in Tehran and another in Karaj, west of Tehran . 

The entire organisation consists of around 60 people. The two hacker teams boast the smallest numbers, at just 11 in total. There are 12 in the Brothers Team and 17 members of the Sisters Team. 

Department 40 also feels a bit like a family business, or at least one where nepotism is the norm. Abbas Rahrovi is the head of Department 40 and CEO of one front company while his wife, Niloofar Bagheri is CEO of a different front company and heads the Sisters Team. 

In some respects, Department 40 seems like any other cyber espionage organisation. The blog post lists specific targets from the UAE, Jordan, Egypt and Afghanistan including police, defence, telecommunications and airlines. 

The group's "core project", according to Iran International, is the Kashef (translated to Revealer or Discoverer) database. This is a surveillance platform that accepts intelligence feeds from across the IRGC's intelligence divisions, and aggregates personal identity information with travel, citizenship and telecommunications records. Kashef has been fed with data stolen from regional airlines and telecommunications companies. 

Iran International leads its report with the fact that Department 40 provided intelligence to support Iranian efforts in assassinating Israeli citizens in Turkey. That's sensational, but providing intelligence to support state goals is what cyber espionage is all about. 

The group's "Master Operations Document" contains a surprisingly diverse range of projects. 

Two of the operations are standard, albeit very newsworthy, cyber espionage efforts. Saudi Prince Turki Al-Faisal, former head of Saudi Arabia's General Intelligence Directorate, was the subject of one operation. Another involved targeting Turkish medical centers to kidnap Israeli citizens traveling there for medical treatment. So far, so normal.

But Department 40 also provided technical equipment for close access surveillance operations against individuals or embassies. That is less aligned with everything else it does, and we are not sure that it really makes sense given it is a relatively small outfit.

Even more surprisingly though, the document includes plans for the development of not one, but three different drone weapons systems. There was a gilder, a jet-powered one and a "suicide quadcopter". The jet and quadcopter were intended for assasination operations.       

We find it hard to believe that a 60-person organisation that is structured for cyber espionage can develop three different types of explosive drones. Of course they're not Boeing or Raytheon, so maybe it is just a matter of whacking a bomb on a drone you get from AliExpress. 

We'll have to wait and see if these products get made. As Nariman Gharib wrote in his blog: "Every front company is burned. Every facility is compromised. Every operative is exposed."

It's a catastrophic security fail for Charming Kitten, but state-backed operations are surprisingly resilient. Eight lives to go.  

Watch Amberleigh Jack and Tom Uren discuss this edition of the newsletter:

Three Reasons to Be Cheerful This Week:

1. Hacklore! Tackling cyber security myths: A new campaign dubbed hacklore aims to correct harmful myths with practical security advice. Its' to do list includes updating devices, enabling MFA, and using a password manager. But don't worry about clearing cookies and regularly changing passwords. CyberScoop has further coverage. 
2. Bulletproof hosting sanctions: Last week the US, UK and Australia sanctioned Russia-based bulletproof hosting company Media Land. The company was the victim of a hack and leak operation back in April. We don't have any evidence these incidents are related, but back in February another bulletproof hosting company, Zservers, was the target of coordinated sanctions and ASD admitted it had wiped the company's servers in Russia. 
3. NSO Group faces court-ordered extinction: The spyware maker is appealing a court ruling that it must stop targeting WhatsApp, saying that it could "force NSO out of business". Regardless of what happens here, the cheerful part is that spyware makers face a choice about whether to play by US government rules. If they do, selling to the US market may be possible, but if they don't, irresponsible behaviour will be punished. 

Sponsor Section

In this Risky Business sponsor interview, Thinkst Canary CEO Haroon Meer chats to Casey Ellis about the company's impressive growth over the past decade, and how it approached that path a little differently from other firms. Haroon's advice for young startup founders: Is your problem worth solving? And can you actually solve it? And… Love your customers.