{"id":57283,"date":"2024-10-03T00:00:00","date_gmt":"2024-10-03T00:00:00","guid":{"rendered":"urn:uuid:d6e79b22-6478-3b4a-bc18-daefd14868ce"},"modified":"2024-10-03T00:00:00","modified_gmt":"2024-10-03T00:00:00","slug":"rogue-ai-what-the-security-community-is-missing","status":"publish","type":"post","link":"https:\/\/www.threatshub.org\/blog\/rogue-ai-what-the-security-community-is-missing\/","title":{"rendered":"Rogue AI: What the Security Community is Missing"},"content":{"rendered":"

<\/p>\n

<\/div>\n

MITRE ATLAS<\/span><\/p>\n

MITRE\u2019s tactics techniques and procedures (TTPs) are a go-to resource for anyone involved in cyber-threat intelligence\u2014helping to standardize analysis of the many steps in the kill chain and enabling researchers to identify specific campaigns. Although ATLAS extends the ATT&CK framework to AI systems, it doesn\u2019t address Rogue AI directly. However, Prompt Injection, Jailbreak and Model Poisoning, which are all ATLAS TTPs, can be used to subvert AI systems and thereby create Rogue AI.<\/p>\n

The truth is that these subverted Rogue AI systems are themselves TTPs: agentic systems can carry out any of the ATT&CK tactics and techniques (e.g., Reconnaissance, Resource Development, Initial Access, ML Model Access, Execution) for any Impact. Fortunately, only sophisticated actors can currently subvert AI systems for their specific goals, but the fact that they\u2019re already checking for access to such systems should be concerning.<\/p>\n

Although MITRE ATLAS and ATT&CK deal with Subverted Rogues, they do not yet address Malicious Rogue AI. There\u2019s been no example to date of attackers installing malicious AI systems in target environments, although it\u2019s surely only a matter of time: as organizations begin adopting agentic AI so will threat actors. Once again, using an AI to attack in this way is its own technique. Deploying it remotely is like AI malware, and more besides\u2014just as using proxies with AI services as attackers is like an AI botnet, but also a bit more.<\/p>\n

MIT AI Risk Repository<\/span><\/p>\n

Finally, there\u2019s MIT\u2019s risk repository, which includes an online database of hundreds of AI risks, as well as a topic map detailing the latest literature on the subject. As an extensible store of community perspective on AI risk, it is a valuable artifact.  The collected risks allow more comprehensive analysis. Importantly, it introduces the topic of causality, referring to three main dimensions:<\/p>\n