Adversarial machine learning is cybersecurity's new frontier

NIST's latest report makes that reality plain, exposing the limits of today's AI security measures and highlighting a growing disconnect between how AI is deployed and how it's defended.

The AI systems we rely on are vulnerable.

I'll say that again: the AI systems that help run our cities and will increasingly support critical decisions - like choosing the best treatment after a medical diagnosis or deciding whether an aircraft should turn left or right - are vulnerable. NIST's latest findings make that reality uncomfortably clear.

What the new NIST report reveals

The National Institute of Standards and Technology (NIST) has released "Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations", its latest iteration in a growing body of guidance aimed at helping organizations understand, categorize, and confront the security challenges unique to AI systems. Not to mention its most thorough report yet on adversarial machine learning. And the verdict is stark: existing security measures for AI and ML systems are insufficient, inconsistent, and, in many cases, theoretical at best. The 2025 edition of its AI 100-2 report deconstructs the entire field of AI security, providing a comprehensive taxonomy of attacks on both predictive and generative AI systems, from poisoning and evasion to privacy compromises and supply chain vulnerabilities.

In addition to mapping the attack landscape, NIST raises concerns about how effective today's defenses really are. Most protective measures are patchwork efforts, solutions crafted through trial and error rather than grounded in strong guarantees. The report stresses that many widely used methods don't hold up when models face unexpected scenarios or scale beyond test environments. There's also a built-in tension between accuracy, robustness, and fairness. Prioritizing one often reduces the others.

Yesterday, this challenge was confined to academic research. Today, it's confronting engineering teams in deployment settings as we speak.

A growing threat in production environments

The urgency here reflects AI's expanding role in systems where failure isn't an option. AI is being fast-tracked into sectors where failures have real-world, irreversible consequences. And yet, even as these systems become more powerful and more embedded, their vulnerabilities are multiplying. Attackers have learned to exploit every stage of the machine learning lifecycle, from training to inference, by subtly manipulating inputs or influencing model behavior. These inputs can appear completely normal to a human, but they can cause AI systems to misfire in critical ways.

Because machine learning relies on statistical patterns rather than deterministic logic, it introduces a wider range of entry points for attackers. Weaknesses can exist in the data, in the way models are trained, or in how they respond to unpredictable input. And as AI systems become more complex, especially in multi-modal architectures or decentralized training environments, the challenges multiply. Add to this the use of open-source models and datasets, often opaque in provenance, and we're looking at a supply chain problem of epic proportions.

What it means for AI vendors

The report also casts doubt on how AI security is being marketed. Many vendors emphasize monitoring tools and detection strategies as core components of their safety frameworks. However, according to NIST, these tools often fall short. In other words, major AI vendors may be over-promising and under-delivering when it comes to AI security (I'm looking at you, Big Tech).

Many vendors tout reactive or detective controls as sufficient safeguards, but NIST begs to differ. Detection itself is inherently difficult, particularly because adversarial inputs often come from the same distribution as legitimate data. Which means, bad inputs closely resemble normal data. On top of that, more rigorous verification techniques are prohibitively expensive and not widely used.

There's also a reliability problem: defenses may work well in test labs but fall apart when exposed to real-world behavior. Without consistent evaluation standards (read: realiable benchmarks), it's hard to know whether a system is protected or simply unproven under pressure.

What needs to change

This is a wake-up call for the industry. The problem isn't just technical, it's structural. We must stop pretending the current solutions are good enough. The field of Adversarial Machine Learning remains largely empirical. Organizations deploying AI systems need to reassess their risk management strategies and move beyond generic safeguards. In highly regulated sectors especially, relying on vendor defaults or broad governance frameworks is inadequate, to say the least.

Security needs to be part of the development process from the start. And we need tailored, use-case specific controls. Security professionals must embed AI-specific threat modeling and risk assessments into the SDLC and third-party risk management processes. There is no universal playbook, each system must be evaluated on its own terms, according to the risks it introduces and the context in which it operates.

In plain words, AI governance frameworks (even ones as rigorous as ISO 42001) won't save you if your technical controls don't match your risk profile.

A turning point for cybersecurity

There's a certain elegance in how NIST handles this conversation. They have done the hard work of organizing a chaotic landscape and identifying where the real risks lie. Also, the report doesn't lean into panic, but into clarity. It names the problem, maps the terrain, and leaves us with one unavoidable truth: adversarial machine learning is not a subset of cybersecurity, it is the new frontier of it.

And like all frontiers, it is as exposed as it is promising. The sooner we accept that, the sooner we can start building AI systems worthy of our trust.