AI Cybersecurity Fundamentals

Three cases land on your desk in one morning. A finance clerk released a payment because the voice on the call sounded exactly like the chief financial officer. A newly deployed email classifier is quarantining legitimate purchase orders. A support assistant read a customer's attached PDF, followed an instruction hidden inside it, and mailed another customer's data out. All three arrive labelled as AI problems, and no single control fixes more than one of them. What separates them is what the model was in each case: the attacker's tool, the defender's tool, or the target of the attack itself. Add the model itself, the machine all three cases run on, and there are four roles in all. This domain owns three of them, one page each: the model itself, the defender's tool and the attacker's tool. It also supplies the words the rest of the exam then uses without re-explaining. A model is the artifact that training produces, inference is that artifact being run on a new input, and an AI system is, in the wording of the AI Risk Management Framework published by the US National Institute of Standards and Technology (NIST), "an engineered or machine-based system that can, for a given set of objectives, generate outputs such as predictions, recommendations, or decisions influencing real or virtual environments"; the same passage adds that "AI systems are designed to operate with varying levels of autonomy". The remaining role, the model as target, is where Domain 2 begins, and the trap this routing question dodges is the answer that treats every mention of AI as one subject: filtering a chatbot's prompts does nothing about a poisoned training set, and teaching staff to spot clumsy grammar does nothing about a cloned voice.

Read the three pages in order, because each one spends the vocabulary the one before it earned. AI Principles and Terminology teaches each term through its security consequence: which learning paradigm a system uses and therefore what an attacker has to reach to steer it, the split between the training stage and the deployment stage that NIST's adversarial machine learning taxonomy uses to sort attacks, why a classifier hands back a score that somebody has to threshold, and why a large language model (LLM) cannot tell your instructions apart from a document you told it to read, the missing separation behind prompt injection, which the 2025 edition of the OWASP Top 10 for LLM Applications ranks first. AI Security Use Cases points that vocabulary at your own environment: which security problems a model actually fits, judged by the data you already hold, and the counter-case where a deterministic check that explains itself beats any classifier. AI-Driven Threats turns the same technology around and asks what an attacker gains from it, which parts of the attack chain measurably changed, and which claims are unverifiable marketing. Reach for the first page when a term in the scenario is doing the real work, the second when the question is whether to deploy a model at all, and the third when the question is what an attacker's use of AI changed for you. At 17% this is the smallest of the four published domains, and the largest, AI Systems Security at 40%, spends its whole weight on ideas defined here.

Both applied pages in this domain turn on the same economics, seen from opposite sides. A model earns its place for a defender exactly where the variation outran the cost of writing and maintaining exact rules, and it earns nothing where the condition can be stated exactly and then holds still. For an attacker the mirror image holds, and it has been assessed rather than guessed: the UK National Cyber Security Centre (NCSC) graded AI capability uplift phase by phase and judged that AI provides capability uplift in reconnaissance and social engineering, almost certainly making both more effective, more efficient and harder to detect, while rating the gain in malware and exploit tooling as minimal uplift for capable state actors and organised cyber crime. Its later assessment to 2027 judges fully automated, end-to-end advanced attacks unlikely to 2027, with skilled actors still needed in the loop. Nothing in either assessment invents an attack that did not exist before, which is why the exam-rewarded answer names the specific control whose margin was priced on the old cost, the awareness training that assumed clumsy language or the patch window that assumed a slow exploit, rather than a brand new category of defence.

Each page in this domain reaches the same rule from a different direction, which is the surest sign it is the domain's real content. A classifier returns a score, and the threshold that converts that score into an alert is a setting your team owns, so a hit opens an investigation rather than closing one. A generative model returns text shaped to approximate the statistical distribution of its training data, which is why NIST names confabulation, confidently stated but erroneous content, as a design property of these systems rather than as evidence of an attack. On the attacker's side the same fact inverts into a warning: a fluent email, a familiar voice and a face on a video call are generated output too, produced by the activity MITRE ATT&CK catalogues as Generate Content (T1683), so none of them carries proof of who is behind it. The working default is to look for the confirming check that does not itself come from a model, and to treat an answer that lets model output trigger something irreversible, a payment, a credential reset, a block on production traffic, without such a check as the distractor.