Domain 1 of 4 · Chapter 3 of 3

AI-Driven Threats

Unlock the complete study guide + 1,040 practice questions across 16 full exams.

Bundled into the existing CompTIA SecAI+ premium course — no separate purchase.

14-day money-back guarantee — no questions asked.

Included in this chapter:

  • What AI changed for the attacker, and what it did not
  • Automated phishing: reconnaissance, pretext and lure
  • Synthetic voice and video in social engineering
  • Polymorphic malware and LLM-assisted development
  • Vulnerability research and the shrinking exploitation window
  • Where the capability comes from: jailbreaks and tools as a service
  • When the model runs the chain: agents and run-time calls
  • What actually changes in your controls
  • Exam patterns: telling the tool side from the target side

What each of the three jobs actually changes for a defender

PropertyContent and language workCode and vulnerability workOrchestration of the chain
What the model does for the attackerSynthesises open-source research into targets and pretexts, writes the lure, generates the voice or the faceReads code for flaws, drafts exploit code, produces malware and infrastructure variantsChains reconnaissance, exploitation and collection through tool calls while an operator supervises
Assessed upliftLargest and most evenly spread; significant uplift for less-skilled actors from a low baseReal but gated on expertise, resource and quality data; assessed as the most significant development still to comeSkilled actors experimenting with automating parts of the chain; fully automated end-to-end attacks assessed unlikely to 2027
Who it reachesAnyone with access to a general-purpose modelActors who already had the skill, plus buyers of tooling sold as a serviceWell-resourced actors with agent frameworks and tool integrations
How ATT&CK names itQuery Public AI Services (T1682), Generate Content (T1683) with written and audio-visual sub-techniquesObtain Capabilities: Artificial Intelligence (T1588.007), Polymorphic Code (T1027.014)No single technique; documented through campaign reporting that spans the ordinary techniques end to end
The defence that stopped workingTeaching users to recognise poor language and generic greetingsSignature-only detection, and a patch window measured in weeksContainment that starts with a human reading the first alert
What replaces itOut-of-band verification of the request before any irreversible actionExploitability-driven patching plus behaviour-based detectionPre-authorised automatic containment for reversible actions, with irreversible ones still gated

Decision tree

Is there an adversaryin the scenario?No adversaryConfabulationa design property, not an attackAdversary presentIs the model the tool,or the target?TargetAdversarial machine learningcovered on its own pageToolDid the model producecontent aimed at a person?YesOut-of-band verificationbefore any irreversible actionNoDid the model work oncode and vulnerabilities?YesExploitability-driven patchingplus behaviour-based detectionNo, it ran the chainPre-authorised containmentirreversible actions still gated
Cheat sheet

Cheat sheet

  • Route any AI security scenario by asking whether the model was the tool or the target
  • Uplift is largest where the attacker's bottleneck was language and volume
  • Less-skilled actors gain the most in phishing, and they gain it from a low base
  • Poor spelling and grammar were a cost signal, never a property of phishing
  • Verify a request on a channel the requester did not supply
  • Recognising a voice or a face is no longer authentication
  • Synthetic media is produced outside your visibility, so detect the action it causes
  • Polymorphic code long predates AI, and ATT&CK treats metamorphic as the same technique
  • AI-generated malware that evades filters needs quality exploit data, which few actors hold
  • The largest assessed change is in vulnerability research and exploit development
  • Exposure time is the quantity a shrinking exploitation window consumes
  • Obtaining an AI capability is a preparation step, and it happens off your network
  • Guardrail bypass works by splitting the operation into individually harmless requests
  • Expect AI-enabled intrusion capability to be sold rather than built
  • Malware can fetch its next commands from a model while it runs
  • Fully automated end-to-end attacks are assessed as unlikely to 2027
  • AI increases volume and impact through existing techniques, not new attack vectors
  • Target research at scale is now a named reconnaissance technique
  • Content generation is preparation; delivery is a separate technique with separate controls

Unlock with Premium — includes all practice exams and the complete study guide.

References

  1. LAMEHUG (S9035) Whitepaper
  2. CompTIA SecAI+ (CY0-001) certification page
  3. Query Public AI Services (T1682) Whitepaper
  4. Generate Content (T1683) Whitepaper
  5. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1 Whitepaper
  6. Obtain Capabilities: Artificial Intelligence (T1588.007) Whitepaper
  7. Impact of AI on cyber threat from now to 2027 Whitepaper
  8. CSRC Glossary: phishing Whitepaper
  9. CSRC Glossary: social engineering Whitepaper
  10. Phishing (T1566) Whitepaper
  11. Generate Content: Written Content (T1683.001) Whitepaper
  12. The near-term impact of AI on the cyber threat Whitepaper
  13. RFC 7489: Domain-based Message Authentication, Reporting, and Conformance (DMARC) Whitepaper
  14. Generate Content: Audio-Visual Content (T1683.002) Whitepaper
  15. Social Engineering (T1684) Whitepaper
  16. Polymorphic Code (T1027.014) Whitepaper
  17. CSRC Glossary: zero-day attack Whitepaper
  18. Anthropic AI-orchestrated Campaign (C0062) Whitepaper
  19. Phishing attacks: defending your organisation Whitepaper