AI in Offensive Security: What Actually Works

Every security vendor now claims AI-powered testing, and most of the claims describe a scanner with a language model writing the summaries. Meanwhile the genuine change is real and less glamorous than the marketing: models have made it affordable to reason across far more context than a human team holds in an engagement window. That shifts the economics of coverage. It does not shift accountability for a finding one inch.

Where it genuinely helps

Attack-path reasoning. The hardest part of a large engagement is holding thousands of enumerated facts in mind simultaneously and noticing which combination matters. Given an asset inventory, a permission model, the code, and the findings so far, a strong model surfaces candidate chains that a human then tries. This is the highest-value use we have found, and it works because the model is doing recall and combination rather than judgment.

Code and infrastructure review at scale. Traditional application testing samples code, because reading all of it was never affordable. Reviewing an entire repository along with its Terraform and Kubernetes manifests is now practical. The recurring win is not exotic: it is the gap between what a policy document claims and what the infrastructure code actually provisions.

Recon synthesis. Certificate transparency logs, DNS history, cloud metadata, public repositories, job postings, and leaked credential corpora produce more signal than any team reads by hand. Correlating it reliably surfaces the forgotten subdomain and the acquisition that was never integrated. Specification diffing then focuses application testing on exactly what changed since last cycle.

Where it fails, specifically

Confident false positives at scale. A model will describe an exploitable authorization flaw in convincing technical detail, complete with a plausible reproduction path, for code that is in fact correctly guarded three layers up. It does this without any signal that it is wrong, which is exactly the failure mode offensive work punishes hardest.

An unvalidated AI finding list is worse than no list. It consumes the client's remediation capacity on issues that were never exploitable, and after the second such report the client stops trusting the real findings too. This is the whole reason the human gate exists, and it is where most of our operator time goes.

Models are also weak at anything requiring persistence against a live target, at knowing when to stop, and at the tacit judgment about business impact that separates a Medium from a Critical. They are bad at recognizing that a technically valid finding does not matter because of a compensating control they were not shown.

The gate that makes it usable

Four rules we do not bend. No autonomous exploitation: AI proposes, operators execute. No finding reaches a report without human reproduction and captured evidence. No severity or CVSS vector is model-assigned without operator review. And anything that could affect production availability requires named operator sign-off against the rules of engagement.

Model output goes into a triage queue, never into the report. Candidates that cannot be reproduced are discarded rather than softened into a hedged maybe. We also track the validation pass rate of model-proposed candidates by engagement class, which is the measurement that separates AI as a capability from AI as a marketing line. Where a technique produces noise, it gets dropped rather than defended.

Why self-hosted models matter for this work

Three reasons, in priority order. First, confidentiality: engagement material includes client source code, traffic captures, credentials, and unremediated findings, which is among the most sensitive data a company has. Running open-weight models on our own infrastructure inside an isolated lab means the answer to where your code goes is a network diagram rather than a vendor policy, and that distinction survives a security review.

Second, operational reliability: commercial API terms are written for general-purpose use and can interrupt legitimate authorized security testing mid-engagement, which is unacceptable when a test window is contractually fixed and a client's change freeze is scheduled around it. Third, cost at volume: local inference means budget never quietly decides how much of your codebase gets read.

Where a frontier model adds real value on reasoning, it operates on abstracted or sanitized inputs rather than raw client artifacts. That routing is deliberate, and it is the same discipline described in smart model routing.

The other half: testing AI systems

If you have shipped agents, retrieval pipelines, or model-backed features, they carry a new attack surface, and testing them is a distinct discipline from web application testing. The recurring issues are prompt injection that reaches real tools and data, retrieval sources an attacker can poison, agent permissions scoped far wider than the task requires, output paths that treat model text as trusted instruction, and human checkpoints that exist in the governance policy but not in the code. Findings map to whatever framework governs your AI program, usually ISO/IEC 42001 or the NIST AI Risk Management Framework.

What to ask a vendor claiming AI-powered testing

Who validated the findings, and how? Where does our source code go, and can you show me the network diagram? What is your validation pass rate on model-proposed candidates? And does anything execute autonomously against our environment? The last answer should be no. If someone offers the same depth for a fraction of the usual price, the savings are coming out of the validation step.

Where BD Emerson fits

Our AI-augmented offensive security service pairs frontier reasoning with self-hosted inference in an isolated lab, and every finding is reproduced and evidenced by a human operator before it reaches you. Because the same firm runs enterprise AI implementations and AI governance programs, our operators know how these systems get built and which controls exist only on the architecture diagram. The wider methodology sits in the offensive security practice.