Continuous vs Annual Penetration Testing: The Eleven-Month Blind Spot

The annual penetration test is a compliance artifact being asked to do a security job. It is very good at the first task. A report dated within the last twelve months, with findings and evidence of remediation, satisfies SOC 2, ISO 27001, and customer security reviews. What it cannot do is tell you anything about the eleven months it does not cover.

That gap is not a theoretical concern. If you ship weekly, your cloud footprint changes daily, and your vendor integrations change under you, then the environment described in your report stopped existing shortly after the tester wrote it.

What actually changes between tests

A new subdomain pointed at a staging box that was never meant to be public. An API endpoint that shipped without the authorization decorator its neighbors have. A cloud role granted broadly during an incident and never narrowed. An acquisition whose infrastructure joined your network before anyone reviewed it. A vendor integration that now holds a token into your environment. Each of these is routine engineering, and each one changes your attack surface the day it lands.

An annual test finds whichever of these happen to exist during its window. Continuous testing finds them close to when they were introduced, which is also when they are cheapest to fix and when the engineer who made the change still remembers why.

How to structure it by asset class

Continuous does not mean testing everything constantly, which would be unaffordable and unnecessary. It means cadence set by change velocity and crown-jewel exposure. The structure we run looks like this.

External perimeter and network: continuous discovery with monthly validation of anything new, plus a semiannual deep assessment. Internal network and Active Directory: monthly credentialed posture checks, semiannual deep dives covering delegation abuse, certificate services paths, credential hygiene, and the lateral routes that turn one workstation into domain-wide access.

Web applications: tested at each release and in depth quarterly, tiered so revenue-critical applications get more operator hours than internal tools. APIs: tested when the contract changes, triggered by specification diffs, so effort lands on what actually moved rather than re-testing stable endpoints. Native iOS and Android: each release screened as a binary and in live traffic, with a full assessment semiannually.

Cloud control plane: continuous configuration drift detection plus a quarterly review that traces real privilege escalation paths rather than reciting misconfigurations. Identity: continuous, covering phishing resistance, MFA bypass classes, token handling, and federation trusts. And an annual full-scope red team over the top, semiannual where the risk profile warrants it.

The part vendors gloss over

A great deal of what is sold as continuous penetration testing is a vulnerability scanner with a dashboard and a quarterly call. Scanning is a legitimate input to discovery, and it is not testing. A scanner tells you a version string looks vulnerable. It does not tell you whether the issue is exploitable in your configuration, what an attacker reaches through it, or that three medium findings chain into a critical breach.

The question that settles it: how many operator hours per month does the program include, and who are the operators? If a proposal cannot answer that plainly, you are buying automation with a service wrapper. Automation should handle discovery and regression. Humans should do the exploitation, the chaining, and the business-logic work tooling cannot reach.

Retesting has to be included

A program that bills retesting as a change order quietly teaches everyone to skip verification, and unverified remediation is how the same finding appears in three consecutive reports. Findings should close on verified retest, never on an assertion that a fix shipped. Recurrence is worth tracking separately, because the same finding class returning is a process defect rather than a bug, and it points at a gap in code review or provisioning rather than at one careless commit.

Measure the program, not the findings

Finding counts fall as a program matures, which makes them a poor health metric and a dangerous target. The metrics that actually describe program health are mean time to detect by attack stage, mean time to remediate against severity SLAs, recurrence rate, attack surface delta per cycle, and self-detection rate, meaning the share of the offensive activity your own tooling caught first. That last number should rise every cycle. It is the closest available proxy for how a real incident would go.

You still need the annual report

Keep it. Auditors, customers, and insurers ask for a point-in-time report, and a continuous program produces one as a byproduct rather than as a separate scramble. That is the argument that usually unlocks budget: one program satisfies the compliance requirement you were already paying for and covers the rest of the year as well. If FedRAMP is in scope, the six mandatory attack vectors and the CA-8(2) red team exercise are additional obligations that coordinate with the same program, covered in the FedRAMP red team requirement.

Where BD Emerson fits

We run continuous penetration testing as an annual subscription scoped on asset count, asset classes, release cadence, and committed operator hours, with deep dives and the red team scheduled inside the term and retesting included. Network, web, API, mobile, cloud, and identity sit under one methodology, because attackers chain across those boundaries and separate vendors per asset class is how cross-domain attack paths stay invisible. The broader offensive security practice covers the red team and purple team tiers.