Black Box vs White Box vs Gray Box Penetration Testing

Black box, gray box, and white box penetration tests differ in how much the testers know before they start. A black box tester receives nothing beyond a target, such as a domain name or an IP range. A gray box tester receives partial knowledge, usually an account for each user role and basic documentation. A white box tester receives full knowledge: architecture diagrams, configuration access, and source code. Less knowledge shows you what an outsider sees and spends more of the budget on discovery, while more knowledge buys deeper findings in the same hours. Most engagements land on gray box, and PCI SSC guidance states that testers typically perform PCI DSS penetration tests as white box or gray box assessments. This guide compares the three approaches, shows what compliance frameworks expect, and gives a rule for splitting a fixed testing budget across them.

What black box, gray box, and white box testing mean

The three labels describe one dial: how much the tester knows about the target's internals on day one. NIST's glossary gives the three steps formal names. It defines black box testing, which NIST SP 800-53A Rev. 5 calls basic testing, as "a test methodology that assumes no knowledge of the internal structure and implementation detail of the assessment object." Gray box testing, also called focused testing, assumes some knowledge of that structure, and white box testing, or comprehensive testing, assumes explicit and substantial knowledge of it. NIST SP 800-53A also names the two properties the dial trades between: depth, which addresses the rigor and level of detail of testing, and coverage, which addresses its scope or breadth.

A second dial is easy to confuse with the first. NIST SP 800-115, the federal technical guide to security testing, uses "white hat" for overt testing run with the IT staff's knowledge and "black hat" for covert testing run without it. That dial measures who on your side knows a test is running, and it moves independently of what the testers know. A black box test can be fully announced, and a covert red team can start from an insider's foothold. Scope the two dials separately, because they answer different questions.

Black box vs gray box vs white box, side by side

The table compares the three approaches on the criteria buyers scope against, at the same scope and day rate.

How starting knowledge changes scope, hours, and findings

Every input you hand the testers removes hours of guessing and opens a class of findings that was out of reach. The four common inputs are nothing beyond the target, credentials, documentation, and source code.

Credentials have the largest effect. The OWASP Web Security Testing Guide, version 4.2, notes that in many cases testers receive one or more valid accounts, and for application testing, PCI SSC's Penetration Testing Guidance strongly encourages organizations to supply credentials so the tester can assume each required role. A second account matters as much as the first, because proving that one customer cannot read another customer's data takes two tenants. A tester cannot reliably demonstrate broken object level authorization, the first risk in the OWASP API Security Top 10 2023, with a single account.

Documentation removes mapping time. The PCI SSC guidance recommends giving the tester a network diagram of every segment in scope, a cardholder data flow diagram, the services and ports expected at the perimeter, and a description of how authorized users reach the cardholder data environment. Source code changes what the test can see at all. The OWASP guide states that there is no substitute for looking at the code, and a code review finds the missing authorization check on a route no screen calls and names the line to fix. In BD Emerson planning, a black box test of a modest application spends one to three of its five to ten tester days on discovery that a gray box test skips, and a white box review adds two to four days of code and design review. The size of the attack surface moves the first number, and the size of the code base and the quality of its documentation move the second.

Which approach do compliance frameworks expect?

No framework mandates a box color, but PCI DSS and FedRAMP describe what testers should receive, and both descriptions point to gray box or white box testing. The table summarizes each driver as of September 2026.

PCI DSS v4.0.1 Requirement 11.4 requires internal and external penetration tests at least once every 12 months and after significant changes, and it requires exploitable findings to be corrected and the test repeated. The standard leaves knowledge level to your documented methodology. PCI SSC's guidance, version 1.1 from September 2017, states that white box and gray box assessments produce more accurate results and that a black box assessment may require more time, money, and resources to meet PCI DSS. Segmentation tests under 11.4.5 follow the same 12-month cycle, performed by a qualified internal resource or a qualified external third party with organizational independence, who does not need to be a QSA or an ASV. BD Emerson is not a QSA company or an ASV; we perform the penetration and segmentation tests, and your QSA assesses the results.

The FedRAMP Penetration Test Guidance, version 3 dated June 30, 2022, makes credentialed testing explicit. For the Tenant to CSP Management System vector, cloud providers supply privileged-level accounts in production, and the tester works with the highest permissions available to customer users. For the Tenant-to-Tenant vector, the tester receives two full production customer tenants. A 3PAO with demonstrated penetration testing proficiency performs the test, or for an agency authorization, an assessment organization the agency designates. BD Emerson performs FedRAMP readiness and penetration testing work and is not a 3PAO, so an independent assessor performs the assessment.

SOC 2, HIPAA, and cyber insurers set no knowledge level. The AICPA Trust Services Criteria name penetration testing in a CC4.1 point of focus as one type of separate evaluation. The HIPAA Security Rule requires a risk analysis under 45 CFR 164.308(a)(1)(ii)(A) and periodic evaluation under 164.308(a)(8), and the proposed rule HHS published on January 6, 2025 would add a penetration test at least once every 12 months; as of September 2026 it is not final. CFC's US cyber supplementary application lists penetration tests and vulnerability scans as separate controls and asks who provides each.

Why do most engagements land on gray box?

Gray box testing puts the budget where the findings are and starts the tester where many attackers start. Verizon's 2026 Data Breach Investigations Report, built on more than 22,000 confirmed breaches, found exploitation of vulnerabilities was the most common initial access vector at 31 percent, while phishing accounted for 16 percent and credential abuse for 13 percent. The first figure argues for testing what is exposed from outside. The other two describe attackers who arrive with a working account, which is the gray box starting point.

Gray box also buys more testing per dollar. A black box tester spends early days rebuilding knowledge your team already holds, while a gray box report ties each finding to a role, an object, and a request that an engineer can replay with the test account. Black box keeps its place where the outsider's view is the point, such as finding internet-facing hosts nobody listed, and white box earns its hours on payment flows, authentication services, and tenant isolation code.

How to choose an approach for each goal and asset

BD Emerson's rule is to start at gray box, remove knowledge only when the question is what an outsider can see, and add code and design access when the system is custom-built and its failure would be expensive. The table applies the rule to five asset types.

For networks, NIST SP 800-115 says internal testers receive the network information someone with their level of access would normally have, which is the gray box starting point of an assumed breach. For AI systems, NIST AI 100-2 E2025 defines white box attacks as having full knowledge of the system, including training data, model architecture, and hyperparameters, and black box attacks as having query access only, and it calls black box attacks the most practical for an adversary. For an application built on a hosted model, hand testers the system prompt, the tool definitions and permissions, and the retrieval sources, because the OWASP Top 10 for LLM Applications 2025 states that a system prompt should not be considered a secret or used as a security control. Our web application penetration testing applies the same defaults for roles and tenants.

How does knowledge level change the report?

The knowledge you give testers sets how specific the fixes in the report can be. A black box report describes symptoms, such as an exposed service or a verbose error, and it can say what an outsider sees without saying why the flaw exists. A clean black box report means the testers found nothing in the hours they had without access, which is weaker evidence than it looks. A gray box report ties each finding to a role and an object, so engineers reproduce it and confirm the fix. A white box report names the root cause, such as the route missing an authorization check or the configuration flag, and one root-cause fix often closes several findings at once.

Retest at the same knowledge level as the original test. PCI DSS 11.4.4 requires the test to be repeated to verify corrections, and a retest without the accounts or code access the original finding needed cannot confirm the fix.

A rule for splitting a fixed testing budget

Allocate testing days by asset risk and let the rule above set each line's knowledge level. BD Emerson's planning rule puts about two-thirds of testing days into gray box work across everything in scope, about a quarter into white box review of the one or two systems whose compromise would cost the most, and about a tenth into black box discovery of the external attack surface, with retest days reserved on top. If you cannot name the system that deserves the white box days, spend them on gray box coverage until you can.

Consider a 250-person SaaS company with a $60,000 annual testing budget, one multi-tenant web application with three roles, a public API, a payment service its own engineers wrote, about 40 internet-facing hosts, and one office on a single Active Directory domain. At $2,000 a tester day, the midpoint of the $1,500 to $2,500 rate in our penetration testing cost analysis, the budget buys 30 tester days. Reserving four for retests, about 15 percent of the 26 testing days, keeps the retest inside the 10 to 20 percent that analysis expects.

The split comes out at 3 black box days, 17 gray box days, and 6 white box days, which is 12, 65, and 23 percent of testing time. Every line sits inside the ranges in our cost analysis, and the deepest testing lands on the code that moves the company's money.

How this relates to internal vs external testing and red teaming

Knowledge level is one of three scoping choices. The second is the starting point: an external test starts from the internet with no credentials, and an internal test starts as an assumed breach from a workstation or VPN, as our comparison of internal vs external penetration testing explains. External tests usually run black box and internal tests gray box, and either can run at any knowledge level. The third is awareness: a penetration test is usually announced and built for coverage, while a red team is covert and built to measure detection, which we compare in red team vs penetration testing. Scanning has its own version of the knowledge dial, and PCI DSS 11.3.1.2 requires internal vulnerability scans to run with authenticated access. Our guide to vulnerability assessment vs penetration testing explains where a scan ends and a test begins.

Frequently asked questions

What is the difference between black box and white box penetration testing? A black box penetration test gives the testers nothing beyond a target, such as a domain or an IP range, so it shows what an outsider with no access can find. A white box test gives them full knowledge, including architecture documents, configuration access, and source code, so it finds flaws invisible from outside and names their root cause.

What is gray box penetration testing? Gray box penetration testing gives the testers partial knowledge before they start, usually an account for each user role, a second tenant for isolation testing, and diagrams or API specifications. NIST calls it focused testing. It simulates a customer, an employee, or an attacker with stolen credentials, and it is the most common choice for compliance and customer-driven tests.

Is black box or white box testing better? The better choice depends on the question. Choose black box to see what an outsider can find on your external attack surface. Choose white box for custom code whose failure would be expensive, such as payment or identity services, because it finds root causes. For most applications, APIs, and internal networks, gray box gives the best coverage for the budget.

Does PCI DSS require white box penetration testing? No. PCI DSS v4.0.1 Requirement 11.4 requires internal and external penetration tests at least once every 12 months and after significant changes, under a documented methodology, without naming a knowledge level. PCI SSC's Penetration Testing Guidance says PCI DSS tests are typically white box or gray box assessments and strongly encourages organizations to supply credentials for each role.

Is a black box penetration test more realistic? A black box test is realistic for one kind of attacker: an outsider with no access. Many breaches start with a working account instead, and Verizon's 2026 Data Breach Investigations Report attributes the initial access in 16 percent of breaches to phishing and 13 percent to credential abuse. A gray box test models those attackers directly, while a black box external test covers exposed services.

What should you give testers for a gray box test? Give the testers an account for every role in scope, a second tenant with matching roles if your product is multi-tenant, and the documents that show how the system fits together, such as network diagrams, data flow diagrams, and API specifications. Name an engineer who can answer questions during the test, because blocked testers spend paid hours waiting.

How much more does white box testing cost? White box testing adds code and design review to a gray box scope. In BD Emerson planning, a white box review of one service adds two to four tester days, which is $3,000 to $10,000 at the $1,500 to $2,500 day rate in our cost analysis. The size of the code base and the number of languages in it move the number within that range.

If you are deciding what to hand your testers, our penetration testing team can map each system in scope to a knowledge level, list the accounts and documents each test needs, and price the retest into the same engagement.