Penetration Testers: The Brutally Honest Guide to Hiring, Evaluating, and Getting Real Value from Pen Testing

From the field:

During an external pen test for a healthcare SaaS company, our recon phase uncovered a forgotten staging environment running on a subdomain that the client’s IT team did not know still existed. It was three major versions behind production. We had admin access within 20 minutes using a known exploit. The staging environment had a database connection to the production database. We could read every patient record. The “vulnerability” was a $50/month server that nobody remembered setting up two years earlier.

From the field:

During an external pen test for a fintech company, we found a forgotten Jenkins server exposed on a non-standard port. The server had default credentials. From Jenkins, we had access to deployment pipelines that pushed code to production. We could have deployed a backdoor to their payment processing system. The “vulnerability” was a $200/month server that nobody remembered existed - running with the factory admin credentials it shipped with.

From the field:

During a web application pen test for an e-commerce platform, we discovered that changing a single parameter in the checkout API request allowed us to set our own price for any product. We bought a $3,000 laptop for $0.01. We immediately reported it as a critical finding and reversed the transaction. The client’s developers had validated prices on the frontend but not the backend API. An automated scanner would never have found this - it is a business logic flaw, not a CVE.

From the field:

On an internal network pen test for a manufacturing company, we started with a standard employee workstation. Within six hours, we had domain admin rights through a Kerberoasting attack on a service account with a weak password. From domain admin, we accessed the SCADA network controlling their factory floor. We could have shut down production lines. The total investment to prevent this: a stronger service account password and network segmentation between IT and OT. Cost of a production shutdown: roughly $400,000 per day.

The Scanner Jockey Red Flag

If your pen tester’s “methodology” is “we run Nessus and give you the report,” you do not have a penetration tester. You have a subscription to a scanning tool with a human-shaped invoice attached. Real penetration testers use scanners as one input into a manual, human-driven testing process. The scanner finds candidates. The tester validates, exploits, and chains them into meaningful attack paths.

Atlant Security Pricing

At Atlant Security, penetration testing starts at $4,000 for a focused external assessment. Most web application tests run $5,000-$12,000. We provide fixed-price proposals within 24 hours - no hourly billing surprises, no scope creep charges. You know exactly what you are paying before we start.

What Drives Prices Up

Four factors consistently increase pen test pricing: (1) Scope expansion - more applications, more IPs, more environments; (2) Compliance requirements - PCI DSS and specific regulatory frameworks often dictate testing methodology and reporting format; (3) Retesting - always negotiate retesting into your original contract rather than paying separately; (4) Urgency - rush engagements (less than 2 weeks notice) typically carry a 25-50% premium.

The Six Components of a Quality Penetration Test Report

1. Executive Summary (1 page). Board-readable. No technical jargon. Summarizes the engagement scope, overall risk posture, critical findings, and business impact. If a non-technical executive cannot understand the risk after reading this page, the report has failed.

2. Findings with CVSS Scores AND Business Context. Every finding needs both a standardized severity rating and a plain-English explanation of what it means for the business. “CVSS 9.8 - Remote Code Execution” is meaningless to a CISO. “An attacker can execute commands on your payment processing server from the internet, potentially modifying transaction amounts or exfiltrating cardholder data” is actionable.

3. Step-by-Step Reproduction Instructions. Your developers need to verify the issue and test their fix. If the report says “SQL injection was found” but does not include the exact URL, parameter, payload, and expected response, your developers cannot reproduce it. And if they cannot reproduce it, they cannot fix it with confidence.

4. Proof of Concept Evidence. Screenshots, request/response captures, command outputs, video recordings where helpful. Evidence proves the finding is real, not theoretical.

5. Prioritized Remediation Plan with Effort Estimates. Not just “fix this” but “fix this first, and it will take your team approximately 4 hours of development work.” Remediation guidance should include specific technical steps, not vague recommendations.

6. Strategic Recommendations. Beyond individual fixes - systemic issues. If seven findings share a root cause of “no input validation,” the strategic recommendation is to implement a centralized input validation framework, not to patch seven individual endpoints.

The Useless Report

I once reviewed a pen test report from another firm - 47 pages, 200+ findings, not a single one had reproduction steps. The client’s developers could not fix anything because they could not reproduce the issues. We had to re-test everything. That is not a report - it is a list. If you have received a report like that, you did not get a penetration test. You got a scanner dump with formatting.

1. Who specifically will be testing my systems?

Good answer: “Your lead tester will be [Name], who holds OSCP and has 7 years of experience in web application testing. Here is their anonymized portfolio.”

Bad answer: “We assign testers based on availability. You will be notified closer to the engagement date.” (This means you might get a junior analyst running automated tools.)

2. What is your methodology beyond running automated tools?

Good answer: “We follow PTES with manual testing for business logic, authentication, and authorization flaws. Automated scanning is about 20% of our process - the rest is manual analysis and exploitation.”

Bad answer: “We use industry-leading tools like Nessus, Qualys, and Burp Suite to comprehensively scan your environment.” (If tools are the methodology, you are paying for a scanner.)

3. Can you show me a sample report?

Good answer: “Absolutely. Here is an anonymized sample. Note the reproduction steps, the business impact analysis, and the prioritized remediation guidance.”

Bad answer: “Our reports are confidential. We cannot share samples.” (Every serious firm has anonymized samples. If they will not share one, they are hiding something.)

4. How do you handle critical findings discovered mid-test?

Good answer: “Critical and high-severity findings are reported immediately via a secure channel - typically within 1 hour of validation. We do not wait for the final report.”

Bad answer: “All findings are included in the final report at the end of the engagement.” (If they find a critical RCE vulnerability on day 1 and do not tell you for two weeks, that is reckless.)

5. What happens if you accidentally cause an outage?

Good answer: “We have a documented escalation procedure. We immediately stop testing, notify your point of contact, and assist with recovery. Our insurance covers any damages.”

Bad answer: “That has never happened.” (It happens to every firm eventually. The question is whether they have a plan for it.)

6. Do you carry professional liability insurance?

Good answer: “Yes, we carry errors & omissions insurance and cyber liability insurance with $X million in coverage. We can provide a certificate of insurance.”

Bad answer: “We will get back to you on that.” (No insurance means you absorb all the risk.)

7. What certifications does your testing team hold?

Good answer: “Our testers hold OSCP, OSEP, GPEN, and CREST CRT. But more importantly, here is what they have found in engagements similar to yours.”

Bad answer: “Our team holds CEH, CompTIA Security+, and various vendor certifications.” (These are awareness-level certifications, not practitioner-level.)

8. How do you scope and price engagements?

Good answer: “We conduct a scoping call, review your architecture, and provide a fixed-price proposal with a clearly defined scope of work within 24-48 hours.”

Bad answer: “We charge $300/hour and estimate the engagement at 40-80 hours.” (Hourly billing with a 2x range means they do not know how long it will take, and you have no cost certainty.)

9. Do you provide retesting after remediation?

Good answer: “Yes, retesting of identified findings is included in the engagement price. We typically schedule it 4-8 weeks after report delivery.”

Bad answer: “Retesting is available as a separate engagement at our standard rates.” (Retesting should be included. If it is not, budget for it.)

10. Can you provide three recent client references?

Good answer: “Absolutely. I will connect you with a financial services client, a SaaS company, and a healthcare organization we tested in the last six months.”

Bad answer: “Due to confidentiality, we cannot share client names.” (Every firm can ask a few satisfied clients for permission to serve as references. If they cannot produce a single reference, they either have no clients or no satisfied ones.)

• After major releases or infrastructure changes. New code means new attack surface. If you deployed a significant feature, migrated infrastructure, or changed your authentication system, you need a test.
• Before compliance audits. SOC 2, PCI DSS, ISO 27001, HIPAA - all either require or strongly benefit from recent penetration test results. Schedule tests 2-3 months before your audit window to allow time for remediation. See our IT security audit services.
• Annually at minimum. Quarterly for high-risk environments (financial services, healthcare, critical infrastructure). The threat landscape changes faster than most organizations update their defenses.
• After a breach or security incident. Validation testing confirms that remediation was effective and that no additional attack vectors remain. Our vulnerability assessments can supplement post-incident testing.
• Before M&A activity. Buyer-side due diligence should include penetration testing of the target’s systems. You do not want to acquire a company and inherit their vulnerabilities.
• When launching a new product or API. Before your SaaS product goes live, before your API opens to partners, before your mobile app hits the app store. Finding vulnerabilities before launch is orders of magnitude cheaper than finding them in production.
• When changing cloud providers or architectures. Migration introduces configuration errors. A cloud penetration test after migration catches misconfigurations that your team may have introduced during the transition.

What is the difference between penetration testing and vulnerability scanning?

Vulnerability scanning is automated - a tool checks your systems against a database of known vulnerabilities and produces a list. Penetration testing is manual - a human expert validates those findings, eliminates false positives, discovers vulnerabilities no scanner can find (business logic flaws, authentication bypasses, chained attacks), and demonstrates real-world exploitability. A vulnerability assessment tells you what might be wrong. A penetration test proves what is actually exploitable.

How long does a penetration test take?

Most engagements take 1-3 weeks of active testing, followed by 1 week for reporting. A focused external network test might take 3-5 days. A comprehensive web application test for a complex SaaS platform might take 2-3 weeks. Red team engagements can span 4-12 weeks. The timeline depends on scope, application complexity, and the number of user roles and authentication flows to test.

Can penetration testing break my systems?

Experienced penetration testers use techniques calibrated to minimize risk to production systems. That said, any testing that involves real exploitation carries inherent risk. Denial-of-service testing, for example, can cause outages if not carefully controlled. Reputable firms discuss risk mitigation before testing begins, define rules of engagement, carry insurance, and have escalation procedures for accidental disruptions. Testing against staging environments first is common practice for risk-sensitive systems.

Should I do black box, grey box, or white box testing?

Grey box testing provides the best value for most organizations. The tester gets limited information (standard user credentials, API documentation, network diagrams) which simulates a realistic attacker who has done initial reconnaissance. Black box wastes time on reconnaissance that could be spent finding deeper vulnerabilities. White box (full source code access) is best for critical applications where you want the deepest possible analysis. Our recommendation: grey box for most tests, white box for your most critical applications.

How often should we get pen tested?

At minimum, annually. If you process payments (PCI DSS), handle healthcare data (HIPAA), or operate in financial services, quarterly testing is advisable. You should also test after any significant application release, infrastructure change, or security incident. Many of our clients at Atlant Security operate on a quarterly testing cadence with different test types rotating each quarter.

What is the difference between a pen test and a red team engagement?

A penetration test finds as many vulnerabilities as possible within a defined scope and timeframe. A red team engagement simulates a real adversary trying to achieve specific objectives (access CEO email, exfiltrate customer data, disrupt operations) using any means necessary - technical exploitation, social engineering, physical access, the full spectrum. Red teams test your detection and response capabilities, not just your defenses. Pen tests are for finding holes. Red teams are for testing whether your security team can catch an attacker in the act.

Do penetration testers need access to our source code?

Not always, but source code access (white box testing) significantly increases the depth of testing. With source code, testers can identify vulnerabilities that are extremely difficult to find through black box testing alone - hard-coded credentials, insecure cryptographic implementations, race conditions, and subtle authorization flaws. For critical applications, we recommend providing source code access. For standard assessments, grey box with API documentation is sufficient.

Can we use internal staff for penetration testing?

Internal pen testing programs have value, but they should supplement - not replace - external testing. Internal testers carry inherent bias: they know the systems, they have relationships with developers, and they may unconsciously avoid testing areas they know are fragile. External testers bring fresh eyes, no political constraints, and experience across hundreds of different environments. Most compliance frameworks also require independent, third-party testing.

What compliance frameworks require penetration testing?

PCI DSS explicitly requires penetration testing at least annually and after significant changes. SOC 2 does not strictly require it but auditors strongly expect it as evidence of security testing controls. ISO 27001 requires regular security testing as part of the ISMS. HIPAA requires risk assessments that typically include penetration testing. GDPR requires “regular testing, assessing, and evaluating” of security measures. In practice, if you are pursuing any serious compliance certification, you need penetration testing. See our security audit services for compliance-driven testing.

How do we prepare for a penetration test?

Preparation makes a massive difference in test quality. Before your engagement: (1) Define clear scope and objectives. (2) Provide the tester with accounts for each user role (if grey/white box). (3) Share API documentation and architecture diagrams. (4) Whitelist the testing team’s IP addresses in your WAF and IDS so testing is not blocked. (5) Notify your hosting provider and cloud vendor (AWS, Azure, GCP all require penetration testing notification). (6) Designate a point of contact who can answer questions and authorize emergency escalation. (7) Sign the rules of engagement and statement of work before testing begins.