Essential Considerations When Selecting a Penetration Testing Partner

After nearly a decade of leading penetration testing engagements and growing our team from one tester to 43 professionals, we've learned that the questions you ask during vendor selection can determine whether you'll receive genuine security value and a successful engagement, or just frustrating checkbox exercises.  

The conversations pen test providers have with prospective clients reveal common pain points that could be avoided with better vendor evaluation. This means it’s imperative that you ask the right questions when considering who to partner with for your penetration testing needs.  

In this blog, we’ll cover key considerations and essential questions you should ask when selecting a penetration testing partner. This way, you can better identify which provider is right for you and get the most out of your engagement while reducing the risk of potential miscommunications. 

Methodology and Communication Framework 

Understanding how a testing firm operates day-to-day matters more than their marketing materials suggest. When you evaluate vendors, you should dig into their actual processes, from contract signing through final deliverable by asking questions surrounding their communication style, testing procedures, engagement timelines, and delivery tools.  

At Schellman, our engagements begin with a kickoff call where you meet your dedicated project manager, followed by secure information sharing through our AuditSource 2.0 platform powered by FieldGuide. This ensures nothing sensitive travels through email and provides you with a centralized location for uploading credentials, downloading status updates, and the final report. 

The communication style during testing highlights our partnership philosophy. We operate penetration tests as collaborative engagements where you know our source IP addresses, receive status updates with all finding details (including screenshots and steps to reproduce), and get immediate notification of high-risk findings within 24 hours. Compare this approach to vendors who disappear for weeks, communicate poorly, and dump findings on you at the end. Some organizations prefer real-time communication through shared Slack or Teams channels, which we always accommodate because we believe in flexible and adaptive communication. 

Testing methodology separates thorough assessments from checkbox exercises. While many firms reference the OWASP Top 10, we use the latest OWASP Web Security Testing Guide as our primary methodology framework for applications. More importantly, we invest time in understanding what specifically concerns your organization and which attack vectors apply to your application or environment. Our methodology is tailored to your specific risks rather than following a generic checklist approach across all clients. 

Responses that should be seen as red flags include vague promises about "regular updates" without specific timelines, reluctance to provide dedicated contacts, or insistence on email-only communication for sensitive findings. Quality vendors should articulate clear escalation procedures and demonstrate secure communication capabilities from the first conversation. 

Technical Capabilities and Infrastructure 

Evaluating technical capabilities requires specificity about your environment. Web applications, thick client applications, iOS and Android mobile apps, AI Red Team assessments, cloud environments, and on-premises systems each demand different skill sets and infrastructure. When mentioning your diverse technology stacks, your potential pen test provider should explain their unique approach for each category. If not, this should raise concerns because generalist claims are often used to mask gaps in technical capabilities. 

For on-premises testing, we deploy virtual machines or physical devices that connect to our penetration testing lab through a VPN connection. This requires coordination to allow list traffic through your network, but it establishes secure testing channels for internal applications. Many vendors lack this infrastructure and discover limitations after contracts are signed. 

Client-side application testing involves spinning up virtual machines in our lab, taking clean snapshots after OS installation, installing your application, and analyzing introduced vulnerabilities. We proxy all traffic to identify suspicious communications or non-standard behaviors. Vendors who cannot demonstrate this capability will struggle with desktop applications, mobile apps, or any software distributed to end users. 

It’s equally important to consider how vendors handle environment-specific challenges. We recently worked with an organization requiring testing of applications deployed in air-gapped networks. Our solution involved physical device deployment with secure callback mechanisms. Based on feedback we’ve heard from organizations during scoping calls, other firms lacked the infrastructure flexibility for such requirements. 

Risk Assessment and Finding Management 

Risk rating discussions expose vendor philosophy more clearly than technical demonstrations. We approach risk as likelihood multiplied by impact, but other vendors apply inflexible formulas without considering environmental context. When we document high-risk findings but lack complete environmental understanding, we engage in collaborative discussions about actual risk levels. 

Contrast this with vendors who mark security improvements as high-risk vulnerabilities or refuse to consider mitigating controls you've implemented. I've seen organizations receive reports flagging missing security headers as high-risk vulnerabilities despite robust compensating controls. Quality vendors distinguish between actual vulnerabilities and security enhancement opportunities. 

During recent engagements, we've observed the value of collaborative risk discussions. Organizations often possess environmental knowledge that affects vulnerability exploitability, but some vendors ignore this context. We adjust risk ratings based on likelihood factors like network segmentation, access controls, or deployment differences between testing and production environments. 

Contract Structure and Pricing Models 

Contract flexibility accommodates varying organizational needs better than a standard engagement model. Traditional per-project contracts work well for organizations with single annual penetration tests or specific compliance requirements. These engagements use fixed-fee pricing based on scope and complexity, with one retest of all originally identified findings included at no additional cost within 30 days. 

Organizations with active development cycles benefit from retainer arrangements. We typically structure 500-hour annual retainers that provide testing flexibility without repetitive procurement processes. Retainer models enable responsive security testing throughout development cycles. Instead of waiting for scheduled annual assessments, you can request testing with two weeks' notice as new features deploy, or applications launch. Hours deduct from your annual bucket, and we handle everything from complex penetration tests to quick security reviews of specific features. 

The invoicing mechanics matter for budget planning. Our retainer invoicing occurs monthly regardless of usage, with annual true-up reconciliation. Hours typically expire after twelve months, or you consume the full allocation and renew. The next question that comes up is always around pricing. You can expect ballpark pricing on 500-hour retainers to estimate $200-250k annually, though final pricing depends on testing complexity and organizational requirements. 

Understanding retest policies requires clarity about pricing models. With project-based engagements, we include comprehensive retesting of all originally identified findings at no additional cost. This fixed-fee approach encourages thorough remediation rather than quick patches. However, retainer arrangements handle retesting differently, those hours come from your allocated bucket since you're paying for testing time rather than specific deliverables. 

Compare this transparency with vendors who avoid pricing discussions or provide only per-project quotes. Both retainer and project-based arrangements require upfront cost clarity because budget planning spans entire fiscal years. Vendors unwilling to discuss pricing ranges during initial conversations often surprise organizations with significant cost variations or up charges for extras (such as retesting time, customer-facing summary letter, etc.). 

Deliverables and Reporting Structure 

Report quality and format determine how effectively you can act on testing results. We provide comprehensive technical reports with detailed findings, attack narratives, and remediation guidance for your technical teams. Additionally, you receive customer-facing summary letters on Schellman letterhead that provide high-level assessment results without sensitive vulnerability details. 

This document proves valuable when customers request security attestations or compliance auditors need testing evidence. The one-page format states testing scope, timeframes, high-level finding counts by risk category, and remediation status without exposing vulnerability specifics that could compromise security if shared externally. 

Frequent project status communication means you know about findings as we discover them rather than waiting for final reports. This enables immediate remediation of serious vulnerabilities and prevents the common scenario where organizations receive reports weeks after testing completion and struggle to reproduce or understand findings. 

It's best practice to evaluate whether vendors provide actionable remediation guidance beyond generic recommendations. Our reports include specific code examples, configuration changes, or architectural modifications needed to address identified vulnerabilities. This comes along with detailed technical guidance showing exactly which parameters are vulnerable, complete with screenshots and reproduction steps. Generic advice without specific context wastes valuable remediation time. 

Team Experience and Organizational Structure 

Team composition affects testing quality more than certifications or marketing claims suggest. Our internal promotion structure means everyone (managers, senior managers, directors, and managing directors) have hands-on penetration testing backgrounds rather than coming from pure sales or project management roles. While they don't perform active testing anymore, they understand penetration testing intricacies because they built their careers as practitioners. This enables better project guidance and quality assurance throughout engagements. 

Understanding specialist allocation helps predict testing effectiveness. Web application specialists bring different expertise than network penetration testers or mobile security experts. We match tester specialization to engagement requirements because expertise depth matters more than generalist breadth for complex applications. 

Vendor scalability affects long-term partnership viability. Organizations with growing security programs need vendors who can accommodate increased testing volume without quality degradation. Our growth from 1 to over 45+ team members since 2014 demonstrates scaling capability but also reflects the challenge of maintaining quality during rapid expansion. 

Integration with Existing Security Programs 

Security program integration extends testing value beyond individual engagements. We coordinate with SOC teams and compliance programs to ensure penetration testing supports broader security initiatives rather than operating in isolation. This includes working with PCI and FedRAMP teams when those compliance frameworks apply to your environment. If you're working with other internal teams at Schellman, we collaborate closely to ensure penetration testing requirements align with your broader compliance and security initiatives. 

This prevents duplicate efforts and ensures testing addresses organizational priorities. Organizations implementing centralized security repositories benefit from vendors who support integration requirements. We work with governance, risk, and compliance teams to ensure testing results populate central systems where auditors and stakeholders can access attestations and executive summaries without chasing down individual reports. 

Preparing for Vendor Conversations 

Document your technical requirements, testing timeline preferences, and budget parameters before engaging in vendor discussions. Prepare specific questions about your technology stack, compliance needs, and organizational constraints. Request sample reports and references from organizations with similar environments and requirements. 

Create evaluation criteria that weigh factors according to your priorities. Technical capability, communication quality, pricing transparency, and cultural fit carry different importance depending on organizational maturity and program objectives. Use consistent evaluation frameworks across vendor discussions to enable fair comparisons. 

Schedule follow-up conversations with your top vendor candidates after initial discussions. Testing partnerships develop over multiple engagements, so assess vendor responsiveness, flexibility, and willingness to adapt approaches based on your feedback. The best partnerships begin with vendors who demonstrate genuine interest in understanding your specific security challenges and organizational context. 

To learn more about Schellman’s penetration testing processes, timelines, or communication styles, contact us today. In the meantime, discover additional insights on how to secure a successful pen test in these helpful resources: 

• Get the Pen Test Report You Deserve: A Leader’s Guide 
• Don’t Buy A Network Pen Test Until You Ask These Questions 
• Essential Considerations for an Effective Mobile App Pen Test 
• Don't Start Your Hardware Penetration Test Before Reading This 
• Are You Getting the Right Red Team?