Chapter 6: A Practical Framework for Vendor Security Assessment

6.1 Bringing It All Together: The Unified Framework

This chapter moves from principles to practice. Use this simple, repeatable process to assess any LLM-centric vendor.

1. Scope the Use Case: Define business goals, data sensitivity, in-scope integrations, and required capabilities.
2. Issue a Standard Questionnaire: Send a structured RFI/RFP based on the checklist below to drive apples-to-apples comparisons.
3. Review Documents: Examine ToS/Privacy Policy, Trust Center, SOC 2 reports, pentest summaries, and architecture diagrams.
4. Score and Decide: Apply an objective scoring model tied to risk appetite; capture mitigations and contract controls.

6.2 The Master Vendor Assessment Checklist

Organize questions into four domains. Require precise, unambiguous answers.

Section 1: Data Governance & Privacy

• Is customer data used for model training? (No / Opt-in / Yes). Default handling and controls?
• Prompt and output logging: retention period, access controls, redaction options, disablement support.
• Data segregation model: single-tenant VPC vs. logical multi-tenant; per-tenant encryption keys.
• PII handling: collection minimization, masking/redaction, data subject request processes (GDPR/CCPA).
• Deletion guarantees: timelines for soft/hard delete for storage, backups, and logs.

Section 2: Application & Infrastructure Security

• Hosting isolation: tenant-dedicated compute/storage/network; sandboxing strategy.
• Encryption: AES-256 at rest; TLS 1.2+ in transit; cipher policy; key management (KMS/HSM), rotation.
• Secrets management: vaulting, scoping, rotation; no secrets in logs; access paths and break-glass controls.
• Abuse prevention: rate limits, anomaly detection, egress controls, DDoS/WAF protections.
• SDLC security: SAST/DAST/dep scanning, SBOMs, supply chain controls, vulnerability SLAs.

Section 3: Prompt & Model Security

• Prompt injection defenses: input filtering, instruction hierarchy, output validation, red-teaming cadence.
• Agency controls: can the system perform actions? scopes? mandatory human approval for state-changing ops.
• Safety systems: jailbreak resistance, content moderation, policy enforcement, measured evals.
• Model provenance: pre-trained sources, fine-tune data controls, evaluation methodology.

Section 4: Compliance & Trust

• Attestations: SOC 2 Type II, ISO 27001, ISO 42001, GDPR/CCPA readiness, regional data residency.
• Trust Center: public documentation of controls, uptime, historical incidents, and security contacts.
• Audit logging: coverage, retention, integrity guarantees; SIEM export and customer access.
• Incident response: timelines, customer notification commitments, forensics, and postmortems.

6.3 Scoring and Risk Assessment

Apply an objective R/A/G scoring per item, then map residual risk using a simple matrix.

• Likelihood: Low / Medium / High based on exposure, history, and controls.
• Impact: Low / Medium / High based on data sensitivity and blast radius.

Combine into quadrants to guide decisions:

• Low: Proceed; standard monitoring.
• Medium: Proceed with compensating controls and contractual clauses.
• High: Conditional approval; require remediation plan and timelines.
• Critical: Do not proceed; consider alternative vendors.

6.4 The Living Document: Continuous Verification

Security posture changes over time. Treat vendor assessment as an ongoing program, not a one-off gate.

• Annual Reviews: Re-run the checklist; refresh evidence and certifications.
• Monitoring: Subscribe to trust center updates; track incidents and maintenance windows.
• Change management: Reassess on material changes (new features, regions, subprocessors).
• Runbooks: Maintain escalation paths, contacts, and incident playbooks for each vendor.

Next, we apply this framework to concrete examples—highlighting best-in-class postures and common pitfalls.