Where every claim in SecProve
comes from.

Conference presentations covering novel attack techniques and defensive research. Essential for cutting-edge offensive/defensive questions. AI Village talks particularly relevant for Pillars B and C.

Demonstrated GPT-4 exploiting real-world web vulnerabilities autonomously. 73% success rate on day-one CVEs. Key reference for questions about AI-augmented offensive capabilities and the asymmetry debate.

Analysis of how LLMs can be used for offensive security tasks and the implications for defensive guardrails. Covers the dual-use nature of security LLMs.

The definitive security risk list for LLM-powered applications. Covers prompt injection, insecure output handling, training data poisoning, and more.

Adversarial Threat Landscape for AI Systems. ATT&CK-style knowledge base of adversarial ML techniques, tactics, and real-world case studies.

Collection of Anthropic's published research on AI safety, alignment, interpretability, and security.

Python Risk Identification Toolkit for generative AI. Automated red teaming framework for testing LLM applications.

Demonstrated that LLMs memorize and can be prompted to regurgitate training data verbatim, including PII. Foundational work on LLM privacy risks.

Showed that gradually escalating benign conversations can bypass safety filters over multiple turns. Defeats per-message safety checks.

Demonstrated indirect prompt injection attacks through RAG documents, emails, and web content. Essential reading for RAG security.

The GCG attack paper. Showed that adversarial suffixes can bypass safety alignment in LLMs, transferring across models.

NVIDIA's open-source LLM vulnerability scanner. Tests for prompt injection, jailbreaking, data leakage, and more.

Demonstrated that long-context LLMs can be jailbroken by providing many examples of the desired behavior. Scales with context window size.

Practical lessons from large-scale LLM red teaming across real products. Covers failure modes, testing methodologies, and organizational patterns. Rare insight into enterprise-scale AI security.

Companion to AI RMF 1.0 specifically for generative AI. Maps 12 GenAI risks to RMF actions. Covers CBRN, CSAM, confabulation, data privacy, environmental, human-AI interaction, information integrity, IP, obscenity, toxicity, value chain.

Extension of the LLM Top 10 specifically for agentic patterns. Covers excessive agency, insecure plugin/tool design, and multi-agent trust boundaries.

Largest prompt injection competition dataset. Taxonomy of prompt injection techniques: context ignoring, fake completion, payload splitting, obfuscation. Empirical data on attack success rates across models.

Systematic analysis of jailbreak techniques: competing objectives and mismatched generalization. Framework for understanding why safety training is inherently incomplete. Essential for nuanced jailbreak questions.