AI Risk Database
airisk.mit.edu
MIT's database of documented AI risk incidents and failure modes. Catalogues real-world cases of AI system failures for researchers and practitioners. Essential reference for threat modeling and risk assessment of deployed AI agent systems.
More in AI Agent Security, Guardrails & Red Teaming
Crowdsourced dataset and game for studying prompt injection attacks and defenses. Players submit attack and defense prompts that get evaluated against each other. Has generated one of the largest public datasets of adversarial prompt interactions.
Splunk's open-source security scanner for analyzing AI agent codebases. Detects vulnerable patterns like unsanitized tool outputs, missing guardrails, and over-privileged tool permissions. Generates visual dependency graphs and security reports.
Open-source LLM prompt injection and security scanner with multiple detection strategies. Combines vector similarity, YARA rules, and canary token techniques for multi-layered defense. Deployable as a local REST API alongside your inference pipeline.
Microsoft's open-source data protection and de-identification SDK for detecting and anonymizing PII in text. Supports 40+ entity types and 15 languages. Used as a preprocessing layer to scrub sensitive data before sending to LLMs.
IBM's comprehensive toolkit for detecting and mitigating bias in AI datasets and models. Contains 70+ fairness metrics and 11 bias mitigation algorithms. Increasingly applied to evaluate and correct bias in LLM-powered decision systems.
The definitive security checklist for LLM applications, covering prompt injection, insecure output handling, supply chain vulnerabilities, and more. Published by OWASP and maintained by a global community of AI security practitioners. The starting point for any LLM security audit.