BSides Vancouver 2026

Organizations are rapidly deploying AI-powered chatbots, copilots, and agentic workflows - often faster than security teams can adapt their testing practices. Traditional pentesting assumes deterministic systems, stable input/output schemas, and well-defined trust boundaries. Those assumptions no longer hold when natural language becomes both the interface and the attack surface, and when models can retrieve data, invoke tools, and trigger real-world actions.

This session explores AI red teaming as a practical, adaptable, and repeatable application security exercise rather than a collection of one-off jailbreak techniques. We’ll examine where risk actually concentrates in modern AI systems - supporting REST API endpoints, the orchestration layer surrounding LLMs, access controls, input/output handling, and why focusing on the model alone misses the most meaningful exposures, along with a look at real-world cases where attackers have exploited AI-powered functionality to impact businesses.

Through demonstration of automated testing techniques using open source AI red teaming tools (e.g., Garak, Promptfoo, DeepTeam, etc.) and industry guidance (including the OWASP Top 10 for LLM Applications), attendees will see how ad-hoc experimentation can mature into a repeatable testing approach: structured test matrices, risk-driven evaluation, and findings translated into business impact such as data exposure, unauthorized actions, cost and availability risks, and regulatory or reputational consequences.

The talk concludes with recommendations for building a layered defense strategy and for integrating continuous AI security testing into existing development workflows. Attendees will leave with a practical mental model for assessing AI risk, communicating it to leadership, and building testing practices that scale alongside rapidly evolving AI deployments