Hacking AI: Attacks and Defenses for AI Systems and Infrastructure
This training is a hands-on introduction to attacking and defending agentic AI systems and Large Language Model (LLM) applications. It introduces various techniques of prompt injection attacks to manipulate the behavior of AI systems, capture sensitive data and attack the infrastructure. As the training progresses, participants are introduced to new concepts and components, such as Guardrails and MCPs. The participants learn how to attack them, as well as how to implement them securely. The course incorporates elements of gamification, making the learning experience interactive, interesting and fun.
In addition to the offensive and defensive techniques of LLM applications, the training covers the infrastructure side of LLM systems as well. Using selected components from the platform powering this workshop — including model serving, chat interfaces, RAG pipelines, and API gateways — participants learn how common default configurations can be exploited and how to harden them.
No prior experience in LLMs, AI security or hacking in general is required, since the training starts from first principles. The lab environment can be easily accessed from your browser. Therefore, no prior setup is needed.
In this workshop, you learn
Attacking and defending LLM applications
- How LLMs work and why their architecture is inherently insecure
- OWASP Top 10 vulnerabilities of LLM applications and AI agents
- Direct and indirect prompt injection techniques
- Capturing confidential data from AI applications
- Understanding, capturing, and hardening of system prompts
- Implementation of Guardrails to protect AI systems, as well as techniques to bypass them
- The Model Context Protocol (MCP) and its trust model assumptions
- Traditional vulnerabilities like SQLi and RCE in AI systems
- Defensive patterns for LLM pipelines and MCP servers
Building and securing self-hosted LLM infrastructure
- The landscape of self-hosted LLM components
- Model serving
- Chat interfaces
- RAG pipelines
- API gateways
- When and why to self-host
- Data sovereignty
- Cost
- Customization
- Offline
- Air-gapped scenarios
- Why reproducible, declarative infrastructure matters when building on inherently non-deterministic AI systems
- Practical self-hosting trade-offs: from single-GPU development setups to multi-node platforms
- Securing LLM infrastructure components:
- Inference endpoints
- Vector databases
- Agent tools
- Code interpreters
What we prepared for you
- A full AI platform, showcasing the infrastructure patterns discussed in the workshop.
- A personal lab workspace per participant with Open WebUI, a live OpenAI Pipelines server, and more. All in the browser.
- Many hands-on exercises to practice the attack and defense techniques explained in the training.
- Deploy-Break-Harden infrastructure scenarios: real misconfigurations from self-hosted LLM components.
Who can attend this workshop?
No prior experience with LLMs or AI security is required. The workshop is practical and starts from first principles. It is a good fit for:
- Penetration testers and red teamers who encounter LLM-integrated systems in assessments
- Application security engineers working on AI or agentic products
- Developers building on the OpenWebUI API or similar backends who want to understand the risks
- Engineers and architects looking to self-host LLM infrastructure for their team or organization
- Anyone curious about the offensive side of AI security
Requirements
- A laptop with a modern browser.
- Basic Python reading skills.
- Willingness to break things.