AI Security Night Amsterdam: Breaching LLM-Powered

I spoke at the AI Security Night hosted by the AI Security Engineers Community at The Social Hub in Amsterdam. I delivered the second talk: “Securing Multi-Tenant AI Platforms: Practical Lessons from OpenShift AI, GPUs, and MLOps”, while Brian Vermeer (@brianvermeer.nl) opened with the headline talk: “Breaching LLM-Powered Applications — Overcoming Security and Privacy Challenges”.

About 40 attendees joined for drinks, pizza, and two deep technical sessions on AI security — from application-level LLM vulnerabilities to platform-level infrastructure isolation.

Remediation: Where Does Your AI Run?

Brian started with a fundamental architectural question: where does your AI actually run? The remediation landscape looks completely different depending on your deployment model:

• SaaS — Provider-hosted, data leaves your boundary
• Private Cloud — Your cloud, your models, still network-connected
• On-Prem — Behind your firewall, full data control
• Local — On-device, zero network exposure

Each model carries different risk profiles for data exfiltration, prompt injection, and model poisoning. The key insight: your security posture must match your deployment reality.

Data Privacy: The OpenAI Problem

A sobering reminder about data privacy. Brian showed OpenAI’s consumer services FAQ: “We may use content submitted to ChatGPT, DALL-E, and our other services for individuals to improve model performance.”

For enterprise use cases, this means:

• Consumer-tier AI services may train on your inputs
• API and business tiers have different data handling policies
• Understanding the distinction between consumer and business accounts is critical for compliance
• Your prompts, responses, images, and files could all be training data

This is why many enterprises mandate on-prem or private cloud deployments for sensitive workloads.

Securing Agentic AI: Function-Level Controls

The most actionable part of the talk covered agentic AI security patterns:

• Write small functions — limit capabilities per tool call
• Respect user permissions — the AI agent should never exceed the user’s authorization level
• Require explicit user confirmation — Human-in-the-loop for destructive or high-value actions
• Restrict high-risk functions — separate read-only from write operations
• Audit and log function calls — full observability on what the agent actually did

The workflow diagram showed a practical e-commerce example: SelectProducts (AiService) → VerifyBasket (AiService) → CalculateTotalPrice (Code) → PlaceOrder (Human-as-a-tool). The critical insight: PlaceOrder requires human confirmation — the AI proposes, the human disposes.

Building Secure AI Services in Java

Brian demonstrated practical implementation using LangChain4j and Java:

@Tool("Book a car for a user")
public String newBooking(@P("Username") String username,
                         @P("Id of the car") Long carId,
                         @P("Startdate") LocalDate startDate,
                         @P("Enddate") LocalDate endDate) {
    // Validation, authorization, logging...
    var user = userService.getUserByUsername(username);
    var car = carService.getCarById(carId);
    var booking = bookingService.createBooking(user, car, startDate, endDate);
    return booking.toString();
}

The AiServices.builder() pattern wires up the chat model, memory, content retriever (RAG), and tools — each a separate security boundary that can be independently audited and restricted.

Live Demo: SupportAI Chatbot Vulnerabilities

Brian demonstrated a live chatbot (supportai.com) to show how poorly constrained AI systems behave. The first test: asking it to write a poem about Devoxx — harmless, but already outside its intended support scope.

The second test was more revealing: asking a support chatbot to explain linear algebra. It happily complied with a detailed math explanation — proving it has no system message constraints, no scope boundaries, and no input validation. A perfect example of why guardrails matter.

The Complete Security Checklist

Brian’s final summary — the definitive checklist for securing AI applications:

1. Prevent security vulnerabilities in other parts of your system
2. Sanitize the input and output of the LLM (GuardRails)
3. Create a strict SystemMessage — scope what the AI can and cannot do
4. Limit LLM capabilities to user privileges
5. Build small-scope services — single responsibility per AI function
6. Programmatically define flow (e.g., Human-in-the-loop)
7. Use the right LLM for the task — not every query needs GPT-4
8. Enforce structured input and output — validate schemas, reject freeform where possible

My Talk: Securing Multi-Tenant AI Platforms

In the second session, I shared practical lessons from building multi-tenant AI/ML environments on Kubernetes and OpenShift AI, focusing on GPU-backed workloads. The talk covered:

• Workload isolation — namespace boundaries, network policies, and pod security standards
• Access control — RBAC patterns for data scientists vs platform engineers vs model deployers
• GPU partitioning — MIG, time-slicing, and how to prevent noisy-neighbor GPU contention
• Namespace and quota design — per-team resource boundaries that prevent runaway training jobs
• Supply-chain security — container image signing, model provenance, and registry policies
• Observability — GPU metrics, cost attribution per team, and anomaly detection
• Governance patterns — audit trails, model registries, and compliance controls for regulated environments

The core message: as AI moves from single-team experiments to shared enterprise platforms, security becomes a platform engineering problem — not just an application concern. The same multi-tenant GPU patterns I presented at Red Hat Summit apply directly to securing AI workloads at scale.

Key Takeaways

1. AI deployment model dictates security posture — SaaS vs on-prem is not just a cost decision, it is a data sovereignty decision
2. Consumer AI services may train on your data — read the fine print before shipping production features on consumer APIs
3. Agentic AI needs function-level security — treat every tool call as an API endpoint with its own authorization
4. Human-in-the-loop is not optional for high-risk actions — the AI should propose, humans should approve
5. Audit everything — you cannot secure what you cannot observe

Smart Routing: Local vs Commercial Models

Brian presented a hybrid routing architecture (credited to Dmytro Liubarskyi) that separates sensitive from non-sensitive requests:

• Local Models — handle PII-containing or confidential queries on-premises
• Commercial Models — handle general queries after passing through a security layer
• Security gate — hard filter on IP + data obfuscation before any request reaches external APIs

This pattern lets enterprises use the best commercial models for general tasks while keeping sensitive data firmly within their boundary. The routing decision happens at the proxy layer — transparent to the end user.

About the Speaker

Brian Vermeer is a Staff Developer Advocate at Snyk, Java Champion, and software engineer with over a decade of hands-on experience. He is passionate about Java, functional programming, and cybersecurity. Brian is a JUG leader for the Virtual JUG and NLJUG, co-leads the DevSecCon community, and is a community manager for Foojay. He is also a military reserve for the Royal Netherlands Air Force and a Taekwondo Master — bringing discipline from both the dojo and the air force to application security.

Community Reactions

The meetup resonated well beyond the room. Davide Cioccia, founder of the AI Security Engineers Community, summarized the evening:

“Great energy at our latest AI Security Engineers Community Netherlands meetup in Amsterdam. Brian Vermeer from Snyk showcased the security and data privacy challenges of AI applications powered by LLMs. Luca Berton shared practical insights on building fully air-gapped, multi-tenant AI/ML environments on Kubernetes and OpenShift AI, with a strong focus on GPU-backed workloads.”

Mahan Yarmohammad Tajari (AI Engineer at NextPax) added:

“A huge thanks to Brian V. for his eye-opening presentation about Security for AI applications which is a major priority for us at NextPax. This was followed by a brilliant session from Luca Berton with a great talk about Multi-tenant Platform Engineering for GPUs and AI model hosting.”

A big thank you to Gerald Crescione and the whole Snyk team for hosting, and to Davide for organizing another packed AI Security Engineers event. The community is already planning the next meetup — join here.

Related Content

• OWASP Top 10 for LLM Applications
• Context Architecture for AI Agents
• LLM Quality vs Cost vs Safety vs Throughput