On Monday 2 June 2025 I went to “Security in AI: an afternoon with Sarah Young”, organised by the Global AI Utrecht chapter at Capgemini in Utrecht. According to the event page it ran from 15:00 to 18:00, with Sarah Young, Principal Security Advocate at Microsoft, talking about AI and cybersecurity. The poster slide on the screen carried the same title and the Global AI Chapter Utrecht branding.
This is a throwback post. I recorded most of the afternoon, and what follows is my paraphrase of the talk and the Q&A, checked against the slides that were readable in my recordings. The numbers are the speaker’s own and I have not re-verified them.
The poster slide before the session started.
She kept it casual and invited questions throughout, so the room of engineers and AI people pushed back and added their own examples. Her agenda was the threat landscape, keyless authentication, layering security controls, data security for AI, and MCP security.
The threat landscape: attackers got faster too
Her point was that AI is great for productivity, including the productivity of attackers. The statistics below are what she presented as figures Microsoft collects itself; I did not check them independently:
- Microsoft tracked about 300 threat actors in 2023 and over 1,500 now.
- The time between a phishing click and an attacker accessing data has dropped from about a day to about an hour, and phishing mail is better written because AI helps non-native speakers.
- A key or secret pushed to a public repository used to be picked up by attackers within a couple of days; she said it is now minutes, found by automated scanning.
- Password attacks against Entra ID went from about 579 per second to about 7,000, and 99% of the identity attacks Microsoft sees, over 600 million a day, are password based. Her advice was to switch on multifactor authentication everywhere, then move towards phishing-resistant methods.
- Her slide on application security, with data from GitHub, showed the amount of code and the number of potential vulnerabilities rising together: we write more code but not more secure code, and flaws in applications remain the top attack vector for breaches.
Her key message, repeated several times: do the basics right. Attackers are not proud of their craft. They look for a hard-coded key, an open API or an admin account without MFA. She said she hears a lot about model poisoning and training-data inference, which are real but rarely the easiest way in. Her example was a breach at an Australian telecom, which she described as an API open on the internet with no authentication in front of a customer database.
Please use keyless auth
Her second point was to use identities instead of keys when building AI applications. A short debate followed with an attendee who argued that keys can be fine when used well, and she held her ground. The reasoning she gave:
- Managed identities in Entra are essentially modern service accounts for machine-to-machine access, with SDKs that handle token refresh. They used to be fiddly, and she said they have improved a lot.
- With resources spread across clouds, SaaS and on-premises, the network can no longer be the perimeter. Identity is the new perimeter, and one central identity store, whether Entra or another modern provider, is easier to manage and protect than scattered keys.
- Keys are painful to rotate and tokens are short-lived. In Entra you can also revoke tokens.
- Some things still need keys today, so keep them to a minimum.
- Least privilege is hard for machine identities because they give no feedback the way humans do. She mentioned Entra features that show which permissions an identity never uses so you can remove them, but said it is better to design the permissions up front.
On agents, an attendee asked whether an agent should be managed like a user rather than an application registration. She confirmed that Microsoft had just announced Entra Agent ID at Build, a separate identity type that is neither human nor machine, with conditional access policies, with a June timeline she mentioned.
Layering security controls
The next slide was a layered diagram: identity, responsible AI controls (content and prompt filters), network security, an egress point if the application is public, and monitoring with detection and containment. Her rules of thumb:
- Layers must be different controls. Five identical firewalls are one control five times.
- Assume breach and keep the blast radius small: minimal permissions plus detections that notice an account suddenly touching resources it never uses.
- Attackers may sit in an environment for months before anyone notices, so detection time matters.
- Network security groups are free and simple; for a modern web app, the only ports open should be 443. Use private endpoints for everything you can, and put a web application firewall in front of any public web app. She explained the choice between Application Gateway for regional traffic and Front Door for global traffic, and said that you cannot deploy the Azure WAF without one of the two.
- Zero trust is not a product; if someone sells it as one, she said, that is also a lie.
She illustrated the layering with air accident investigations: serious crashes need many things to go wrong, and the only way to guarantee no crash is to never take off, which is no answer for a business.
Defender for Cloud is the tool she wanted builders, not only security people, to know: cloud security posture management, with a free basic tier, that gives hygiene recommendations with risk levels and fixes, some of which can be remediated automatically. She described a Build demo with a colleague where they took a basic chat app through the recommendations until they were all done, including a stricter responsible AI content filter, diagnostic logs into a Log Analytics workspace for the security team, a managed identity, built-in authentication so users log in with their own permissions, a VPN gateway for developers and a WAF with Front Door at the egress. The sample repository for that lab is open source.
Data security and oversharing
She opened this part with the observation that breaches are usually about data, and that most security work protects what surrounds the data rather than the data itself. The talk then walked through the flow of an AI application: prompt in, application, LLM, retrieval from internet, internal stores and unknown dark data, content out.
- Oversharing is internal: a copilot finds a salary spreadsheet that you technically had access to but would never have found. Several attendees had their own versions of this story. Leakage is when it goes outside.
- AI also creates data: transcripts, generated documents and telemetry such as every prompt, with the question of who owns and can find it.
- Restrict AI access to data, which is least privilege again. Give an agent only what it needs, never an “omni brain” with access to everything; she compared it to making an agent a global admin.
- Both sides need controls: data classification and information protection on the data, and application controls and content filters on the AI. In Microsoft’s Foundry content filters, prompt-injection and jailbreak detection are on by default, with indirect prompt injection a setting you turn on.
- Her analogy: treat AI like a small child. If you tell it a secret, it will tell someone; so do not give it the secret.
MCP security
The last section was about the Model Context Protocol. She described it as an open-source protocol from Anthropic that standardises how LLMs plug into tools and data, the “USB” of AI, and said it exploded in popularity within a couple of months. The risks she covered:
- Authorization. Until about a month before, there was no way to use a third-party identity provider, so you had to write your own authorization server, “which we never do”. The specification had since been changed so you can plug in providers such as Entra or Okta, with some issues remaining.
- Over-permissioning. The server runs with one identity and has all of its permissions; the protocol did not yet let you scope what each tool can reach.
- Tool poisoning is, in her words, indirect prompt injection under a new name: a tool advertises metadata with hidden instructions.
- The rug pull, where a tool is changed after it was verified, is the classic confused-deputy or supply-chain pattern, seen before in Python packages. She said she had not seen much of it in the wild.
- Because prompt injection is non-deterministic, filters need constant updating, and Microsoft uses an AI red team to attack its own products. She mentioned the open-source AI red teaming lab that came out of Build, built on the team’s tooling.
She was explicit that she is not telling people to avoid MCP, only to know the risks and add compensating controls, and said that the spec changes so fast that her own blog post was out of date a week later.
She ended with pointers to the Build material: the secure AI app samples, the red teaming lab, and a session on writing secure code by Michael Howard, the author of the book of that name.
A short interview
At the end I asked Sarah for a few words on camera; she introduced herself as a Principal Security Advocate at Microsoft, matching the event page. Her takeaway in one sentence: AI security is not as new as you think, most of the problems are existing application security problems, and getting the basics right goes a long way. Looking five years ahead, she expects agents and AI to do a lot more, expects attackers to use them too, and said the work stays exciting because defenders can protect in new ways.
My take: this is the most useful framing of AI security I have heard this year. Every control she named already exists in your cloud account. The new part is that the thing calling your APIs may now be an agent with an identity nobody reviewed, and that is a platform team problem as much as a security one.
