Attack corpus
Every prompt, context payload, and chain we ran — pass and fail — handed over as a re-runnable set.
AI / ML
AI security.
Your model is a new kind of user — one that reads untrusted text and holds real credentials. We attack the model, its context, and everything it is allowed to touch.
// 01 — What it is
The prompt is an input. Treat it like one.
AI features fail the way early web apps failed: untrusted input reaching a privileged interpreter. A retrieved document becomes an instruction. An agent with a legitimate tool becomes a confused deputy. A summariser with database access becomes an exfiltration path.
We test the whole system, not the model in isolation — prompts, retrieval sources, tool permissions, output handling, and the application code that trusts the response. Every finding arrives with the chain that produced it and an evaluation case that proves the fix.
- Direct and indirect prompt injection through retrieved and user content
- Agent and tool-use abuse: confused deputy, over-scoped tokens, unbounded actions
- RAG data exposure: cross-tenant retrieval, embedding leakage, index poisoning
- Output handling in the surrounding app: rendered HTML, executed code, downstream calls
- Model supply chain: weights provenance, third-party plugins, fine-tune data hygiene
- Poisoned contextInstruction hidden in a retrieved document, ticket, or web pageInitial access
- Injection landsModel treats retrieved text as instruction, not dataExecution
- Tool abuseAgent calls a permitted tool with attacker-chosen argumentsEscalation
- Credential reachTool runs with a service token wider than the user's own scopePrivilege
- Data exfiltrationPrivate records summarised into a response, webhook, or outbound callObjective
- LLM APPSChat · copilots
- AGENTSTool use · autonomy
- RAGRetrieval · indexes
// 02 — How we run it
Five phases from system map to regression suite.
System mapping
Models, prompts, retrieval sources, tools, and downstream permissions — documented before we send a single request. What the system can be asked, what it can call, and what it can reach.
Threat modelling
We model against MITRE ATLAS and the OWASP LLM Top 10, then narrow to the abuse cases that actually matter for your product and your data.
Adversarial testing
Direct and indirect prompt injection, jailbreak chains, tool-use abuse, RAG poisoning, and output-handling flaws in the app around the model.
Chain reproduction
Every success reproduced with the exact prompt, retrieved context, and tool calls — so your team can replay it deterministically.
Guardrails & eval
Guardrail changes, tool-permission diffs, and a regression eval suite that fails loudly if the behaviour returns.
// 03 — What you get
Chains closed. Regressions caught in CI.
Guardrail diffs
Concrete changes to system prompts, tool scopes, and output handling. Not 'add a filter'.
Regression eval suite
Test cases wired to your CI so a prompt or model change cannot silently reopen a finding.
Monitoring guidance
What to log and what to alert on: injection signatures, anomalous tool calls, egress from agent paths.
// next step
Find out what your agent can be talked into.
Scoping call · attack corpus · eval suite handover