Prompt injection in AI-assisted dispute resolution
How manipulation attempts can enter through emails and evidence, why a suspicious instruction must not decide a case, and what a fair layered defence looks like.
Evidence can contain instructions. The system must treat them as evidence.
Prompt injection occurs when text or other content presented to a language model tries to alter the model’s role, reveal protected information, or cause an unauthorised action. The attempt may be direct in an email, or indirect inside a PDF, image, website, quoted conversation, or document that an agent is asked to analyse.
That distinction matters in a dispute. Parties are expected to submit adversarial claims and documents. The material may quote technical discussions about prompt injection, reproduce a suspicious message for evidentiary reasons, or contain text authored by someone else. A security signal therefore cannot be treated as proof of who created the text or what the underlying dispute is worth.
A manipulation attempt is a security event and potentially an evidentiary fact. It is not an automatic judgment for the other party.
Why dispute systems face a sharper version of the risk
An ordinary assistant can sometimes refuse a suspicious request and end the interaction. A dispute system has additional duties: preserve relevant material, keep the parties’ private channels separate, let each side answer material allegations, and avoid turning a technical detector into a hidden decision-maker.
The input is also unusually diverse. Emails, archives, word-processing files, images, recordings, and forwarded threads may all become part of a case. OWASP identifies emails, documents, web pages, hidden text, and multimodal content as possible carriers of indirect prompt injection. OpenAI and Anthropic likewise describe prompt injection as an open problem for agents that combine untrusted content with tools or sensitive data.
Why “award the case to the other side” is the wrong default
An automatic loss rule would be easy to exploit. A party could forward an opponent’s document containing suspicious text, quote an attack for the purpose of reporting it, or submit a file that was modified by a third party. False positives would then become outcome-changing evidence without attribution or a chance to respond.
A fairer sequence is to contain the technical risk first and evaluate conduct second. The system can preserve the original, record where it came from, prevent the content from changing system behaviour, and ask for an explanation. Only after attribution and context are assessed should deliberate manipulation affect credibility, admissibility, procedure, or any final evaluation.
Layered defence without a magic filter
No single classifier, keyword list, or system prompt can guarantee safety. Current guidance from OpenAI, Anthropic, OWASP, and NIST points toward defence in depth: reduce what an exposed model can access, separate untrusted content from authoritative instructions, validate actions independently, monitor outcomes, and keep human override available for higher-risk decisions.
The important controls are architectural. They should continue to work even if a model misclassifies cleverly hidden text.
- Treat every party message, attachment, retrieved page, and tool result as untrusted case data.
- Keep system authority, party content, and shared evidence in clearly separated data paths.
- Apply least privilege so a model reading a document cannot automatically disclose data or perform an irreversible action.
- Validate tool calls, recipients, permissions, and case identity outside the model.
- Quarantine or restrict suspicious material while retaining provenance and an auditable original.
- Screen outputs and proposed actions for leakage, role changes, and unauthorised side effects.
- Require human review or explicit party confirmation for consequential or ambiguous steps.
- Test repeatedly with direct, indirect, encoded, quoted, and multimodal attack examples.
What DIN.ORG can explain publicly — and what it should not publish
Participants should understand the governing principles: untrusted material is handled as case data, private channels remain separated, suspicious content does not automatically determine the merits, and consequential outcomes must follow a fair process. Publishing those commitments helps parties and authorised agents behave predictably.
Publishing internal system prompts, detector thresholds, exact pattern lists, or a complete map of security controls would not improve procedural transparency. It would make evasion easier and create a misleading impression that the published list exhausts the defence. Public documentation should describe guarantees, roles, data boundaries, review options, and reporting paths while operational details remain restricted and continuously tested.
The procedural answer is as important as the technical one
Prompt injection is not only a model-security issue. In dispute resolution it is also a provenance, due-process, and evidence-handling issue. A robust platform needs both halves: technical controls that prevent untrusted content from gaining authority, and procedural controls that avoid punishing a party before attribution and context are established.
That combination is deliberately less dramatic than declaring an instant winner. It is also more resilient, because an attacker cannot obtain a favourable outcome merely by planting suspicious text in material associated with someone else.
Sources and further reading
The technical claims in this article are grounded in the primary and authoritative sources below. DIN.ORG’s process descriptions refer to the service as operated on the publication date.