How do you defend enterprise LLM applications against prompt injection and jailbreak exploits? (Part 2 Focus)

Defend enterprise LLM apps by locking the system prompt, enforcing runtime policy checks, and layering automated moderation with injection‑specific detectors.

Step‑by‑step implementation

1. Immutable system prompt – Store the prompt in a read‑only config (e.g., Azure Key Vault secret SystemPromptV1). Pass it via the system role on every ChatCompletion call; set temperature=0 and max_tokens=1024 to limit stochastic drift.
2. Input sanitization – Pre‑process user text with a regex that strips known injection patterns (/^\s*(!?)(?i:ignore|system|assistant)\b/). Reject if length > 4 KB or if pattern score > 0.8 using prompt-injection-detector from langdetect‑v2.3.
3. Realtime policy engine – Deploy OPA as a sidecar. Example Rego rule:

package llm.guardian

allow {
    not input.prompt_contains_jailbreak
    input.token_count < 2048
}

The app calls opa.eval with JSON { "prompt_contains_jailbreak": input.matches, "token_count": count(tokens) }.
4. Moderation layer – Invoke OpenAI’s POST /v1/moderations with model=text-moderation-latest. Block if results[0].flagged && results[0].categories["jailbreak"] > 0.7.
5. Response post‑filter – Run a hallucination detector (e.g., google‑gemini‑safety‑v1 detect_hallucination) and drop any answer with confidence < 0.85.
6. Audit & alert – Log every rejected request to Azure Monitor, trigger a Logic App if > 5 rejections per minute from the same client ID.

Method comparison

| Guardrail | Latency impact | False‑pos % | Maintenance |
|----------------------|----------------|------------|-------------|
| System prompt lock | < 5 ms | 0.1 | Low |
| OPA sidecar | 15‑30 ms | 0.3 | Medium |
| OpenAI moderation | 40‑60 ms | 0.5 | Low |