LLM Guardrails Proxy

What you'll learn

• Identifying and blocking prompt injection attacks
• Masking Personally Identifiable Information (PII) before it goes to a 3rd-party API
• Validating output for safety (e.g., toxicity, hallucinations)
• Building intermediate proxy layers

Architecture

User Prompt
  → Proxy API (FastAPI)
  → Guardrail Check 1: Prompt Injection (Llama Guard)
  → Guardrail Check 2: PII Masking (Presidio)
  → Safe Prompt → OpenAI API
  → Output
  → Guardrail Check 3: Toxicity / Hallucination Check
  → Safe Output to User

Steps

1. Build a basic FastAPI proxy server that forwards requests to the OpenAI API and returns the response.
2. Integrate a prompt-injection detection layer. You can use Meta's Llama Guard (via an API or local inference) or simply an LLM-as-a-judge prompt to detect malicious instructions.
3. Integrate Microsoft Presidio to detect and mask PII (like phone numbers, SSNs, and names) from the prompt before it is sent to OpenAI.
4. On the return path, intercept the OpenAI response and run it through a toxicity checker. If it fails, return a standardized safety error instead of the text.
5. Write tests submitting known jailbreak prompts (e.g., "DAN" prompts) to verify your proxy successfully blocks them.