Private Hybrid AI: Building an Automatic PII Redaction Proxy for External LLM APIs

Engineers want the reasoning horsepower of frontier cloud models, but security teams cannot tolerate the risk of leaking Personally Identifiable Information (PII) or confidential corporate assets. Relying entirely on local open-source models often leaves performance on the table, while sending raw prompts directly to commercial endpoints compromises your data perimeter.

You don’t have to choose between intelligence and compliance.

The middle path is a Private Hybrid AI Architecture: a self-hosted, inline redaction proxy that automatically intercepts outgoing prompts, strips or tokenizes sensitive entities using a fast local model, forwards sanitized text to external APIs, and re-hydrates the response before it reaches the end user.

The Hybrid Model: Local Redaction Meets Cloud Intelligence

A hybrid AI architecture splits prompt processing into two distinct phases: local privacy enforcement and external reasoning.

Instead of routing raw prompts over the open web, your applications point to a local proxy endpoint. The proxy inspects incoming payloads for PII—such as email addresses, Social Security numbers, API keys, and custom entity formats—using a high-speed, local Named Entity Recognition (NER) model (e.g., Microsoft Presidio or a fine-tuned spaCy pipeline running on ONNX).

┌─────────────────┐       ┌────────────────────────────────────────────────────────┐       ┌──────────────────────┐
│  Client App /   │       │               INLINE REDACTION PROXY                   │       │  External LLM API    │
│  Developer CLI  │       │                                                        │       │ (OpenAI / Anthropic) │
└────────┬────────┘       │  1. Intercept Request                                  │       └──────────────────────┘
         │                │  2. Local NER Scan (Presidio / spaCy)                  │                   ▲
         │                │  3. Tokenize PII -> [EMAIL_1], [IP_1]                  │                   │
         │  Raw Prompt    │  4. Store Mapping in Vault                             │  Sanitized Prompt │
         ├───────────────►│                                                        ├───────────────────┘
         │                │                                                        │
         │                │  5. Intercept Response                                 │
         │ Re-hydrated    │  6. Re-hydrate Tokens -> Original Values               │
         │    Response    │                                                        │
         ◄────────────────┤                                                        │
                          └────────────────────────────────────────────────────────┘

Once detected, the proxy replaces sensitive tokens with deterministic placeholders (e.g., john.doe@techelite.org becomes <EMAIL_1>). It stores the bidirectional mapping key in temporary, encrypted local memory (such as Redis with aggressive key expiration).

The sanitized prompt travels safely to the external API. When the external LLM returns its response referencing <EMAIL_1>, the proxy intercepts the payload, restores the original values, and serves the completed response back to the client. The cloud provider never sees raw data, and your application receives fully contextual answers.

Building the Pipeline: Deploying Presidio & FastAPI Sidecars

To minimize latency overhead, build your redaction proxy using an asynchronous Python framework (FastAPI) coupled with Microsoft Presidio or spaCy.

Step 1: Install Dependencies

Set up your proxy environment with local entity detection capabilities and an in-memory key-value store:

pip install fastapi uvicorn presidio-analyzer presidio-anonymizer redis httpx

Step 2: Configure the Asynchronous Redaction Engine

Initialize the Presidio analyzer and anonymizer engines inside a lightweight proxy server. The script below defines custom anonymization operators that swap sensitive data with reversible hash tokens:

from fastapi import FastAPI, Request, Response
from presidio_analyzer import AnalyzerEngine
from presidio_anonymizer import AnonymizerEngine
from presidio_anonymizer.entities import OperatorConfig
import httpx
import redis
import json

app = FastAPI()
analyzer = AnalyzerEngine()
anonymizer = AnonymizerEngine()
r = redis.Redis(host='localhost', port=6379, db=0)

EXTERNAL_LLM_URL = "https://api.openai.com/v1/chat/completions"

@app.post("/v1/chat/completions")
async def proxy_llm(request: Request):
    body = await request.json()
    raw_prompt = json.dumps(body)
    
    # 1. Analyze text for PII
    results = analyzer.analyze(text=raw_prompt, entities=["EMAIL_ADDRESS", "PHONE_NUMBER", "IP_ADDRESS"], language='en')
    
    # 2. Anonymize entities with indexed placeholders
    anonymized_result = anonymizer.anonymize(
        text=raw_prompt,
        analyzer_results=results
    )
    
    sanitized_body = json.loads(anonymized_result.text)
    
    # 3. Forward sanitized payload to external LLM
    headers = {k: v for k, v in request.headers.items() if k.lower() != 'host'}
    async with httpx.AsyncClient() as client:
        llm_response = await client.post(
            EXTERNAL_LLM_URL,
            json=sanitized_body,
            headers=headers,
            timeout=60.0
        )
        
    return Response(content=llm_response.content, status_code=llm_response.status_code)

Re-hydration Architecture: Rebuilding Context Without Leaks

Simple replacement techniques like blanket redacting text to [REDACTED] break model reasoning. If a prompt contains two distinct email addresses and both become [REDACTED], the cloud LLM cannot differentiate between the sender and recipient when constructing a response.

1. Deterministic Token Mapping

Use indexed placeholders (<EMAIL_1>, <EMAIL_2>, <PERSON_1>). This preserves the structural relationships within your data, allowing the external model to follow multi-entity logic perfectly.

2. Session-Bound Ephemeral Storage

Store entity mapping tables in Redis bound strictly to the unique request ID, with a Time-To-Live (TTL) of no more than 300 seconds:

import uuid

session_id = str(uuid.uuid4())
mapping_key = f"pii_map:{session_id}"

# Store mapping dictionary with auto-expiration
r.setex(mapping_key, 300, json.dumps(entity_mapping_table))

3. Outbound Response Re-hydration

When the external LLM responds, run a reverse string replacement pass using the mapping stored under mapping_key. Once the re-hydrated text builds successfully, delete the Redis key immediately to clear memory residual state.

Pro-Tip: Neutralizing Semantic PII & Code Base Leaks

The Hidden Mistake: Regex & Standard NER Over-reliance

Standard regex patterns catch structured data like credit card numbers or email strings easily. However, they consistently miss semantic PII and proprietary code identifiers—such as internal server hostnames (db-primary.internal.corp), custom JWT secret formats, or private API keys embedded in code blocks.

If an engineer pastes a stack trace containing an internal domain or AWS access key format not covered by standard SpaCy models, your proxy passes it straight to the cloud.

The Fix: Regex Pattern Extensions + Local Small Language Models (SLMs)

Combine pattern matching with custom regex definitions and lightweight local SLMs (such as Microsoft Phi-3 or Llama-3-8B-Instruct) tasked exclusively with entity detection.

Add custom regex patterns to your Presidio analyzer for internal infrastructure assets:

from presidio_analyzer import Pattern, PatternRecognizer

# Custom recognizer for internal server naming conventions
internal_host_pattern = Pattern(name="internal_host", regex=r"[a-zA-Z0-9-]+\.internal\.corp", score=0.95)
host_recognizer = PatternRecognizer(supported_entity="INTERNAL_HOST", patterns=[internal_host_pattern])

analyzer.registry.add_recognizer(host_recognizer)

For complex unstructured text, route the prompt through a local 8B model running on Ollama first. Instruct it to output only a JSON list of identified sensitive terms before the proxy performs string replacement. This hybrid double-pass guarantees high accuracy without sacrificing execution speed.

Secure Your Data Flow Today

Building a zero-trust AI workflow doesn’t require locking yourself out of modern cloud AI capabilities. By deploying an inline PII redaction proxy, you create an unyielding security perimeter around your data pipelines.

Take these concrete steps to implement your proxy:

1. Audit outbound LLM calls: Identify every service or developer environment currently hitting third-party AI APIs.
2. Deploy Presidio or custom NER sidecars: Host lightweight, local entity-detection services close to your applications.
3. Establish token mapping & ephemeral storage: Ensure placeholders preserve logic while map keys expire automatically in memory.
4. Enforce proxy routing: Update your client SDKs and application base URLs to route all outbound AI requests through your internal proxy boundary.

With an automated proxy sitting between your infrastructure and the cloud, you can safely harness cloud LLM reasoning while keeping your sensitive data entirely under your control.