Request redaction
Hide sensitive text before sending messages to the business upstream or JEV. Use reversible encryption when the client needs the original values back.
What it solves §
Conversations, tool arguments, and tool results may contain email addresses, phone numbers, business identifiers, or keys. Request redaction applies regex rules to an outbound copy inside the gateway, reducing exposure to upstream services. It does not judge whether a request violates policy, and it is not merely log masking.
| Method | What upstream sees | When to use it |
|---|---|---|
| Fixed replacement | Your replacement text; the original cannot be restored | Hide information when its original value is not needed later |
| Reversible encryption | Stable ciphertext that keeps distinct original values distinguishable | Distinguish business entities or return original values to client-executed tools |
Under the same AccessKey, the same original value produces the same ciphertext so the model can preserve references. Different AccessKeys are isolated. The model can manipulate ciphertext but cannot understand its original meaning or use it to look up the original email address or account.
Configure rules §
- As an administrator, open Settings → Request redaction. Rules are global and independent of the experimental-feature switch. With no rules, no new redaction is applied.
- Start with presets for email, Chinese mobile numbers, common API keys, or private keys, or add a custom Go/RE2 regex. Presets are a starting point; narrow matches to your own data formats.
- Choose Fixed replacement and enter replacement text, or select Reversible encryption. Replacement text is literal: values such as $1 are not expanded into capture groups.
- Saved rules apply to new requests. Verify matches, responses, and tool calls with fictional data before using real traffic.
Overlapping matches in the original text are merged, and the earliest rule handles the merged range. Rules do not repeatedly match the previous rule's replacement. Avoid placing broad rules that match entire messages first.
Request and response flow §
- The client sends the original messages to GPT-Load. The gateway selects the target using the existing permissions, models, and routing configuration.
- After parameter overrides, the gateway replaces or encrypts matching outbound text. Both the business upstream and JEV receive the processed content.
- Upstream produces a response or tool call. Valid ciphertext in conversation responses is restored before reaching the client, including ordinary responses, SSE, and Responses WebSocket.
- Client-side tools receive restored arguments. When the next turn includes history or tool results, the gateway protects outbound content again.
Only complete ciphertext that authenticates under the current AccessKey can be restored. If the model rewrites, truncates, or misspells it, the gateway will not guess the original; that fragment may remain encrypted. Fixed replacement is never reversible.
Preserve signed thinking and history blocks intact. The gateway carries the restoration records needed to validate and restore the upstream text and signature on the next turn. Clients should retain these blocks unchanged rather than editing their signatures or contents.
A verifiable example §
Add the rule TEST-PROJECT-[0-9]+. First choose Fixed replacement with [PROJECT] as the replacement. Ask the model to repeat the fictional identifier below, then switch to Reversible encryption and send a new request.
curl http://127.0.0.1:3001/v1/chat/completions \
-H "Authorization: Bearer $GPT_LOAD_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"YOUR_MODEL","messages":[{"role":"user","content":"Repeat exactly: TEST-PROJECT-123"}]}'With fixed replacement, upstream sees only [PROJECT]. With reversible encryption, it sees ciphertext; if returned unchanged, the client sees TEST-PROJECT-123 again. This verifies the rule and restoration path, but does not guarantee the model will always follow the repeat instruction.
If you need to inspect the actual outbound payload, observe fictional data on a test upstream you control. Do not log production messages or real keys for debugging.
Coverage and limits §
- Supported message text, tool arguments, and tool results are processed, not every JSON field indiscriminately. Protocol fields such as model names, IDs, roles, and tool names retain their semantics.
- Images, audio, attachment contents, remote history absent from the request, and opaque encrypted content are not inspectable plaintext. Native numeric fields are not encrypted using string rules either.
- Chat, Responses, Anthropic, and Gemini conversations support response restoration. Non-conversation requests such as images, embeddings, and reranking only redact their supported outbound text; they do not offer the same response restoration.
- Client-executed tools can receive restored arguments. Upstream-hosted search, MCP, and code-execution tools still receive ciphertext and cannot depend on original values.
- Encryption increases text length and may increase token usage. Streaming output can also pause briefly while waiting for a complete ciphertext token or tool JSON fragment.
Redaction and AI Guardrails can be enabled together, but JEV reviews redacted content. Hidden original values are not sent to JEV in plaintext; choose rules that match your protection goals.
JEV AI Guardrails: decisions, warnings, and blocking →
Troubleshooting and keys §
For request_redaction_failed, check rule scope, text structure, and processing size limits. The gateway rejects requests it cannot process safely. If responses still contain ciphertext, check whether the model altered it, the AccessKey identity changed, or the protocol supports restoration.
Reversible encryption derives from the instance ENCRYPTION_KEY and AccessKey identity, without per-session mappings. Deleting rules does not destroy decryption capability, but it does not retroactively process upstream history. After disabling rules, do not assume that every streaming-history path will still trigger restoration.
Keep the database and original encryption master key to preserve the same AccessKey identity and decryption capability. Recreating an AccessKey or replacing the instance master key cannot restore ciphertext from the old identity.