How gate detection works

Deterministic PII detection through rules, not models. Learn what gate catches, the documented gaps, and why this approach prioritizes reproducibility and data locality.

The two-layer detection pipeline

gate uses a deterministic, rules-based approach: regex patterns, column name heuristics, and checksums (like Luhn validation). Every redaction decision is traceable to an explicit rule, and detection happens entirely on-device.

Layer 1: SQL parsing

The first layer parses SQL queries to extract column names and flag those matching PII heuristics for forced redaction. This layer operates on column names alone and is best-effort: wildcards, aliases, and complex nested queries receive fallback coverage rather than perfect accuracy.

Layer 2: Value-level detection

Once query results come back, gate applies three checks:

Forced-column redaction: columns marked in Layer 1 are redacted unconditionally.
Column name tokenization: every column name is checked against ~50 PII categories (Government IDs, Financial, Contact, Health & medical, etc.).
Value patterns: regex and checksum validation (Luhn, ISBN, etc.) on the actual data.

Why rules, not a model?

gate uses deterministic rules instead of LLM-based detection because:

Locality: no data leaves the machine; detection is 100% local.
Reproducibility: the same input always produces the same redaction. No variance across model versions or API changes.
Speed: sub-10ms latency per query — fast enough for real-time interactive use.
Auditability: every redaction decision traces back to a specific rule, pattern, or heuristic that your team can review and tune.

The trade-off: rules can't catch PII in unstructured free-text prose. A language model might infer that "my eldest son, born in 1993, went to MIT" is biometric + education PII; a regex can't. For environments where control and reproducibility matter more than coverage, this is the right choice.

Documented detection gaps

gate is explicit about what it doesn't catch. Understanding these gaps is essential to configuring gate correctly for your schema:

Non-adversarial threat model only

gate protects against inadvertent PII leakage when an AI agent queries your database and includes sensitive data in its response. It does not protect against:

Prompt injection attacks: an agent maliciously instructed to bypass or disable redaction.
Compromised infrastructure: a malicious human operator on the machine running gate.
Pre-existing context: PII already embedded in system prompts, conversation history, or earlier turns.

Format-specific bypasses

Stripped formatting: SSN `123456789` (no dashes) is not matched by the SSN regex.
Regional variation: non-US phone formats often evade detection; AU/NZ formats are explicitly handled, but international numbers vary.
Encoding: base64 or URL-encoded PII passes through undetected.

Data format gaps

Unstructured prose: free-text columns containing narrative PII are not redacted (Layer 2 can catch values, but natural-language context-dependence requires a language model).
Non-JSON output: CSV, plain-text, or binary results bypass JSON-specific redaction.
MCP resource reads: gate does not intercept MCP tool use (resource reads or prompt messages), only SQL query interception.

Configuring for your schema

These gaps are why gate scan exists. Running it before connecting an AI agent to production gives you a report of which columns gate detects by name alone, and which you should manually mark for forced redaction.

The single highest-leverage configuration is wildcard_policy: reject: it prevents queries like SELECT * from returning unscanned data, forcing agents to name columns explicitly and allowing gate to apply both Layer 1 and Layer 2 detection.