Documentation

Auth Check: Validates the Bearer token against the secret store. Rejects unauthorized requests immediately.

Injection Check: Runs 30+ regex/heuristic patterns against the prompt content. Produces a 0.0-1.0 injection score. Prompts above the threshold (default 0.5) are blocked before any LLM call.

Custom Rules: Evaluates user-defined blocking rules stored in the KV store. Rules support contains, regex, starts_with, equals, and not_contains operators with AND/OR/NOT logic.

Cache Check: Looks up an exact-match cache key (hash of model + messages + temperature). SimpleCache is per-POP with microsecond latency. KV Store is global with millisecond latency.

Detect Modality: Scans message content for images (OpenAI image_url format), OCR requests, image generation prompts ("create an image", "generate a picture"), and code blocks. Detected modality determines which capability filters apply during routing.

Classify: Analyzes prompt complexity to assign one of 9 tiers (fast, balanced, quality, reasoning, code, vision, ocr, image_gen, embedding). Uses keyword heuristics, question detection, code block detection, image detection, and content analysis.

Route: Scores all 26 available models for the assigned tier using user preference weights (cost, speed, quality) and internal quality rankings (Claude Opus 4 at the top, down through GPT-5.4, Claude Sonnet 4, DeepSeek R1, GPT-4o, and so on). Models without the required capability (e.g., vision, ocr, image_gen) are automatically excluded. Picks the highest-scoring model from connected providers with a fallback chain if the primary provider fails.

Audit Log: Writes a fire-and-forget entry to the KV store keyed by tenant and date. Zero added latency to the response.

9 Routing Tiers

• Fast — Simple questions, greetings, factual lookups. Models: Gemma 9B (Groq), GPT-4o-mini, Mistral Small, Phi-3 (Ollama).
• Balanced — Summarization, translation, general Q&A. Models: Llama 3.3 70B (Groq/Together), DeepSeek Chat, GPT-4o-mini.
• Quality — Creative writing, detailed analysis. Models: Claude Sonnet 4, GPT-4o, DeepSeek V4, Mistral Large.
• Reasoning — Complex math, logic, multi-step proofs. Models: Claude Opus 4, GPT-5.4, DeepSeek R1.
• Code — Programming, debugging, code generation. Models: DeepSeek Coder, GPT-4o, Claude Sonnet 4.
• Vision — General image understanding (describe, analyze, compare images). Models: GPT-4o, Claude Sonnet 4. For text extraction, see OCR tier. For image creation, see Image Generation tier.
• OCR — When text extraction is needed, ModelRouter routes to Mistral OCR -- a purpose-built document parser that achieves state-of-the-art accuracy at $0.002/page. This is 10x cheaper and more accurate than sending OCR tasks to GPT-4o. Specialized OCR models preserve table structure, formulas, and layout that general-purpose models lose. Models: Mistral OCR, DeepSeek OCR, GPT-4o (fallback).
• Image Generation — Image generation routes to FLUX models via Together AI -- faster, cheaper, and higher quality than DALL-E 3. FLUX.1 schnell generates images in 2 seconds. FLUX.1 Pro matches or exceeds DALL-E 3 quality at lower cost. Models: FLUX.1 Schnell (free), FLUX.1 Pro, DALL-E 3 (fallback).
• Embedding — Vector embeddings for search/RAG. Models: text-embedding-3-small, nomic-embed-text.

Model Quality Rankings

Models are ranked by overall quality. These rankings are used as one of three factors (alongside cost and speed) when scoring models within a tier:

1. Claude Opus 4 — Highest quality reasoning and nuanced analysis
2. GPT-5.4 — Strongest general-purpose, large context window
3. Claude Sonnet 4 — Excellent balance of quality and cost, supports vision
4. DeepSeek R1 — Specialized reasoning with transparent thinking chains
5. GPT-4o — Strong multimodal, supports vision and OCR
6. DeepSeek Coder — Code specialist, very low cost
7. DeepSeek V4 — Solid general-purpose at included-tier pricing
8. Mistral Large — Good quality with function calling support
9. Llama 3.3 70B — Strong open-source, free on Groq
10. GPT-4o-mini — Fast and cheap for simpler tasks

How a Prompt is Classified

When a prompt arrives, ModelRouter reads it and decides what kind of work it is. The classifier runs through checks in priority order -- the first match wins:

How a Model is Selected Within a Tier

Once the tier is chosen, ModelRouter scores each available model in that tier using your preference weights (the cost, speed, and quality sliders):

Models are ranked 0 to 1 within their tier for each dimension. The highest-scoring model wins.

• If you set Cost to 100%: the cheapest model in the tier is always chosen.
• If you set Quality to 100%: the highest-rated model in the tier is always chosen.
• If you set Speed to 100%: the lowest-latency model in the tier is always chosen.

Additional constraints:

• Only models from your connected providers are considered (you must have the API key configured).
• Only models with the required capabilities are considered (e.g., a vision request only considers vision-capable models).
• If the primary model fails (provider error, timeout), the next-best model in the tier is tried automatically (fallback chain across all 7 providers).

Worked Example

A user sends: "Write a Python function to sort a list"

Vision Detection

ModelRouter reads each message's content blocks. In the OpenAI format, a message can contain multiple content blocks:

If any content block has type image_url or image, the prompt is classified as VISION. This overrides all other classification -- even if the text says "write code", the presence of an image means vision routing.

Within the VISION tier, only models with vision capability are considered:

• GPT-4o (OpenAI) — supports image URLs and base64
• Claude Sonnet 4 (Anthropic) — supports image URLs and base64

If the text also contains OCR keywords ("extract text", "read the text", "OCR"), it is classified as OCR instead, which routes to specialized OCR models like Mistral OCR and DeepSeek OCR that preserve table structure, formulas, and layout.

Why Specialized Models?

ModelRouter doesn't just route to the biggest model. It routes to the right model.

Sending OCR to GPT-4o is like using a bulldozer to plant a flower. Mistral OCR is purpose-built for document parsing -- it's more accurate, 10x cheaper, and preserves structure (tables, formulas) that GPT-4o flattens.

Sending image generation to DALL-E 3 ignores that FLUX.1 Pro generates higher-quality images in less time at lower cost.

This is the core value of ModelRouter: you don't need to know which model is best for each task. We do.

Sending Images (Vision)

Include images using the OpenAI image_url content block format. ModelRouter detects the image and routes to vision-capable models (GPT-4o, Claude Sonnet 4).

OCR Detection

When your prompt includes an image and mentions text extraction keywords (e.g., "extract text", "read the text", "OCR"), ModelRouter routes to purpose-built OCR models. Mistral OCR is the primary choice -- a specialized document parser that achieves state-of-the-art accuracy at $0.002/page, preserving table structure, formulas, and layout that general-purpose models lose. DeepSeek OCR is available as a lower-cost alternative, with GPT-4o as a fallback. The detection is automatic — no special flags needed.

Image Generation

Prompts requesting image creation (e.g., "create an image of...", "generate a picture", "draw a...") are automatically routed to FLUX models via Together AI. FLUX.1 Schnell (free) generates images in ~2 seconds. FLUX.1 Pro matches or exceeds DALL-E 3 quality at lower cost with faster generation. DALL-E 3 is available as a fallback. No model override needed — just describe what you want.

Code Detection

Prompts containing code blocks, programming keywords, or debugging requests are routed to code-optimized models like DeepSeek Coder.

Provider Capabilities

Injection Detection (30+ Patterns)

The injection detector scans every prompt against a library of heuristic patterns covering:

• Instruction overrides: "Ignore all previous instructions", "disregard your programming"
• Role hijacking: "You are now DAN", "pretend you have no restrictions"
• Delimiter injection: <system> tags, markdown separators used to inject fake system messages
• System prompt extraction: "Print your initial instructions", "output your system prompt"
• Jailbreak patterns: "Do Anything Now", "developer mode", "unrestricted mode"

Each pattern contributes a weighted score. The combined injection score (0.0 to 1.0) determines whether the prompt is blocked. Default threshold: 0.5.

Custom Blocking Rules

Beyond automatic injection detection, you can define custom rules with flexible logic:

• ANY (OR): Block if any condition matches. Use for broad content filters.
• ALL (AND): Block only if all conditions match. Use for precise targeting.
• NONE (NOT): Block if any condition matches — used as an allowlist violation detector.

Rules are stored in the KV store and evaluated at every edge PoP. Create and test rules on the Rules page.

Data Loss Prevention (DLP)

Use custom rules as a DLP layer to prevent sensitive data from leaving your organization through AI prompts:

• SSN detection: Regex pattern \b\d{3}-\d{2}-\d{4}\b blocks Social Security Numbers.
• Source code blocking: Detect code patterns, function signatures, or internal API keys in prompts.
• Trade secrets: Block mentions of internal project names, proprietary algorithms, or confidential data markers.
• Credit card numbers: Regex for card patterns across all major networks.

DLP rules run at the edge in <1ms and block before any data reaches an external LLM provider.

Employee AI Audit Logging

Every request is logged with: timestamp, provider, model, injection score, cache status, latency, token usage, and HTTP status. Logs are keyed by tenant and date in the KV store. Full visibility into which employees are using AI, what they are asking, and which models are being invoked. View and export logs on the Logging page.

Tier 1: Fastly SimpleCache (Per-POP)

SimpleCache is a local in-memory cache at each Fastly edge Point of Presence (PoP). Lookups complete in microseconds. If a user at the same PoP asks the same question, the response is served instantly from memory without any network calls.

Tier 2: Fastly KV Store (Global)

The KV Store is a globally distributed key-value database. If the SimpleCache misses, ModelRouter checks the KV Store. KV lookups take milliseconds and work across all PoPs worldwide. This means a cache entry written in San Francisco is readable in Tokyo.

When Caching Applies

• Caching is active when temperature=0 (deterministic output)
• Streaming responses (stream=true) are not cached
• The cache key is a SHA-256 hash of: model + messages array + temperature
• Cache entries expire based on configurable TTL (default: 1 hour for SimpleCache, 24 hours for KV)

Rule Structure

Operators

Logic Types

• any (OR): The rule fires if any one condition matches. Best for broad content filters. Example: block prompts containing "SSN" OR "credit card number".
• all (AND): The rule fires only if every condition matches. Best for precise targeting. Example: block prompts where content contains "extract" AND content contains "database".
• none (NOT): The rule fires if any condition matches. Used for allowlist enforcement. Example: block if role is NOT "user" (i.e., block system/assistant injection).

Fields

• content: The text content of the message being evaluated.
• role: The message role (user, system, assistant).
• model: The requested model name.

Planned Features

• Per-tenant Fastly services: Each tenant gets a dedicated Compute service with isolated configuration, secret stores, and backend definitions.
• Isolated KV stores: Tenant-specific KV stores for rules, audit logs, and cache. No data leakage between tenants.
• Provisioning API: Programmatically create, configure, and tear down tenant services via a management API. Automate onboarding.
• White-labeling: Custom domains, branding, and UI themes per tenant. Each tenant can point their own domain at their ModelRouter service.
• Usage metering: Per-tenant request counts, token usage, and cost tracking for billing and chargeback.
• Role-based access: Tenant admins manage their own rules and settings. Platform admins manage all tenants.