Manage and Secure AI Traffic
API7 AI Gateway is the AI traffic management layer of API7 Gateway. It provides a unified control point between your applications and LLM providers, enabling platform engineers to centralize authentication, enforce guardrails, control costs, and gain observability across all AI traffic — without modifying application code.
This document provides an overview of AI Gateway capabilities, architecture, and supported providers.
Architecture
AI Gateway runs as a set of plugins on the API7 Data Plane, configured through the Control Plane. Requests flow through a processing pipeline before reaching the LLM provider:
The pipeline stages correspond to AI plugins running on the Data Plane. Each stage is optional — configure only what you need.
Key Capabilities
| Capability | Description | Plugin |
|---|---|---|
| LLM Proxying | Unified API to 8+ providers, including any OpenAI-compatible endpoint | ai-proxy (opens in Plugin Hub docs) |
| Multi-Model Routing | Load balancing, failover, health checks, and priority-based routing across models and providers | ai-proxy-multi (opens in Plugin Hub docs) |
| Protocol Conversion | Anthropic to OpenAI protocol translation with automatic response conversion, transparent to clients | ai-proxy (opens in Plugin Hub docs) |
| Prompt Engineering | Reusable templates and system prompt injection at the gateway layer | ai-prompt-decorator (opens in Plugin Hub docs), ai-prompt-template (opens in Plugin Hub docs) |
| Content Guardrails | PCRE-based prompt filtering and external toxicity detection | ai-prompt-guard (opens in Plugin Hub docs), ai-aws-content-moderation (opens in Plugin Hub docs), ai-aliyun-content-moderation (opens in Plugin Hub docs) |
| PII Redaction | Mask sensitive data in prompts before they reach LLM providers | ai-request-rewrite (opens in Plugin Hub docs) |
| Token Rate Limiting | Budget control per consumer, route, or model instance based on token counts | ai-rate-limiting (opens in Plugin Hub docs) |
| RAG Integration | Retrieval-Augmented Generation with Azure AI Search and Azure OpenAI embeddings | ai-rag (opens in Plugin Hub docs) |
| AI-Driven Transformation | Intelligent request rewriting powered by an LLM | ai-request-rewrite (opens in Plugin Hub docs) |
| AI Observability | Built-in logging (summaries and payloads) and Prometheus metrics for token accounting | ai-proxy (opens in Plugin Hub docs) |
| MCP Gateway | Expose OpenAPI services as MCP tools for AI agents | openapi-to-mcp (opens in Plugin Hub docs) |
Supported LLM Providers
API7 AI Gateway supports the following LLM providers with dedicated driver implementations:
| Provider | Provider Key | Guide |
|---|---|---|
| OpenAI | openai | Route Traffic to OpenAI |
| Azure OpenAI | azure-openai | Integrate Azure OpenAI |
| Anthropic | anthropic | Connect to Anthropic Claude |
| DeepSeek | deepseek | Route Traffic to DeepSeek |
| Google Gemini | gemini | Integrate Google Gemini |
| Vertex AI | vertex-ai | Route Traffic to Vertex AI |
| OpenRouter | openrouter | Access LLMs via OpenRouter |
| AWS Bedrock | bedrock | ai-proxy AWS Bedrock example (opens in Plugin Hub docs) |
| OpenAI-Compatible | openai-compatible | Connect Any OpenAI-Compatible LLM |
Any LLM API that follows the OpenAI chat completion format can be proxied using the openai-compatible provider type.
Next Steps
- Get Started — Proxy your first LLM request in 5 minutes.
- LLM Providers — Configure a specific provider.
- Use Cases and How-To Guides — Explore advanced capabilities like routing, guardrails, and cost control.