Apache APISIX AI Gateway
Use Apache APISIX as an open-source LLM gateway and proxy for model routing, load balancing, retries, fallback, token rate limiting, security, and observability.
What it is
Apache APISIX AI Gateway Use Apache APISIX as an open-source LLM gateway and proxy for model routing, load balancing, retries, fallback, token rate limiting, security, and observability. One gateway, two workloads Keep the routing, security, observability, and operations model your teams already use while adding LLM traffic. Open-source data plane, open plugin model, and portable configuration for APIs and AI workloads. Authentication, traffic control, observability, serverless, and AI plugins. Load balancing, fallback, token controls, RAG, prompt policies, moderation, and auditing are all available as open-source plugins. Built for production AI traffic Route across OpenAI, DeepSeek, Claude, Mistral, Gemini, and other providers with health checks and weighted balancing. Control token consumption by Route, Service, Consumer, Consumer Group, or custom attributes in standalone and cluster deployments.
What to verify before adopting
- Confirm current pricing, usage limits, supported regions, and contract terms on the official website.
- Test integrations, export paths, access controls, and failure recovery with low-risk data first.
- Review privacy, data retention, security, and compliance requirements for your market.
- Keep a migration or rollback plan before making the product part of a critical workflow.