Higress

Higress AI Gateway

Higress AI Gateway extends the core Higress gateway with AI-specific capabilities including LLM proxy routing, multi-provider support for OpenAI-compatible APIs, prompt templating, token-based rate limiting, semantic caching, and AI observability. It enables teams to manage and govern traffic to large language model services.

GitHub

Documentation

📖
Documentation
https://higress.io/en-us/docs/plugins/ai/

SDKs

📦
GitHubRepository
https://github.com/alibaba/higress

Other Resources

🔗
Getting Started
https://higress.io/en-us/docs/user/ai-proxy/

API entry from apis.yml

apis.yml Raw ↑ 
aid: higress:higress-ai-gateway
name: Higress AI Gateway
description: Higress AI Gateway extends the core Higress gateway with AI-specific capabilities including
  LLM proxy routing, multi-provider support for OpenAI-compatible APIs, prompt templating, token-based
  rate limiting, semantic caching, and AI observability. It enables teams to manage and govern traffic
  to large language model services.
humanURL: https://higress.io/en-us/docs/plugins/ai/
tags:
- AI
- API Gateway
- Cloud Native
- LLM
properties:
- type: Documentation
  url: https://higress.io/en-us/docs/plugins/ai/
- type: Getting Started
  url: https://higress.io/en-us/docs/user/ai-proxy/
- type: GitHubRepository
  url: https://github.com/alibaba/higress