Skip to main content Skip to search Skip to main navigation
Contact

Kong AI Gateway

Product information "Kong AI Gateway"

Kong AI Gateway is the AI-specific layer of the Kong API platform. It sits in front of the large language models an organization already uses — OpenAI, Azure OpenAI, Amazon Bedrock, Google Vertex AI, Anthropic, Mistral or a self-hosted model — and presents all of them through a single, OpenAI-compatible endpoint. Credentials, rate limits, request routing, caching and observability are configured once on the gateway rather than in every client, so the teams consuming AI never handle upstream provider keys at all.

The Kong AI Gateway Integration connects UAC workflows to that gateway. Because Kong normalizes every upstream behind one API, a single integration reaches whichever model your gateway fronts today and whichever it is repointed to tomorrow, with no change to your task definitions. Governance stays where your platform team already manages it: the spend caps, model allow-lists and audit trail configured in Kong apply to automation traffic exactly as they do to application traffic.

Beyond single-prompt inference, the integration supports autonomous agentic execution over the Model Context Protocol (MCP). Tools published through Kong's MCP routes are discovered at runtime and offered to the model, which then decides which to call and in what order; the integration dispatches each call, feeds the result back, and repeats until the model produces a final answer or a configured turn limit is reached. Prompts can be enriched by files or UAC Variables and outputs can be persisted for reuse across multiple tasks and workflows — enabling you to chain AI operations together or share insights across your automation ecosystem.

Key Features

  1. Provider-agnostic by design: one integration reaches any model Kong fronts through its OpenAI-compatible AI Proxy routes, so switching or load-balancing upstream models requires no change to your tasks.
  2. Autonomous agentic automation over MCP, with tool aggregation across multiple MCP servers, an optional allow-list restricting which tools a task may invoke, and a configurable turn limit.
  3. Dedicated "List MCP Tools" action to inventory the tools every connected MCP server publishes, for building and auditing agentic tasks.
  4. Dynamic discovery through the Konnect API or, for Self-Managed gateways, the Kong Admin API populates the AI Gateway, LLM Service, Model, MCP Connection and Allowed Tools fields from the gateway's own configuration, with free-text entry always available as a fallback.
  5. Two authentication methods matching Kong's own plugins: API key authentication with a configurable header name, and OAuth2 client credentials with per-route token caching and transparent refresh.
  6. Enrich prompts with data from files and UAC Variables, and save the latest response or the entire conversation for use throughout your automation ecosystem.
  7. Structured output support with JSON schemas for predictable, machine-consumable responses on models that support this capability.
  8. Provides "Advanced Options" to tune the behavior of the selected model — temperature, top P, frequency and presence penalties, maximum tokens, seed and stop sequences.
  9. Dry Run mode renders the fully resolved prompt and, for agentic tasks, discovers the available tools without issuing a billable model call.
  10. Token usage is reported on the task instance and, on agents with OpenTelemetry enabled, emitted as gen_ai.client.token.usage metrics following the OpenTelemetry GenAI semantic conventions.
Extension Name: ue-kong-ai-gateway
Universal Template Name: Kong AI Gateway
Vendor Product: Kong AI Gateway
Version: 1.0.0
Vendor: Kong
Compatibility : UC/UA 7.8 and above
Workflow

Use Case 1: AI Briefing

ue_kong_ai_gateway_uc1

Description

This workflow asks an AI Agent a question and lets it answer using the MCP tools published through Kong AI Gateway. The first Kong AI Gateway Task lists the MCP tools available to the AI Agent, which confirms the gateway and its MCP Servers are reachable without consuming any LLM tokens. Only when that succeeds does the second Kong AI Gateway Task run the AI Agent, which calls those tools to gather facts, answers the question, and saves the answer as a Markdown file on the Agent.

The question, the Kong routes, the model and the report location are all UAC Variables, so the same workflow can be scheduled as a recurring briefing or launched on demand with a different question.

Business Value

  • Put AI-generated answers, grounded in live data from your own MCP tools, into scheduled automation with no custom code.
  • Avoid paying for LLM tokens when the gateway or its MCP Servers are unavailable: the workflow stops at the free tool check.
  • Keep a record of which tools the AI Agent had access to on every run, for auditing.
  • Track AI spend per run through the token usage reported on each task instance.
  • Reuse one workflow for many questions by changing a single variable.

Technical Capabilities

  • Uses the "List MCP Tools" action to inventory the tools each Kong MCP route publishes, without a model call.
  • Uses the "Agentic MCP" action to let the model decide which tools to call and in which order, capped by a configurable turn limit.
  • Authenticates with a Kong API Key, while Kong itself holds the credentials of the LLM provider and every MCP Server.
  • Reports prompt, completion and total tokens, turns used and tool-call count as output fields, and the full tool-call transcript in the extension output.
  • Saves the final answer to a file on the Agent for use by downstream tasks or other systems.