Introduction

Knull - AI Control Plane

Knull is a high-performance, standalone AI Control Plane designed to run as a local or edge-resident control plane and data plane for AI traffic. It provides a unified interface for routing requests to multiple LLM providers with built-in observability, authentication, and budget enforcement.

How it Works

Knull consists of a Control Plane (Knull Core) and a Data Plane (Envoy Proxy).

  1. Unified API: Your applications send OpenAI-format requests to Knull.
  2. Routing & Policy: Knull identifies the client via API key, checks their token budget, and selects the optimal backend model.
  3. Translation: If the backend is not OpenAI (e.g., Anthropic or Bedrock), Knull's External Processor (ExtProc) translates the request on the fly.
  4. Observability: Knull counts every token and logs usage to its internal database.

Key Features

  • Standalone Binary: Single executable manages Envoy and the External Processor without Kubernetes.
  • Multi-Provider Support: Unified access to Azure OpenAI, Anthropic, AWS Bedrock, GCP VertexAI, and more.
  • Token Budgets: Enforce granular token limits per API key and per model.
  • MCP Support: Native Model Context Protocol proxying for AI tool integration.
  • Management UI: Built-in Next.js dashboard for monitoring and configuration.

Quick Start

1. Build and Run

# Build the binary
make build
 
# Start Knull with an example config
./bin/knull run examples/knull.yaml

2. Configure a Provider

Set your API key as an environment variable (Knull automatically substitutes these in YAML):

export OPENAI_API_KEY="sk-..."

3. Test the Gateway

curl http://localhost:1975/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "Hello Knull!"}]
  }'

Next Steps

Architecture