For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.

Chatbot UI

Deploy Chatbot UI in Kubernetes and route its LLM traffic through agentgateway.

Deploy Chatbot UI in Kubernetes and route its LLM traffic through agentgateway to keep API keys server-side.

Before you begin

1. Set up an agentgateway proxy.
2. Set up access to the OpenAI LLM provider.

Get the gateway URL

export INGRESS_GW_ADDRESS=$(kubectl get svc -n agentgateway-system agentgateway-proxy \
  -o jsonpath='{.status.loadBalancer.ingress[0].ip}')

echo "Gateway address: $INGRESS_GW_ADDRESS"

export INGRESS_GW_ADDRESS=$(kubectl get svc -n agentgateway-system agentgateway-proxy \
  -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')

echo "Gateway address: $INGRESS_GW_ADDRESS"

kubectl port-forward -n agentgateway-system svc/agentgateway-proxy 8080:80

Set up the OpenAI backend

1. Export your OpenAI API key.

   export OPENAI_API_KEY="your-key-here"

2. Create a Kubernetes Secret for your API key.

   kubectl apply -f- <<EOF
   apiVersion: v1
   kind: Secret
   metadata:
     name: openai-secret
     namespace: agentgateway-system
   type: Opaque
   stringData:
     Authorization: $OPENAI_API_KEY
   EOF

3. Create an AgentgatewayBackend for OpenAI.

   kubectl apply -f- <<EOF
   apiVersion: agentgateway.dev/v1alpha1
   kind: AgentgatewayBackend
   metadata:
     name: openai
     namespace: agentgateway-system
   spec:
     ai:
       provider:
         openai: {}
     policies:
       auth:
         secretRef:
           name: openai-secret
   EOF

4. Create an HTTPRoute to forward traffic to the backend.

   kubectl apply -f- <<EOF
   apiVersion: gateway.networking.k8s.io/v1
   kind: HTTPRoute
   metadata:
     name: openai
     namespace: agentgateway-system
   spec:
     parentRefs:
       - name: agentgateway-proxy
         namespace: agentgateway-system
     rules:
       - matches:
         - path:
             type: PathPrefix
             value: /
         backendRefs:
         - name: openai
           namespace: agentgateway-system
           group: agentgateway.dev
           kind: AgentgatewayBackend
   EOF

Deploy Chatbot UI

Deploy Chatbot UI as a Kubernetes workload and point it at the agentgateway service.

kubectl apply -f- <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: chatbot-ui
  namespace: agentgateway-system
spec:
  replicas: 1
  selector:
    matchLabels:
      app: chatbot-ui
  template:
    metadata:
      labels:
        app: chatbot-ui
    spec:
      containers:
      - name: chatbot-ui
        image: ghcr.io/mckaywrigley/chatbot-ui:main
        ports:
        - containerPort: 3000
        env:
        - name: OPENAI_API_KEY
          value: "placeholder"
        - name: OPENAI_API_HOST
          value: "http://agentgateway-proxy.agentgateway-system.svc.cluster.local"
---
apiVersion: v1
kind: Service
metadata:
  name: chatbot-ui
  namespace: agentgateway-system
spec:
  selector:
    app: chatbot-ui
  ports:
  - port: 3000
    targetPort: 3000
EOF

The following table describes each environment variable:

Verify the connection

1. Port-forward to Chatbot UI.

   kubectl port-forward -n agentgateway-system svc/chatbot-ui 3000:3000

2. Open http://localhost:3000 and send a message.

3. Confirm the request appears in the agentgateway proxy logs.

   kubectl logs deployment/agentgateway-proxy -n agentgateway-system --tail=5

   You should see a log entry showing the request was forwarded to the OpenAI endpoint with the configured model:

   info  request gateway=agentgateway-system/agentgateway-proxy listener=http route=agentgateway-system/openai endpoint=api.openai.com:443 http.method=POST http.path=/v1/chat/completions http.status=200 protocol=llm gen_ai.operation.name=chat gen_ai.provider.name=openai gen_ai.request.model=gpt-4o gen_ai.usage.input_tokens=4569 gen_ai.usage.output_tokens=10 duration=2242ms

Next steps