Skip to content
Danube Messaging

Kubernetes

Deploy a Danube messaging cluster on Kubernetes and verify it with a producer and consumer. This guide uses a local Kind cluster for simplicity, but the same Helm charts work on any Kubernetes cluster (EKS, GKE, AKS, etc.).

Overview

Section titled “Overview”

A Danube deployment consists of two Helm charts:

ChartWhat it deploys
danube-envoyAn Envoy gRPC proxy that routes client requests to the correct broker
danube-coreDanube brokers (StatefulSet) with embedded Raft consensus, and Prometheus for metrics

The proxy is installed first because the brokers need to know its external address at startup. This address is called the connectUrl, it tells clients how to reach the cluster from outside Kubernetes.

Prerequisites

Section titled “Prerequisites”

Deploy Danube

Section titled “Deploy Danube”

  1. Create a Kubernetes cluster (skip if you already have one)

    Create Kind cluster
    kind create cluster
    kubectl cluster-info --context kind-kind

  2. Add the Danube Helm repository

    Terminal window
    helm repo add danube https://danube-messaging.github.io/danube_helm
    helm repo update

    This makes two charts available: danube/danube-envoy and danube/danube-core.

  3. Install the Envoy proxy

    The Envoy proxy is the single entry point for all client traffic. It handles gRPC routing so that each request reaches the broker that owns the target topic.

    Terminal window
    kubectl create namespace danube
    helm install danube-envoy danube/danube-envoy -n danube

    Wait for the proxy pod to be ready:

    Terminal window
    kubectl get pods -n danube -w

  4. Discover the proxy address

    The proxy service is exposed as a Kubernetes NodePort:

    Get proxy address
    PROXY_PORT=$(kubectl get svc danube-envoy -n danube \
      -o jsonpath='{.spec.ports[?(@.name=="grpc")].nodePort}')
    NODE_IP=$(kubectl get nodes \
      -o jsonpath='{.items[0].status.addresses[?(@.type=="InternalIP")].address}')
    echo "Proxy address: ${NODE_IP}:${PROXY_PORT}"

    Save this address for the next step and for connecting clients.

  5. Install Danube core

    Create a ConfigMap from the broker config, then install the chart with the proxy address:

    Create ConfigMap
    kubectl create configmap danube-broker-config \
      --from-file=danube_broker.yml=danube_broker.yml \
      -n danube
    Install Danube core
    helm install danube-core danube/danube-core -n danube \
      -f values-minimal.yaml \
      --set broker.externalAccess.connectUrl="${NODE_IP}:${PROXY_PORT}"

    This deploys:

    • 3 broker pods (StatefulSet) with persistent storage and embedded Raft consensus
    • 1 Prometheus pod for metrics collection

    Wait for all pods to be ready:

    Terminal window
    kubectl get pods -n danube -w

    Expected output:

    Expected pods
    NAME                                      READY   STATUS    AGE
    danube-core-broker-0                      1/1     Running   2m
    danube-core-broker-1                      1/1     Running   2m
    danube-core-broker-2                      1/1     Running   2m
    danube-core-prometheus-xxxxxxxxx          1/1     Running   3m
    danube-envoy-xxxxxxxxx                    1/1     Running   5m

  6. Verify the deployment

    Check that the brokers registered in proxy mode:

    Terminal window
    kubectl logs danube-core-broker-0 -n danube | grep "broker registered"

    You should see each broker reporting a unique broker_url (its internal DNS name) and a shared connect_url (the Envoy proxy address).

Produce and Consume Messages

Section titled “Produce and Consume Messages”

Use danube-cli to send messages through the proxy:

Terminal 1: Produce messages
danube-cli produce \
  -s http://${NODE_IP}:${PROXY_PORT} \
  -t /default/test_topic \
  -m "Hello from Danube" -c 5
Terminal 2: Consume messages
danube-cli consume \
  -s http://${NODE_IP}:${PROXY_PORT} \
  -t /default/test_topic \
  -m test_sub

Now produce more messages in Terminal 1, the consumer receives them in real time.

How Proxy Mode Works

Section titled “How Proxy Mode Works”

In a multi-broker cluster, each topic is owned by a specific broker. When a client connects, it needs to reach the right broker for its topic.

  1. The client connects to the Envoy proxy (the connectUrl)

  2. It sends a topic lookup request, which Envoy round-robins to any broker

  3. The broker responds with its internal address (broker_url) for that topic

  4. On subsequent requests, the client includes an x-danube-broker-url gRPC metadata header with the target broker’s internal address

  5. Envoy’s Dynamic Forward Proxy reads this header, resolves the broker’s internal DNS name, and routes the request directly to the correct pod

This means clients only need a single external address (the proxy) regardless of how many brokers are in the cluster.

Inspect Cluster State

Section titled “Inspect Cluster State”
Port-forward and inspect
# Port-forward the admin API of any broker
kubectl port-forward danube-core-broker-0 50051:50051 -n danube


# In another terminal
danube-admin cluster status
danube-admin brokers list

Access Prometheus

Section titled “Access Prometheus”
Terminal window
kubectl port-forward svc/danube-core-prometheus 9090:9090 -n danube

Open http://localhost:9090 in your browser to query Danube metrics.

Cleanup

Section titled “Cleanup”
Remove everything
helm uninstall danube-core -n danube
helm uninstall danube-envoy -n danube
kubectl delete namespace danube
kind delete cluster

This removes all Kubernetes resources including PersistentVolumeClaims (since the namespace is deleted).