Skip to content
Danube Messaging

Cluster & Topic Configuration Workflow

Danube v2 eliminates static broker YAML files (danube_broker.yml / edge.yaml) in favor of a clear 4-layer configuration model. Each concern lives at its natural operational boundary—hardware stays local to each host, cluster policies replicate automatically over Raft, topics declare workload intents, and recovery exports provide an audited break-glass escape hatch.

Layer 1: Node Startup (`danube up`)        ─▶ Local ports, disks, TLS/auth, node role, seed nodes
Layer 2: Cluster State (Raft)              ─▶ Shared storage, write buffer, WAL policy, load manager
Layer 3: Topic Creation (`topics create`)  ─▶ Partitions, schema subject, storage mode & `--intent`
Layer 4: Break-Glass Recovery              ─▶ `export-config` snapshots + `--recovery-config` override

Layer 1: Per-Node Hardware & Network (danube up)

Section titled “Layer 1: Per-Node Hardware & Network (danube up)”

At boot time, a broker only needs to know its local identity, storage paths, bind addresses, security mode, and how to join the cluster. Every other operational setting lives inside the Raft cluster state.

By default, Client gRPC, Schema Registry, and Admin gRPC multiplex onto a single port (6650), so there is no separate admin port to configure unless you want firewall isolation via --admin-addr.

Terminal window
danube up \
  --mode standalone \
  --data-dir ./danube-data \
  --broker-addr 127.0.0.1:6650

Key per-node flags on danube up:

  • Storage mounts: --data-dir (Raft metadata and local segments, default ./danube-data) and optional --wal-dir (isolates the write-ahead log onto a dedicated NVMe volume).
  • Network & routing: --broker-addr (default 127.0.0.1:6650), optional --admin-addr (splits Admin gRPC onto a dedicated port; unset by default), --advertised-addr / --connect-url, --raft-addr (default port 7650), --prom-exporter.
  • Cluster membership & specialization: --seed-nodes, --join (for adding nodes dynamically to an existing cluster), and --node-role broker|lakehouse|all (default all).
  • Security: --auth-mode none|tls|auto-tls, plus --tls-cert, --tls-key, --ca-cert, --jwt-secret, and --super-admins.

Layer 2: Cluster-Wide Shared State in Raft

Section titled “Layer 2: Cluster-Wide Shared State in Raft”

Once the cluster forms Raft quorum, cluster-wide policies are stored under /cluster/config/* (storage, write_buffer, wal, load_manager, bootstrap, and target_voter_count) and replicated across every broker. Changes take effect without SSH-ing into nodes or managing per-host config files.

You can manage Layer 2 either imperatively with danube ctl cluster or declaratively with GitOps manifests (danube ctl apply).

Imperative Management (danube ctl cluster)

Section titled “Imperative Management (danube ctl cluster)”

Because Admin gRPC multiplexes on 6650, danube ctl defaults to DANUBE_ADMIN_ENDPOINT=http://127.0.0.1:6650 (override with --endpoint if connecting remotely or using a dedicated --admin-addr).

Terminal window
# Inspect active storage backend
danube ctl cluster storage status


# Configure S3 object store for tiered cold storage
danube ctl cluster storage object-store \
  --backend s3 \
  --root s3://danube-tiered-prod/segments \
  --option region=us-east-1


# Or configure shared POSIX/NFS filesystem storage
danube ctl cluster storage shared-fs --root /mnt/shared-danube


# Switch back to node-local storage only
danube ctl cluster storage local --force

Declarative GitOps (danube ctl apply)

Section titled “Declarative GitOps (danube ctl apply)”

Check your desired cluster topology, storage settings, namespaces, and topics into Git and apply them idempotently:

desired_state.yaml
namespaces:
  - /agents
  - /telemetry


topics:
  - name: /agents/tasks
    intent: task_queue
    partitions: 8
    storage_mode: cloud_tiered


  - name: /agents/tool-audit
    intent: audit_log
    partitions: 4
    schema_subject: tool-invocation-v1
    storage_mode: cloud_tiered


  - name: /telemetry/tokens
    intent: realtime
    partitions: 4

Top-level manifest sections supported alongside namespaces and topics include shared_fs, object_store, write_buffer, and load_manager.

Always preview changes with --dry-run before reconciling:

Terminal window
# Preview changes without mutating cluster state
danube ctl apply -f desired_state.yaml --dry-run


# Apply changes additively (safe default)
danube ctl apply -f desired_state.yaml


# Reconcile strictly by pruning unmanaged namespaces/topics
danube ctl apply -f desired_state.yaml --prune --force

Layer 3: Topic Creation (Explicit Structure vs. Implicit --intent)

Section titled “Layer 3: Topic Creation (Explicit Structure vs. Implicit --intent)”

At the topic layer, Danube separates structural identity (what the stream is) from operational behavior (how durable, fast, or fault-tolerant it must be):

  1. Specify structural metadata explicitly when creating or updating a topic:

    • Topic name (/namespace/topic or --namespace <ns> <topic>)
    • Partition count (--partitions N, default 0 for non-partitioned)
    • Optional schema contract (--schema-subject <subject>)
    • Optional per-topic storage tier (danube ctl topics storage-mode <topic> --mode local|cloud_tiered)

  2. Delegate durability and poison handling implicitly via --intent (or override dispatch explicitly with --dispatch-strategy non_reliable|reliable):

    • realtime — NonReliable in-memory delivery, zero disk I/O
    • event_stream — Reliable, OS-cache flush WAL, 3 retries (500ms exponential backoff), routes poison messages to <topic>-dlq
    • task_queue — Reliable, fsync_batch WAL, 5 retries (1000ms exponential backoff), routes poison messages to <topic>-dlq
    • audit_log — Reliable, fsync_batch WAL, 10 retries (500ms exponential backoff), blocks on poison messages (no DLQ)
Terminal window
# Explicit structure + implicit intent
danube ctl topics create /orders/events \
  --partitions 6 \
  --schema-subject order-event-v2 \
  --intent event_stream


# Override per-topic tiered storage mode when object storage is configured
danube ctl topics storage-mode /orders/events --mode cloud_tiered

Layer 4: Break-Glass Recovery

Section titled “Layer 4: Break-Glass Recovery”

For disaster recovery, cluster audits, or emergency troubleshooting when Raft quorum cannot yet be reached, export live configuration snapshots and apply node-local overrides with --recovery-config:

Terminal window
# Export live cluster & broker configuration as YAML or JSON
danube ctl cluster export-config --format yaml --output cluster-backup.yaml


# Target a specific broker ID if needed
danube ctl cluster export-config --broker-id 1 --format json --output broker1-backup.json

If an emergency restart requires overriding broker/cluster settings prior to Raft re-convergence, boot the broker with --recovery-config:

Terminal window
danube up \
  --mode cluster \
  --data-dir /var/lib/danube/data \
  --recovery-config ./emergency-recovery.yaml

Next Steps

Section titled “Next Steps”
  • Stream Intents — Exact expansion settings and retry/DLQ semantics.
  • Broker Modes — Comparing standalone, cluster, and zero-YAML edge deployments.
  • Danube for AI Agents — Workload intents, SDK primitives, branching, and the danube mcp control plane.
  • CLI Messaging & Operations — Producing, consuming, seeking, peeking, and replaying DLQs with danube ctl.