Quickstart

Controlled BetaVerification Required

Message Broker stream and queue quickstart.

Run a local Message Broker cluster, create a stream, publish and replay records, claim and acknowledge queue work, inspect the worker result, and keep controlled-beta limits visible.

Current availability

Private Beta. Public beta is scheduled for October 5, 2026.

Result

You will have a local broker workflow that creates docs-023.telemetry, publishes reading-1 and reading-2, replays them from offset 0, claims job-1 from docs-023.jobs, acknowledges the delivery, and inspects payload-free queue state showing the work is complete.

Maturity Status

Stream And Queue Workflow

Controlled Beta

Stream create, publish, replay, queue claim, ack, and payload-free inspection routes are implemented in the controlled-beta broker surface.

Review Boundary

Verification Required

This tutorial does not approve production support, performance, scale, external identity-provider validation, Kafka protocol compatibility, or cross-site replication outcomes.

Prerequisites

Required tools

Required

cargo --version for the Rust toolchain
curl --version for HTTP requests
bash --version for local cluster scripts
sed for extracting the broker-assigned delivery ids

Product setup

Required

Message Broker source at /path/to/Service.MessageBroker or an equivalent local checkout.
Loopback ports 9101, 9102, and 9103 available for the local cluster.
A disposable BROKER_CLUSTER_DIR so tutorial state stays separate from shared local data.

Local assumptions

Required

The local cluster starts node local-1 at http://127.0.0.1:9101.
The local cluster uses insecure-local auth mode, so the tutorial does not require credentials.
The workflow uses tutorial-scoped stream and consumer group names.

Optional helpers

Optional

jq can make local JSON inspection easier, but the copy/paste path only requires shell, curl, and sed.
The service validator Support/Validate-DeveloperWalkthrough.sh exercises a broader route set when product QA needs a scripted check.

See local prerequisites for the shared tool baseline.

Start The Service

Use one terminal for the local cluster and a second terminal for the workflow commands. The cluster script starts three loopback nodes and enables HTTP metrics.

Start an isolated local cluster

Service terminal

cd /path/to/Service.MessageBroker
export BROKER_CLUSTER_DIR="${TMPDIR:-/tmp}/message-broker-docs-023-cluster"
STORNAMICS_LOCAL_CLUSTER_DIR="$BROKER_CLUSTER_DIR" \
  bash Support/Start-LocalCluster.sh

Expected output

Building broker binary...
Started local-1 on 127.0.0.1:9101 ...
Started local-2 on 127.0.0.1:9102 ...
Started local-3 on 127.0.0.1:9103 ...
Local cluster is ready.

Check readiness

Workflow terminal

export BROKER_ENDPOINT=http://127.0.0.1:9101
curl -s "$BROKER_ENDPOINT/health"

Expected output

{"status":"ok","service":"stornamics-message-broker","node_id":"local-1","http_bind":"127.0.0.1:9101","storage":{"status":"ready",...},"security":{"auth_mode":"insecure-local",...}}

Complete The Workflow

Configure tutorial values

Copy/paste

Set one endpoint and tutorial-scoped names for the stream, queue stream, consumer group, and replay consumer.

export BROKER_ENDPOINT=http://127.0.0.1:9101
export BROKER_STREAM=docs-023.telemetry
export BROKER_QUEUE_STREAM=docs-023.jobs
export BROKER_GROUP=docs-023-workers
export BROKER_CONSUMER=docs-023-reader

Expected output

These variables are shell-local. The streams and consumer group names are scoped to DOCS-023.

Create a stream

Copy/paste

Create the stream used for the publish and replay workflow through the versioned management route.

curl -s -X PUT \
  -H 'Content-Type: application/json' \
  --data '{}' \
  "$BROKER_ENDPOINT/v1/streams/$BROKER_STREAM"

Expected output

{"api_version":"v1","stream":"docs-023.telemetry","created":true,...}

Publish two records

Copy/paste

Append two local records. The broker returns offsets and durable acknowledgement fields for each publish.

curl -s -X POST \
  -H 'Content-Type: application/json' \
  --data '{"message_id":"reading-1","key":"machine-17","event_time_ms":1700000000000,"payload":"hello","durability_mode":"local-append"}' \
  "$BROKER_ENDPOINT/v1/streams/$BROKER_STREAM/records"

curl -s -X POST \
  -H 'Content-Type: application/json' \
  --data '{"message_id":"reading-2","key":"machine-17","event_time_ms":1700000000001,"payload":"world","durability_mode":"local-append"}' \
  "$BROKER_ENDPOINT/v1/streams/$BROKER_STREAM/records"

Expected output

{"api_version":"v1","stream":"docs-023.telemetry","offset":0,"next_offset":1,"message_id":"reading-1","payload_bytes":5,...}
{"api_version":"v1","stream":"docs-023.telemetry","offset":1,"next_offset":2,"message_id":"reading-2","payload_bytes":5,...}

Replay records

Copy/paste

Fetch from offset 0 and include replay consumer metadata so the inspection route has activity to report.

curl -s \
  "$BROKER_ENDPOINT/v1/streams/$BROKER_STREAM/records?start_offset=0&max_records=10&consumer_group=$BROKER_GROUP&consumer_id=$BROKER_CONSUMER"

Expected output

{
  "api_version": "v1",
  "stream": "docs-023.telemetry",
  "records": [
    {"offset":0,"message_id":"reading-1","payload_hex":"68656c6c6f"},
    {"offset":1,"message_id":"reading-2","payload_hex":"776f726c64"}
  ],
  "next_offset": 2,
  "end_of_stream": true,
  "diagnostics": {"returned_records":2}
}

Inspect replay activity

Copy/paste

Use the payload-free replay consumer inspection route to confirm the replay was observed without returning message payloads.

curl -s \
  "$BROKER_ENDPOINT/v1/streams/$BROKER_STREAM/consumer-groups/$BROKER_GROUP/replay-consumers"

Expected output

{"api_version":"v1","stream":"docs-023.telemetry","consumer_group":"docs-023-workers","consumers":[{"consumer_id":"docs-023-reader","returned_records":2,"end_of_stream":true,...}]}

Create a queue stream and publish work

Copy/paste

Use a separate stream for queue work so the worker result is easy to inspect.

curl -s -X PUT \
  -H 'Content-Type: application/json' \
  --data '{}' \
  "$BROKER_ENDPOINT/v1/streams/$BROKER_QUEUE_STREAM"

curl -s -X POST \
  -H 'Content-Type: application/json' \
  --data '{"message_id":"job-1","key":"worker","event_time_ms":1700000000100,"payload":"work","durability_mode":"local-append"}' \
  "$BROKER_ENDPOINT/v1/streams/$BROKER_QUEUE_STREAM/records"

Expected output

{"api_version":"v1","stream":"docs-023.jobs","created":true,...}
{"api_version":"v1","stream":"docs-023.jobs","offset":0,"next_offset":1,"message_id":"job-1",...}

Claim queue work

Copy/paste

Claim one available record for the worker group and keep the response so the ack step can reuse the broker-assigned ids.

CLAIM_RESPONSE="$(curl -s -X POST \
  -H 'Content-Type: application/json' \
  --data '{"max_messages":1,"lease_timeout_millis":30000}' \
  "$BROKER_ENDPOINT/v1/streams/$BROKER_QUEUE_STREAM/consumer-groups/$BROKER_GROUP/claims")"

printf '%s\n' "$CLAIM_RESPONSE"

Expected output

{"api_version":"v1","stream":"docs-023.jobs","consumer_group":"docs-023-workers","subscription_id":"sub-...","deliveries":[{"delivery_id":"del-...","attempt":1,"record":{"message_id":"job-1","payload_hex":"776f726b"}}],"delivery_count":1}

Acknowledge the delivery

Copy/paste

Extract the subscription and delivery ids from the claim response, then ack the active delivery.

SUBSCRIPTION_ID="$(printf '%s' "$CLAIM_RESPONSE" | sed -nE 's/.*"subscription_id":"([^"]+)".*/\1/p')"
DELIVERY_ID="$(printf '%s' "$CLAIM_RESPONSE" | sed -nE 's/.*"delivery_id":"([^"]+)".*/\1/p')"

curl -s -X POST \
  -H 'Content-Type: application/json' \
  --data '{}' \
  "$BROKER_ENDPOINT/v1/streams/$BROKER_QUEUE_STREAM/subscriptions/$SUBSCRIPTION_ID/deliveries/$DELIVERY_ID/ack"

Expected output

{"api_version":"v1","stream":"docs-023.jobs","subscription_id":"sub-...","delivery_id":"del-...","offset":0,"attempt":1,"acknowledgement":{"durability":{"outcome":"durable",...}}}

Confirm no work remains

Copy/paste

Claim again with the same worker group to prove the acknowledged delivery is no longer available.

curl -s -X POST \
  -H 'Content-Type: application/json' \
  --data '{"max_messages":1,"lease_timeout_millis":30000}' \
  "$BROKER_ENDPOINT/v1/streams/$BROKER_QUEUE_STREAM/consumer-groups/$BROKER_GROUP/claims"

Expected output

{"api_version":"v1","stream":"docs-023.jobs","consumer_group":"docs-023-workers","deliveries":[],"delivery_count":0}

Inspect queue state

Copy/paste

Inspect the consumer group and completed leases. These routes are payload-free and show the worker result without returning the job body.

curl -s \
  "$BROKER_ENDPOINT/v1/streams/$BROKER_QUEUE_STREAM/consumer-groups/$BROKER_GROUP"

curl -s \
  "$BROKER_ENDPOINT/v1/streams/$BROKER_QUEUE_STREAM/consumer-groups/$BROKER_GROUP/leases?include_completed=true"

Expected output

{"api_version":"v1","stream":"docs-023.jobs","consumer_group":"docs-023-workers","description":{"mode":"shared_queue","highest_next_offset":1,"progress_offset":1,"lag_messages":0,"in_flight_count":0,...}}
{"api_version":"v1","stream":"docs-023.jobs","consumer_group":"docs-023-workers","leases":[{"offset":0,"attempt":1,"state":"acked",...}]}

Check controlled-beta replication boundary

Copy/paste

Replication status is visible, but a default local cluster does not configure workers or prove cross-site outcomes.

curl -s "$BROKER_ENDPOINT/v1/replication/status"

Expected output

{"api_version":"v1","status":"not_configured","node_id":"local-1","runtime_boundary":"read-only status binding; no replication workers or network transports are started by this route",...}

Verify Success

The quickstart is successful when replay returns both message ids and next_offset:2, the queue claim returns delivery_count:1, the ack response includes a durable acknowledgement, the second claim returns delivery_count:0, and lease inspection shows the first delivery in acked state.

Clean Up

Stop the local cluster

Copy/paste

Stop only the tutorial cluster by sending the same runtime directory to the stop script.

cd /path/to/Service.MessageBroker
export BROKER_CLUSTER_DIR="${TMPDIR:-/tmp}/message-broker-docs-023-cluster"
STORNAMICS_LOCAL_CLUSTER_DIR="$BROKER_CLUSTER_DIR" \
  bash Support/Stop-LocalCluster.sh

Expected output

Stopped local-1 ...
Stopped local-2 ...
Stopped local-3 ...

Remove tutorial data

Copy/paste

Remove the disposable local cluster directory after the brokers have stopped.

rm -rf "$BROKER_CLUSTER_DIR"

Expected output

The DOCS-023 local cluster directory is gone. Do not remove shared broker data directories.

Troubleshooting

Common first-run failures
Symptom	How to recognize it	Likely cause	Fix
Local cluster says a node is already running	`Support/Start-LocalCluster.sh` reports `Run ./Support/Stop-LocalCluster.sh first`.	A previous cluster is still running, or the same `BROKER_CLUSTER_DIR` contains live PID files.	Run the cleanup stop command with the same `BROKER_CLUSTER_DIR`, then start the cluster again.
Health check cannot connect	`curl` reports connection refused or times out against `127.0.0.1:9101`.	The cluster is still building, a node failed startup validation, or another process owns port `9101`.	Check `Support/Check-LocalCluster.sh` and the node log under `$BROKER_CLUSTER_DIR/logs/node-1.log`.
Create stream returns `stream_already_exists`	The response status is `409 Conflict` or the JSON error code is `stream_already_exists`.	The stream already exists in the tutorial cluster state from an earlier run.	Reuse the existing stream for the rest of the tutorial or set a new tutorial-scoped stream name.
Publish returns `stream_not_found` or `missing_stream`	The response body contains `stream_not_found` or `missing_stream`.	The publish step ran before the stream create step or used a different stream variable.	Create the stream again and confirm `BROKER_STREAM` or `BROKER_QUEUE_STREAM` matches the publish URL.
Ack returns `delivery_not_found`	The response body contains `delivery_not_found`.	The extracted `SUBSCRIPTION_ID` or `DELIVERY_ID` is empty, stale, or from another stream.	Re-run the claim step, confirm `delivery_count` is `1`, and reuse the ids from that response.
Ack returns `lease_expired`	The response body contains `lease_expired`.	The 30-second delivery lease expired before the ack request reached the broker.	Claim the message again and ack the fresh delivery id, or use a longer `lease_timeout_millis` while debugging.
Replay inspection shows no consumers	The `replay-consumers` response has an empty `consumers` array.	The replay request omitted `consumer_group` or `consumer_id`, or used a different group name.	Replay again with the documented query string and confirm `BROKER_GROUP` and `BROKER_CONSUMER` are set.

Evidence

Source-backed quickstart checks
Workflow step	Evidence	Status
Local cluster startup and health	Service.MessageBroker/Support/Start-LocalCluster.sh, Support/Check-LocalCluster.sh, and README.md local cluster instructions	Source-backed; local run still needs reviewer acceptance
Stream create, publish, and replay	Service.MessageBroker/docs/native-api-contract.md, docs/http-management-api-scope.md, and Support/Validate-DeveloperWalkthrough.sh	Source-backed
Replay consumer inspection	Service.MessageBroker/docs/http-management-api-scope.md and runtime inspection route tests	Source-backed
Queue claim and ack	Service.MessageBroker/docs/runtime-shared-queue-http-routes.md, docs/durable-queue-state-integration.md, and Support/Validate-DeveloperWalkthrough.sh	Source-backed
Controlled-beta limits	Service.MessageBroker/docs/controlled-beta-onboarding-package.md and docs/fact-sheets/message-broker.md	Source-backed; claims remain review-bound

Limits

Controlled-Beta Scope

This quickstart proves a local controlled-beta stream, replay, and shared-queue worker path; it is not a general release support promise.

Local Cluster

The commands use a three-node local cluster in insecure-local auth mode and a disposable runtime directory.

Payload Visibility

Replay and claim responses include payload bytes as hex, while consumer-group, lease, and replay-consumer inspection routes are payload-free.

Replication Boundary

Replication status is inspectable, but cross-site durability, availability, failover, and topology outcomes require separate evidence.

Next Steps

Available now
Review Message Broker evidence
Use the evidence index before strengthening stream, queue, auth, replication, or pilot-readiness claims.
Available now
Read tutorial prerequisites
Check the shared local tool baseline and product-specific setup expectations for first-success tutorials.
Available: DOCS-031
Read HTTP reference
Look up stream, record, cursor, queue, audit, metrics, storage, auth, and replication route details.
Available: DOCS-040
Compare route and auth scope
Separate controlled-beta routes from auth, replication, queue, CLI, SDK, and Kafka protocol boundaries.
Available: DOCS-059
Use SDK examples
Review tested Rust SDK publish, replay, cursor, queue worker, retry, and dead-letter flows.