Sources

The Sources view combines authored source definitions with records retained in ClickHouse and read-only Kafka metadata. Browsing source health never consumes Kafka records or advances consumer offsets.

Source health

The Sources index summarizes every authored source. Open a managed Kafka source to inspect:

• retained row count and storage
• oldest and newest replay boundaries
• arrival freshness and throughput
• the effective consumer group
• landed, committed, and broker-end offsets by partition

Kafka lag and last arrival are different signals:

• Kafka lag is the broker high-water offset minus the effective consumer group’s committed offset, summed across partitions.
• Last arrival is the elapsed time since the newest _replay_landed_at value in the ClickHouse landing table.

Kafka metadata is loaded without joining or consuming from the group. If broker metadata or a required committed offset is unavailable, the UI reports Kafka lag as unavailable; it never substitutes last-arrival age. Broker metadata is cached and refreshed outside the main state request, so a first load may briefly show unavailable.

See Topics for broker inventory, Messages for retained payloads, and Replay and Lineage for normalized replay columns and boundary semantics.