chmonitorBlog
StarDashboard
All posts

ClickHouse monitoring: the dashboards every cluster actually needs

What to monitor in ClickHouse, which system tables power each view, and how chmonitor turns them into a real-time dashboard — from query latency and merges to replication lag and disk pressure.

MonitoringJuly 10, 2026

ClickHouse exposes almost everything you need to operate it through its own system tables — but nobody wants to hand-write that SQL at 3am. This is the ClickHouse monitoring map: the five areas that matter, the system tables behind them, and how chmonitor wires them into a dashboard your whole team can read.

What ClickHouse monitoring covers

Production monitoring isn’t one graph; it’s a set of views that answer specific questions: are queries slow, are merges piling up, is replication behind, is disk filling, is memory safe. Each view below maps to a system table chmonitor queries on a fixed interval.

Steps

1. Query performance

The heart of ClickHouse monitoring is system.query_log (and system.query_thread_log for per-thread detail). Watch p95/p99 latency, failed-query rate, and slowest queries:

SELECT
  quantiles(0.50, 0.95, 0.99)(query_duration_ms) AS p50_95_99,
  countIf(type = 'ExceptionWhileProcessing')      AS errors,
  count()                                          AS queries
FROM system.query_log
WHERE event_time > now() - INTERVAL 1 HOUR

chmonitor’s Running Queries view shows live queries; the slowest-queries post shows how to rank them.

2. Merges and mutations

system.merges and system.mutations tell you whether background work is keeping up. A growing queue means inserts are outrunning merges — a classic merge storm:

SELECT
  table,
  count() AS active_merges,
  countIf(is_mutation) AS active_mutations
FROM system.merges
GROUP BY table

3. Replication and lag

On replicated tables, system.replication_queue and system.replicas expose lag and queue depth. chmonitor surfaces replication lag per table so a stuck replica doesn’t silently drift:

SELECT
  database,
  table,
  queue_size,
  absolute_delay
FROM system.replicas
WHERE is_readonly OR absolute_delay > 10

See the replication lag deep-dive for triage.

4. Disk and parts pressure

Disk fills fast when parts aren’t merging or TTLs aren’t firing. system.disks and system.parts give capacity and part counts:

SELECT
  name AS disk,
  formatReadableSize(free_space)     AS free,
  formatReadableSize(total_space)    AS total
FROM system.disks

If free space drops under a threshold, chmonitor flags it before writes start failing — the same pressure covered in disk-full emergency.

5. Memory and errors

system.events and system.errors catch OOM and spikes. The memory limit exceeded and system errors spike posts walk the diagnostic queries.

Why a dashboard, not SQL

Each query above is easy alone and annoying at scale. chmonitor runs them on an interval, keeps history, and renders them as charts — so you see a latency trend, not a single number, and you catch the drift before it becomes an incident. For Kubernetes, see our ClickHouse monitoring on Kubernetes guide.