blockwatcher-metrics

blockwatcher-metrics is blockwatcher’s Prometheus wiring: it installs the process-global recorder that every metrics facade call anywhere in the workspace writes into, and it serves the /metrics scrape endpoint that renders whatever that recorder has accumulated since process start. It is a module crate in the sense the workspace map uses the word: not because it varies chain behavior, but because it is the one crate in the workspace allowed to carry an HTTP stack for this purpose, so that nothing in the core ring has to.

Its production dependencies are exactly the packages scripts/check-dep-graph.sh’s ALLOW_BLOCKWATCHER_METRICS entry lists, and no workspace crate at all (crates/blockwatcher-metrics/Cargo.toml; confirmed independently by the dependency graph, where blockwatcher-metrics is one of the nodes in the whole table with no outgoing edge):

• axum: the HTTP framework serve’s /metrics router runs on
• metrics-exporter-prometheus: PrometheusBuilder/PrometheusHandle and the Prometheus recorder itself
• thiserror: the derive behind MetricsError
• tokio (net/time/sync/macros/rt features): the listener, the upkeep interval, and the shutdown-signal plumbing
• tracing: the crate’s own warning-level logging

That absence of any workspace crate is the point: blockwatcher-core’s own metrics.rs module calls the bare metrics facade crate directly (a header-only dependency with no network stack of its own) and never touches axum or metrics-exporter-prometheus. Only this crate does, which is exactly what scripts/check-dep-graph.sh checks: its ALLOW_BLOCKWATCHER_CORE entry names metrics but not axum or metrics-exporter-prometheus, and carries no family exemption at all, so neither could appear anywhere in blockwatcher-core’s transitive tree either; its ALLOW_BLOCKWATCHER_METRICS entry names both, with FAMILY_EXEMPT_BLOCKWATCHER_METRICS="axum hyper tower-http" covering the transport crates axum itself pulls in. Splitting the exporter from every emitter is what lets a pipeline task keep calling metrics::counter! for free while the HTTP server, its router, and its accept loop live in a crate the core ring never has to compile. See Workspace map § Verifying the rings and Architecture decisions § Chain knowledge stays out of the core for the general rule this crate is one instance of.

This page covers how the exporter itself works (installing the recorder, serving the scrape route, running upkeep) and how the two per-pipeline observability destinations blockwatcher-core maintains relate to it. Observability already documents every exported metric’s name, type, and label from an operator’s point of view; this page does not repeat that table.

Key takeaways

• blockwatcher-metrics installs the process-global Prometheus recorder every metrics facade call writes into, and serves the /metrics scrape endpoint that renders it.
• It is the one crate in the workspace allowed to carry an HTTP stack for this purpose, so nothing in the core ring has to.
• It depends on no workspace crate at all; blockwatcher-core’s own metrics.rs calls the bare metrics facade directly and never touches axum or the Prometheus exporter.

Responsibilities

• Installs the process-global Prometheus recorder exactly once per process, via PrometheusBuilder::new().install_recorder(), and hands back the same PrometheusHandle on every later call in the same process rather than erroring or reinstalling: the metrics crate has no uninstall, so cumulative series continue across a repeated call instead of resetting (install_recorder, crates/blockwatcher-metrics/src/lib.rs).
• Serves GET /metrics on an already-bound TcpListener as Prometheus exposition text (text/plain; version=0.0.4; charset=utf-8). Alongside the route itself, serve spawns a second task that calls handle.run_upkeep() every 5 seconds and keeps calling it for the whole time the listener is up. See Observability § Turning it on for what an operator gets out of that (serve, router, lib.rs).
• Stops accepting new connections the moment shutdown resolves but lets an in-flight scrape finish, via axum::serve(..).with_graceful_shutdown(..) (lib.rs).
• Owns axum so nothing else in the workspace has to. The doc comment at the top of lib.rs makes the point directly: owning axum there, rather than in the binary, is what keeps the HTTP framework out of the binary’s own dependency graph (lib.rs).

Not this crate’s job: deciding what gets emitted, under what name, or with what labels: every metric constant and its call site lives in the crate that owns the event (blockwatcher-core’s metrics.rs for pipeline events, blockwatcher-rpc’s connection pool for its own, per Observability § Other metrics on this endpoint); binding the listener or reading [metrics].listen: the blockwatcher binary does both and hands this crate an already-bound socket, in boot and serve_metrics (crates/blockwatcher/src/run.rs); the control-plane HTTP API, a wholly separate axum::Router on its own socket (blockwatcher-api); mapping a pipeline counter onto anything (see How data flows through it below; this crate is downstream of that mapping, never a party to it).

Key types and traits

How data flows through it

This crate sits downstream of every emitter, never upstream: nothing inside it calls metrics::counter! or metrics::gauge! on its own behalf. install_recorder only makes the process-global recorder exist; serve only renders whatever that recorder has already accumulated by the time a scrape arrives.

flowchart LR
    core["blockwatcher-core<br/>metrics.rs<br/>pipeline/*, engine/drain.rs, engine/invalidate.rs"] -->|"metrics::counter!/gauge!"| rec[("process-global<br/>metrics::Recorder")]
    rpc["blockwatcher-rpc<br/>connection pool"] -->|"metrics::counter!"| rec
    install["install_recorder()<br/>lib.rs"] ==>|"installs once,<br/>returns PrometheusHandle"| rec
    upkeep["upkeep task<br/>every 5s while serve() runs<br/>lib.rs"] -->|"handle.run_upkeep()"| rec
    rec -.->|"handle.render()"| scrape{{"GET /metrics<br/>lib.rs"}}

From a pipeline event to a scrape line

blockwatcher-core maintains two independent, same-call-site destinations for every pipeline event, and only one of them ever reaches this crate:

Both blockwatcher-core destinations are incremented from the same call site: see, for one example, Processor::process_one’s decode loop, which bumps self.counters.decoded and calls metrics::count(metrics::EVENTS_DECODED, ..) on adjacent lines. Observability § The same events, twice makes the case for why an operator wants both; this page’s narrower point is structural: this crate is the renderer for exactly one of the two rows in that table, and has no view onto the other row’s contents at all: it cannot, since nothing about PipelineCounters ever reaches the metrics facade this crate’s recorder is listening to.

Neighbours

blockwatcher-metrics depends on, in production (no workspace crate among them: crates/blockwatcher-metrics/Cargo.toml; confirmed by the dependency graph):

• axum
• metrics-exporter-prometheus
• thiserror
• tokio (net/time/sync/macros/rt features)
• tracing

and, in [dev-dependencies] only:

• metrics: records a real counter in its own black-box test
• reqwest: scrapes the served endpoint over real HTTP in that same test

The following crate depends on it directly:

• blockwatcher (binary): calls install_recorder before seeding and passes the resulting PrometheusHandle to serve_metrics, a thin wrapper spawning blockwatcher_metrics::serve, once [metrics].enabled names a listen address, in boot and serve_metrics (crates/blockwatcher/src/run.rs)

Reading the source

1. Start at lib.rs’s module doc comment: the one-sentence reason this crate owns axum instead of the binary.
2. METRICS_HANDLE, MetricsError, and install_recorder: the process-global install-once, reuse-forever contract.
3. router and serve: the /metrics route, the upkeep task, and the graceful-shutdown wiring that lets an in-flight scrape finish before the listener actually stops.
4. tests/serve.rs: the crate’s one black-box test, and also its shortest correct usage example: install, record one counter, serve, scrape over real HTTP, shut down, and confirm the port is actually released.