blockwatcher-api
blockwatcher-api is blockwatcher’s REST control plane: it turns an HTTP request into
exactly one call against blockwatcher-core’s ControlHandle, or against core’s
typed storage facade for a read, and turns whatever comes back into a JSON
response. Every mutation goes through ControlHandle, the same path a
--seed load and a running engine’s own boot use, so a resource never sees
two independent validate/persist code paths; every read goes through core’s
Resources facade, taking no engine lock, so a status or a GET stays
answerable while a mutation is in flight (lib.rs).
Its production dependencies are exactly the names
scripts/check-dep-graph.sh’s ALLOW_BLOCKWATCHER_API entry lists
(crates/blockwatcher-api/Cargo.toml):
blockwatcher-types: every resource shape a route (de)serializes, and the ids used to look one upblockwatcher-ports:Storage(the traitApiState.storageis adynpointer to) andStorageError(matched insideerror.rs’s classification)blockwatcher-core:ControlHandle,EngineError, and every other engine type a route names, listed in full in Key types and traits belowaxum: the HTTP framework the whole route table andserveare built onserde: the derive machinery behind every request and response bodyserde_json: parsing a request body and rendering a response onetokio(netfeature):TcpListener, the typeserveaccepts already boundtracing: the crate’s own structured logging (an auth rejection, a 500’s real detail)
blockwatcher-api is one of the crates in the core ring,
and the only one permitted to carry an HTTP stack at all:
Workspace map § Three rings names the single
exemption scripts/check-dep-graph.sh grants it, for exactly the axum
family (axum, and what it pulls in) and no other forbidden family, so this
crate can serve HTTP without ever being able to see a chain SDK or a storage
driver either. See Architecture decisions § Chain knowledge stays out of the
core for
the rule this crate is the one core-ring exception to.
The HTTP API reference already documents
every route, request and response body, and status code from an operator’s
point of view. This page does not restate that table; it covers how the six
route files split the work, how the auth middleware and the error mapping
are built, and what keeps this crate itself chain-agnostic despite serving
resources (a network’s
source.config, a spec’s
payload) that are opaque,
module-specific JSON on the wire.
Key takeaways
- Every mutation goes through
ControlHandle, the same path a--seedload and boot use; every read goes through core’sResourcesfacade without taking an engine lock. blockwatcher-apiis the one core-ring crate allowed to carry an HTTP stack, exempted only for theaxumfamily.- A network’s
source.configand a spec’spayloadcross this crate as opaque JSON; it never looks inside either one. - This page covers route organisation, auth, and error mapping; the HTTP API reference already documents every route from an operator’s point of view.
Responsibilities
- Builds the complete route table and wraps it in one authentication
middleware layer,
router(lib.rs). - Enforces
Authorization: Bearer <token>on every route but/health, matching the presented secret against the labelled[auth]table (auth/mod.rs) and insertingIdentity { label, scope }. See Authentication middleware. - Maps every failure, engine-raised or decided by this layer on its own,
onto one wire error shape and the right HTTP status (
error.rs). See Error mapping. - Generates every resource kind’s whole CRUD surface from one macro, so
PUT/GET/DELETEagainst a network, a spec, a sink, or a monitor are the same code path repeated rather than each hand-kept separately (routes/resources.rs). - Serves the route table on an already-bound listener until a shutdown
future resolves, then lets in-flight requests finish (
serve.rs), which is what keepsaxumout of theblockwatcherbinary’s own dependency graph (serve.rs).
Not this crate’s job: deciding whether a write is valid, compiling a
predicate, or persisting anything: every one of those already happened
inside ControlHandle by the time a handler’s await returns
(blockwatcher-core); running a pipeline, a source, a decoder, a matcher, or a
sink (also blockwatcher-core, dispatching into the module crates behind its
port traits); knowing what any chain’s wire format means (no chain SDK
appears anywhere in this crate’s dependency tree at all, per the exemption
described above); exposing anything over Prometheus (blockwatcher-metrics owns
a wholly separate axum::Router on its own socket); reading [api].listen
or resolving [auth] from instance configuration (the blockwatcher
binary does both and hands this crate an already-bound listener and an
already-built token table, in boot, crates/blockwatcher/src/run.rs).
Key types and traits
| Name | Kind | Role |
|---|---|---|
ApiState | struct | Everything a handler needs: control (ControlHandle, the only mutation path), storage (Arc<dyn Storage>, what reads are served from), and auth (Arc<Vec<ApiToken>>, labelled credentials, never resolved secret values) (lib.rs) |
router | fn | Builds the merged route table from all six route files and wraps the whole thing in the auth middleware (lib.rs) |
serve | async fn | Runs router(state) on an already-bound TcpListener until shutdown resolves, via axum::serve(..).with_graceful_shutdown(..) (serve.rs) |
require_bearer | async fn (middleware) | The one authentication check every route but /health passes through (auth/mod.rs). Inserts Identity { label, scope }. |
require! | macro | Per-MethodRouter authorization: .route_layer(require!(Scope::Admin)) (auth/mod.rs) |
ApiError | enum | Engine(EngineError) or Api { status, code, message }: every way a request can fail, collapsed to one type so every failure leaves through one body shape (error.rs) |
ErrorBody, ErrorDetail | struct | The wire error shape: {"error": {"code", "message", "actual_version"?}} (error.rs) |
resource_routes! | macro | Generates one resource kind’s list/read/write/remove handlers and the routes they answer on, invoked once per kind (routes/resources.rs) |
Route organisation
routes/ holds one file per concern, and routes::mod’s own module list is
the complete inventory (routes/mod.rs). Each file answers a distinct
question, which is also why a new resource kind or a new cross-cutting
concern gets a new file rather than a growing one:
| File | Answers | Routes |
|---|---|---|
health.rs | Is this process up at all? | GET /health, exempt from auth |
status.rs | What is every pipeline doing right now? | GET /status |
schema.rs | What vocabulary can a predicate against this spec address? | GET /specs/{id}/schema |
resources.rs | Create, read, update, delete, list, for any resource kind | GET/PUT/DELETE on /networks, /specs, /sinks, /monitors (collection and {id}) |
monitors_ops.rs | What can an operator do to one monitor beyond CRUD? | POST /monitors/{id}/pause, /resume, /test |
networks_ops.rs | What can an operator do to one network beyond CRUD? | POST /networks/{id}/pause, /resume, /skip; DELETE /networks/{id}/checkpoint; GET/DELETE on /networks/{id}/dead-letters (collection) and POST .../replay/DELETE on /networks/{id}/dead-letters/{match_id} |
The split mirrors the same distinction blockwatcher-core’s own control/ module
draws between generic CRUD (writes.rs/deletes.rs, one write and one
delete method per kind) and kind-specific extra verbs (skip.rs,
dead_letters.rs): resources.rs is one macro invoked four times because
every kind’s CRUD answers identically, while monitors_ops.rs and
networks_ops.rs exist precisely because a monitor’s extra operations
(pause, resume, a dry-run test) and a network’s (pause, resume, skip,
checkpoint reset,
dead-letter replay) are not the same set and must not be
forced into one shared shape just because they are both “operations on a
resource.” health.rs, status.rs, and schema.rs each get their own file
for the same reason blockwatcher-core’s progress.rs sits apart from
pipeline/: none of the three is CRUD or an operation on a stored resource
at all, and folding a liveness probe or a read-only snapshot into a file
named for something else would make that file’s own scope harder to state.
Every route file exports one router() -> axum::Router<ApiState>, and
lib.rs::router does nothing but .merge all six together and layer the
auth middleware on top (lib.rs):
flowchart LR
req(("incoming<br/>request")) --> mw{"require_bearer<br/>auth/mod.rs"}
mw -->|"path is '/health'"| health["health::router()<br/>routes/health.rs"]
mw -->|"any other path,<br/>valid bearer"| merged{{"merged route table<br/>lib.rs"}}
mw -->|"missing or wrong<br/>bearer token"| err401["401 unauthorized<br/>WWW-Authenticate: Bearer"]
merged --> status["status::router()"]
merged --> schema["schema::router()"]
merged --> resources["resources::router()<br/>(every kind via one macro)"]
merged --> monops["monitors_ops::router()"]
merged --> netops["networks_ops::router()"]
resources.rs additionally carries the shared helpers every one of its four
generated kinds calls through: if_match/required_if_match (parsing and
requiring a strong numeric ETag out of If-Match), written (201 for a
create versus 200 for an update), found (the bare resource plus its
version in an ETag), parse (a body that fails to deserialize becomes an
EngineError::InvalidResource naming the offending key), and same_id (a
path id and a body id that disagree is refused rather than silently
resolved one way or the other) (resources.rs).
Authentication middleware
require_bearer (auth/mod.rs) is axum::middleware::from_fn_with_state,
layered once around the entire merged router rather than mounted only on
the routes that need it (lib.rs). The /health exemption is a path
check inside the middleware, ahead of the router, not a separate mount
alongside it: a request to a path that does not exist at all is refused for
lacking a token before the router gets a chance to reveal, via a 404
versus something else, whether the path would have matched anyway
(auth/mod.rs).
For any other path, the middleware:
- Resolves
state.auth(aSecretRef) to the expected token, fresh, on this request. A resolution failure here is unreachable once actually serving traffic, since the composition root resolves the same reference at boot and refuses to start without it (crates/blockwatcher/src/config.rs’sInstanceConfig::auth); the defensive path that remains answers500rather than panicking (auth/mod.rs). - Reads
Authorization, and accepts only thebearerscheme, case-insensitively, per RFC 7235 (bearer_credential,auth/mod.rs). - Compares the presented token against the expected one in constant time
on length-equal inputs (
constant_time_eq,auth/mod.rs): the only thing an unequal-length comparison leaks is that the lengths differ, never which byte. - On a match, calls
next.run(request); on anything else, answersunauthorized(auth/mod.rs): a401withWWW-Authenticate: Bearer, and atracing::warn!naming the method and path but never the presented or expected token in any form, because a rejected credential is still a secret and one that merely arrived at the wrong deployment is often the right token somewhere else (auth/mod.rs).
The token itself is never held between requests: ApiState.auth carries
only the SecretRef (the env:NAME reference), and step 1 above resolves
it anew on every single request that isn’t /health (lib.rs).
Error mapping: from EngineError to HTTP status
ApiError (error.rs) is the one type every handler’s Result
resolves its error side to, whether the failure came from the engine
(ApiError::Engine, via From<EngineError>) or was decided by this layer on
its own (ApiError::Api { status, code, message }, used for a malformed
If-Match, a missing one on a delete, or a genuinely internal condition).
classify (error.rs) is the exhaustive match from every
EngineError variant onto its status and wire code, with no catch-all arm:
a new variant added to blockwatcher-core fails this crate’s build until someone
gives it a status here, rather than silently inheriting whatever a
neighbouring arm happens to answer.
EngineError variant | Status | Code |
|---|---|---|
UnknownModule, InvalidResource, UnsupportedChain, Compile, ModuleInit, MissingReference, StillReferenced | 422 | unknown_module / invalid_resource / unsupported_chain / compile_failed / module_init_failed / missing_reference / still_referenced |
Storage(VersionConflict) | 412 | version_conflict |
Storage(NotFound) | 404 | not_found |
Storage(AlreadyExists) | 409 | already_exists |
ShuttingDown | 503 | shutting_down |
Unavailable | 503 | unavailable |
NoRunningPipeline | 404 | no_running_pipeline |
ReplayFailed | 502 | replay_failed |
Conflict | 409 | conflict |
CheckpointProvenance | 409 | checkpoint_provenance |
Storage(Backend), Storage(InvalidConfig), DuplicateModule, DuplicateChainDecoder, InvalidEngineConfig | 500 | internal |
The first group is every refusal a caller caused and can fix by editing the
request; the last group is a broken invariant of the process itself (the
duplicate-registration pair cannot even arise from a request, since a boot
that reached the point of serving already rejected them), given explicit
arms rather than folded into a catch-all so a future variant cannot land
there by default (error.rs, comment).
One case gets special handling ahead of classify: ApiError::stored
(error.rs), which every handler that reads or writes storage calls
instead of the plain From conversion. It rewrites InvalidResource
specifically to a 500, because arriving there from a stored record (as
opposed to a caller-supplied body, which handlers reject earlier through
parse and its own From) can only mean a record already in storage no
longer deserializes: an older binary’s row, a hand edit, a newer schema. That
is the deployment’s problem, not the request’s, so it goes to the log at
error level and the wire gets the generic 500 body instead of a 422
that would blame the caller for state they never touched and cannot fix.
ApiError::from(EngineError) (the plain, un-rewritten path) is reserved for
the handlers where an InvalidResource genuinely is the caller’s own doing
(the skip handler maps its rewind refusals through plain
ApiError::from; the replay and monitor-test handlers do the same through
their own small mappers, replay_error in routes/networks_ops.rs and
test_error in routes/monitors_ops.rs), so the choice of mapper at each
call site encodes which side of that distinction a given handler is on.
body() (error.rs) renders the fixed {"code":"internal","message": "internal error"} for any 500, regardless of which branch produced it,
and into_response (error.rs) is the one place the real detail
(a storage backend’s message, a connection string, a stored record’s
identity) reaches tracing::error! before the response goes out: the log
gets everything, the wire gets nothing past the fixed string. Every other
status keeps its variant’s own Display text as message, and
actual_version (error.rs) is populated only for
Storage(VersionConflict), carrying the version storage actually found so a
caller’s retry can send the If-Match it should have sent the first time.
Staying chain-agnostic
blockwatcher-api sits in the core ring specifically because it is not permitted
to know what any chain’s wire format means, and nothing about how it is
built requires it to: every handler speaks in blockwatcher-types resource shapes
(Network, Spec, SinkDef, Monitor) and calls exactly two engine
surfaces, ControlHandle for a mutation and Resources for a read, neither
of which exposes anything chain-specific in its own signature. A few
concrete details make this a property of the code, not just a stated
intention:
- A network’s
source.configand a spec’spayloadcross this crate as opaqueserde_json::Valuefields inside their resource shape (blockwatcher_types::Network/Spec): this crate never looks inside either one. Whethersource.configparses as a validevm-rpcpool, orpayloadas a valid Solidity ABI array, is decided by the named module’s own code, reached throughControlHandle::put_network/put_spec, never by anything inroutes/. GET /specs/{id}/schema(routes/schema.rs) is the one route that looks like it might hand back chain detail, and deliberately doesn’t: it callsControlHandle::spec_schema, which compiles the spec through its chain’s own decoder intoblockwatcher_types::SchemaSet, a chain-agnostic field/type-name vocabulary. It serves the decoder’s translation of the artifact, never the artifact itself, which is exactly what keeps this route’s response shape identical whether the spec behind it is an EVM ABI or something no chain family has been written for yet, per theschemahandler’s own doc comment (routes/schema.rs).- Nothing in this crate’s dependency tree can reach a chain SDK at all: the
exemption
scripts/check-dep-graph.shgrants it covers theaxumfamily only, so a chain SDK pulled in anywhere in this crate’s tree would fail the same transitive check that already forbids it forblockwatcher-coreitself (see the dependency list at the top of this page, and Workspace map § Three rings).
Neighbours
blockwatcher-api depends on, in production (the same eight-entry list named in
full above):
blockwatcher-typesblockwatcher-portsblockwatcher-coreaxumserdeserde_jsontokio(netfeature)tracing
and, in [dev-dependencies] only:
blockwatcher-testkit: shared test scaffoldingblockwatcher-ports(fakesfeature): the in-memoryStorage/Source/etc. fakes a route handler test runs against instead of a real moduleasync-trait: implementing test-only trait impls the fakes crate doesn’t already providereqwest: driving the router over real HTTP in this crate’s own teststokio(macros/rt-multi-thread/timefeatures): async test execution
The following crate depends on it directly (per the dependency table):
blockwatcher(binary): boots the engine, builds aControlHandle, binds the API’s listener, and hands both plus the resolvedauthreference toblockwatcher_api::serveasApiState, inbootandserve_api(crates/blockwatcher/src/run.rs)
Reading the source
lib.rs: the module doc comment,ApiState, androuter: the whole crate’s shape in one file.auth/:require_bearerandrequire!(mod.rs),Scope(scope.rs), and the audit trail (audit.rs).error.rs:ApiError,classify, andstoredversusfrom: read this before any route file, since every fallible handler returnsApiError.routes/mod.rs, then the files it lists, in the order Route organisation above covers them:health.rs,status.rs,schema.rs,resources.rs(the macro, then its invocations, one per resource kind),monitors_ops.rs,networks_ops.rs.serve.rs: the graceful-shutdown wrapper theblockwatcherbinary calls once the API listener is bound and the engine is up.