HTTP API reference

blockwatcher’s REST control plane is an axum router mounted on the [api].listen address from instance configuration. Every mutation goes through the same ControlHandle a --seed load and the engine’s own boot path use, and every read goes through core’s typed storage facade without taking any lock the engine holds: a read stays answerable while a write is in flight. There is no path prefix: routes are exactly as shown below, against the listener’s own address, assembled in router (crates/blockwatcher-api/src/serve.rs, lib.rs).

flowchart LR
    rpc["RPC endpoints<br/>(external chains)"]
    sources["Sources<br/>evm-rpc · evm-mempool"]
    sinks["Sinks<br/>webhook · script · log"]
    storage["Storage<br/>checkpoints · dead letters · resources"]
    api["REST API"]
    metrics["Metrics"]
    engine["engine<br/>bounded channels · checkpoints"]

    subgraph pipeline["Engine pipeline"]
        direction LR
        decoder["Decoder"]
        matcher["Matcher<br/>predicates"]
        gate["Gate<br/>threshold · max_once"]
        decoder --> matcher
        matcher --> gate
    end

    rpc --> sources
    sources -->|"decode and match"| decoder
    gate --> sinks
    api -->|"manages resources"| storage
    storage <--> engine
    engine -->|"drives"| pipeline
    engine -.->|"reports"| metrics

    classDef module fill:none,stroke:#a9a3e3
    classDef core fill:none,stroke:#8a8d86,stroke-dasharray: 5 5

    class rpc,sources,sinks,decoder,matcher,gate module
    class engine,api,storage,metrics core
classDef dim fill:none,stroke:#999999,color:#999999,opacity:0.35
classDef focus fill:#ffd43b,stroke:#d9480f,stroke-width:3px,color:#1a1a1a
class rpc,sources,sinks,storage,metrics,engine,decoder,matcher,gate dim
class api focus
click sources "../concepts/selectors.html"
click decoder "../concepts/chain-agnosticism.html"
click matcher "../concepts/predicates.html"
click gate "../concepts/gates.html"
click sinks "../concepts/delivery.html"
click storage "../concepts/resources.html"
click api "http-api.html"
click metrics "observability.html"
click engine "../concepts/pipeline.html"

Key takeaways

• Every route except GET /health requires a bearer token from the labelled [auth] table, checked in constant time; a missing or unknown token is 401, a valid token with too weak a scope is 403. Every failure returns the same ErrorBody shape.
• Writes go through the same ControlHandle a --seed load and the boot path use; reads go through core’s typed storage facade without taking any lock a write holds.
• Every resource kind shares one CRUD shape with ETag/If-Match optimistic concurrency: PUT with no If-Match creates, with If-Match updates, and DELETE requires If-Match.
• A monitor test is a dry run: it reports matches without delivering to any sink, registering a cursor, or moving a checkpoint.
• Network operations (pause, resume, skip, checkpoint delete, dead letters) act on persisted state and the running pipeline directly; none of them replay through the normal ingestion path.

Every request travels the same shape, from the door to the response body:

flowchart LR
    req["HTTP request"] --> health{"path is /health?"}
    health -->|"yes"| ok["200 ok<br/>no token required"]
    health -->|"no"| auth{"require_bearer"}
    auth -->|"missing/wrong"| e401["401 unauthorized"]
    auth -->|"valid"| scope{"require(min scope)"}
    scope -->|"too weak"| e403["403 forbidden"]
    scope -->|"allowed"| route{"read or write?"}
    route -->|"write"| handle["ControlHandle<br/>same path --seed and boot use"]
    route -->|"read"| facade["core's typed<br/>storage facade"]
    handle --> resp["JSON response"]
    facade --> resp

Bodies are JSON in both directions unless noted. The route table is grouped below the same way the route modules split it: health, status, schema, resources, monitor operations, network operations (crates/blockwatcher-api/src/routes/mod.rs).

Authentication

Every route except GET /health requires:

Authorization: Bearer <token>

checked by require_bearer (crates/blockwatcher-api/src/auth/mod.rs) against the labelled [auth] table. Each row’s secret is an env:NAME reference resolved at request time; the process never holds a copy of the value between requests. The comparison is constant-time on length-equal inputs; scheme matching is case-insensitive per RFC 7235. /health is exempted by an explicit path check ahead of the router, not by a separate mount, so a request to a path that genuinely doesn’t exist is refused for lacking a token before the router gets a chance to reveal that it wouldn’t have matched anyway. A valid token is then authorized per MethodRouter by require!(Scope::…): read lists and reads, operate pauses/resumes/ replays, admin writes resources (including script sinks).

A missing or unknown token gets:

{ "error": { "code": "unauthorized", "message": "a bearer token is required" } }

with status 401 and a WWW-Authenticate: Bearer response header.

curl -H "Authorization: Bearer $BLOCKWATCHER_API_TOKEN" http://127.0.0.1:8080/status

Errors

Every failure (an engine refusal or one the API layer decides on its own) leaves through the same body shape, ErrorBody/ErrorDetail (crates/blockwatcher-api/src/error.rs):

{
  "error": {
    "code": "version_conflict",
    "message": "version conflict: expected 3, actual 5",
    "actual_version": 5
  }
}

actual_version appears only on a 412 version conflict; every other error omits it. A 500 always answers with a fixed generic body, {"error":{"code":"internal","message":"internal error"}}, regardless of what actually failed; the real detail goes to the server log only, never the wire, in body and into_response (error.rs).

Health

GET /health

The one route served without a token: a liveness probe has to answer even when secrets aren’t available yet. Asserts nothing about any pipeline.

Response 200:

{ "status": "ok" }

curl http://127.0.0.1:8080/health

Status

GET /status

A point-in-time snapshot of every currently running, paused, or abandoned pipeline, sorted by network id (blockwatcher_core::EngineStatus, crates/blockwatcher-core/src/status.rs).

Response 200, one entry per network in pipelines:

{
  "instance": "b3f1a2c4-9e21-4a6a-8c3e-1f2d3a4b5c6d",
  "paused_monitors": ["usdc-sepolia-alerts"],
  "pipelines": [
    {
      "network": "sepolia",
      "source": { "status": "catching_up", "behind": 42 },
      "checkpoint": { "cursor": { "primary": 11424039, "secondary": 54 }, "module": "evm-rpc" },
      "head": { "primary": 11424081, "secondary": 0 },
      "lag": 42,
      "in_flight_events": 0,
      "event_queue": { "len": 0, "capacity": 1000 },
      "sink_queues": { "log-sink": { "len": 0, "capacity": 100 } },
      "counters": {
        "decoded": 118, "undecodable": 0, "matched": 12, "match_errors": 0,
        "delivered": 12, "dead_lettered": 0, "dispatch_failed": 0,
        "checkpoint_write_failed": 0, "dead_letter_write_failed": 0,
        "misrouted": 0, "checkpoint_regressions_refused": 0,
        "untimestamped": 0, "gate_persist_failed": 0, "gate_hits_dropped": 0,
        "gated": 0, "gate_emitted": 0
      },
      "dead_letter_count": 0,
      "gate_outbox_depth": 0,
      "paused_monitors": []
    }
  ]
}

instance is an opaque per-process identity; two reads returning different values mean the engine restarted in between. It is null when the composition root embedding blockwatcher supplies none.

Top-level paused_monitors is every monitor an operator has paused, as persisted — pipeline membership does not enter into it, so a monitor on a paused network still appears here. It is null when storage would not answer, deliberately never an empty list: a caller cannot tell “nothing is paused” from “the store did not answer” from an empty list, and this field is what a consumer reads as the truth about pause.

source.status is one of starting, live, degraded (carries reason), catching_up (carries behind), paused (the control-plane pause view, never something a source itself reports), or abandoned (no running pipeline and no pause — a failed restart left the network unattended). checkpoint is null until the first event on that network fully completes; lag is null whenever either side it subtracts is unknown.

dead_letter_count is read fresh from storage on every call — unlike counters.dead_lettered, an in-memory count that resets to 0 whenever this pipeline restarts (a sink or monitor edit restarts every network that references it). A caller that wants “how many dead letters exist right now” should read dead_letter_count, not the counter. It is null, never 0, when that storage read fails.

gate_outbox_depth is how many committed gate emissions are still awaiting delivery: the durable outbox rows written when a gate emitted that no sink worker has settled yet. Also read fresh from storage on every call, so a paused network’s pending emissions stay visible for as long as they sit undelivered; a non-zero depth on a paused pipeline means a resume still owes deliveries. Like dead_letter_count, it is null, never 0, when the storage read fails.

A pipeline entry’s own paused_monitors is the monitors that running pipeline is actually suppressing, read from its published set. Compare with the top-level field: that is what an operator asked for, this is what is in force. They diverge only when a republish failed after a persisted pause, which is logged and repaired by the next rebuild.

curl -H "Authorization: Bearer $TOKEN" http://127.0.0.1:8080/status

Schema

GET /specs/{id}/schema

The whole vocabulary a predicate written against this spec may address: compiled through the spec’s chain decoder, not read off the raw stored payload, so it’s the decoder’s own translation rather than the ABI or IDL text itself.

Response 200 (blockwatcher_types::SchemaSet):

{
  "events": [
    { "name": "Transfer", "kind": "event", "fields": [
      { "name": "from", "ty": "address" },
      { "name": "value", "ty": "uint" }
    ] }
  ],
  "namespaces": {
    "tx": [{ "name": "hash", "ty": "bytes" }]
  }
}

Errors: 404 not_found if the spec doesn’t exist.

curl -H "Authorization: Bearer $TOKEN" http://127.0.0.1:8080/specs/usdc-erc20/schema

Resources: networks, specs, sinks, monitors

Every resource kind answers on the identical CRUD shape, generated once by the resource_routes! macro so a client written against any kind works against all of them (crates/blockwatcher-api/src/routes/resources.rs):

The collection GET returns the bare list, unpaginated: resource counts are operator-scale, not event-scale. Every item route carries optimistic concurrency through ETag / If-Match:

• GET {id} returns the resource with its current version in ETag ("3").
• PUT {id} with no If-Match is a create: 201 with the new version in ETag, or 409 already_exists if the id is already there. PUT {id} with If-Match: "<version>" is a conditional update: 200 with the new version on success, 412 version_conflict (body carries actual_version) if the version has moved on, 404 not_found if the record is gone. The path’s id and the body’s id field must agree, or the write is refused as 422 invalid_resource.
• DELETE {id} requires If-Match: "<version>": omitting it is 428 precondition_required, since a delete has no later read that would reveal it clobbered a change it never saw. A version that doesn’t match is 412 version_conflict; success is 204.

If-Match takes a strong, quoted, numeric ETag exactly as GET returned it ("3"); anything else is 400 malformed_if_match.

Resource bodies, from crates/blockwatcher-types/src/resource.rs (every shape rejects an unrecognized field except Monitor.selectors[], whose extra keys are decoder-owned and checked later, at compile time):

Network: source.config’s shape is the named module’s own; for evm-rpc (EvmRpcConfig, crates/blockwatcher-evm/src/source/rpc/config.rs) that’s a pool of endpoints (each an EndpointDef with name, a url_secret reference, optional priority/rate_limit/weight, crates/blockwatcher-evm/src/source/endpoint.rs), a required start_block, and the module’s own tunables:

{
  "id": "sepolia",
  "chain": "evm",
  "source": {
    "module": "evm-rpc",
    "config": {
      "start_block": 11424310,
      "endpoints": [
        { "name": "primary", "url_secret": "env:SEPOLIA_RPC_URL", "priority": "high", "rate_limit": { "rps": 10 } }
      ],
      "confirmations": 12,
      "max_lag_blocks": 100,
      "poll_interval_ms": 3000,
      "logs_window": { "initial": 1000, "max": 5000 },
      "probe_interval_ms": 30000
    }
  }
}

Spec: payload is the chain’s own decode artifact, opaque to core (a Solidity ABI array for evm):

{ "id": "usdc-erc20", "chain": "evm", "payload": [ { "type": "event", "name": "Transfer", "inputs": [] } ] }

SinkDef: retry is optional; a sink with none falls back to [engine].default_retry. throttle is optional too, and caps how often the sink admits deliveries; a sink with none is not throttled at all, and a present throttle object fills in any omitted field with 60 deliveries per 60,000ms. aggregate is optional as well, and batches several matches into one digest delivery; a sink with none delivers every match on its own, unbatched. See Delivery policies for what each field means and how a rejection is enforced:

{
  "id": "ops-slack",
  "module": "webhook",
  "config": { "url_secret": "env:OPS_SLACK_WEBHOOK_URL" },
  "retry": { "max_attempts": 5, "initial_backoff_ms": 200, "max_backoff_ms": 30000 },
  "throttle": { "max_deliveries": 60, "window_ms": 60000 },
  "aggregate": { "window_ms": 30000, "max_batch": 100 }
}

Monitor: predicate is optional (no predicate matches everything the selectors decode); actions names one or more SinkDef ids:

{
  "id": "usdc-sepolia-transfers",
  "network": "sepolia",
  "selectors": [{ "spec": "usdc-erc20", "addresses": ["0x1c7D..."], "events": ["Transfer"] }],
  "predicate": "args.value > 1_000e6",
  "actions": ["ops-slack"]
}

An optional gate is a sibling of predicate. Absent is passthrough:

{
  "id": "usdc-burst",
  "network": "sepolia",
  "selectors": [{ "spec": "usdc-erc20", "addresses": ["0x1c7D..."], "events": ["Transfer"] }],
  "predicate": "args.value > 1_000e6",
  "gate": { "module": "threshold", "config": { "count": 3, "window_ms": 3600000 } },
  "actions": ["ops-slack"]
}

Errors common to every write: 422 codes covering unresolvable references, an uncompilable predicate or gate, an unconstructible module config, and so on: see Resources: write-time validation for exactly what’s checked per kind, and 404 not_found / 409 already_exists / 412 version_conflict for the concurrency cases above. DELETE on a Network, Spec, or SinkDef additionally refuses as 422 still_referenced while any stored monitor still names that id.

A gate write is 422 when:

• the module name is unknown (the message lists catalog gate names, the same shape as an unknown sink module)
• this monitor’s selectors do not expose block.timestamp: gate requires 'block.timestamp'; this monitor's selectors do not expose it
• count or window_ms is out of range (threshold.count is 2..=10000; window_ms is 1..=86400000)

curl -H "Authorization: Bearer $TOKEN" http://127.0.0.1:8080/monitors

etag=$(curl -s -o /dev/null -w '%header{etag}' \
  -H "Authorization: Bearer $TOKEN" \
  http://127.0.0.1:8080/monitors/usdc-sepolia-transfers)

curl -X PUT http://127.0.0.1:8080/monitors/usdc-sepolia-transfers \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -H "If-Match: $etag" \
  -d '{"id":"usdc-sepolia-transfers","network":"sepolia","selectors":[{"spec":"usdc-erc20","addresses":["0x1c7D..."],"events":["Transfer"]}],"predicate":"args.value > 2_000e6","actions":["ops-slack"]}'

curl -X DELETE http://127.0.0.1:8080/monitors/usdc-sepolia-transfers \
  -H "Authorization: Bearer $TOKEN" -H "If-Match: \"4\""

Monitor operations

POST /monitors/{id}/pause
POST /monitors/{id}/resume
POST /monitors/{id}/test

pause and resume persist the monitor’s pause: a process restart does not clear it, and every path that rebuilds the monitor set — a network, sink, or spec write that restarts the pipeline, or a fresh boot — reads the persisted set back, per the pause handler’s own doc comment, which covers both operations (crates/blockwatcher-api/src/routes/monitors_ops.rs). The persist is what succeeds; publishing the change to a pipeline that happens to be running is best-effort, logged rather than reported on failure. Both answer 204 on success.

test is a dry run: it reports every match the monitor’s current selectors, predicate, and gate (fresh empty journal, no persist of gate_hits) would produce against supplied input, delivering nothing: no sink is called, no cursor is registered, and no checkpoint moves, which is what makes it safe to point at a monitor serving live traffic. The request body is exactly one of payloads (caller- supplied raw payloads) or fetch (a bounded history read through the network’s own source module):

{ "payloads": [ { "json": { "from": "0x...", "value": "1000000" } } ] }

{ "fetch": { "from": { "primary": 11424000, "secondary": 0 }, "to": { "primary": 11424100, "secondary": 0 }, "limit": 50 } }

Response 200:

{
  "results": [
    { "matched": [ { "id": "...", "monitor": "usdc-sepolia-transfers", "network": "sepolia", "event": { "kind": "event", "name": "Transfer", "fields": {}, "cursor": { "primary": 0, "secondary": 0 } } } ], "undecodable": 0, "no_match": false, "eval_errors": 0 }
  ],
  "explanations_supported": true,
  "held": 0,
  "dropped": 0
}

held and dropped live on the report (not each result): they count this request’s gate Retain and Discard decisions. A monitor with no gate leaves both at 0. One payload against threshold with count: 3 returns matched: [] and held: 1. Three in-window payloads fire one digest whose matches are flattened into matched.

Each result’s explanation field is present only when a decoded event didn’t match and the matcher module was consulted for why: omitted when nothing needed explaining, JSON null when the matcher has no explain support, and the explanation tree otherwise.

no_match is true when no selector wanted this input at all — a log at the wrong address/topic0, a function no monitor named — as opposed to a decoded event the predicate rejected, or an unremarkable input with nothing to report. All three otherwise leave matched empty and undecodable/eval_errors at 0; no_match is what tells them apart.

Errors: 422 invalid_resource when both or neither of payloads / fetch is present, or when fetch targets a source that can’t fetch history (message "this source cannot fetch history"); 413 too_many_payloads past the shared cap of 100 inputs (blockwatcher_core::MAX_TEST_INPUTS).

curl -X POST http://127.0.0.1:8080/monitors/usdc-sepolia-transfers/pause \
  -H "Authorization: Bearer $TOKEN"

curl -X POST http://127.0.0.1:8080/monitors/usdc-sepolia-transfers/test \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"payloads":[{"json":{"from":"0x1","to":"0x2","value":"5000000"}}]}'

Network operations

POST   /networks/{id}/pause
POST   /networks/{id}/resume
POST   /networks/{id}/skip
DELETE /networks/{id}/checkpoint
GET    /networks/{id}/dead-letters
DELETE /networks/{id}/dead-letters
POST   /networks/{id}/dead-letters/{match_id}/replay
DELETE /networks/{id}/dead-letters/{match_id}

pause and resume persist the network’s pause in exactly the same sense as the monitor operations above: a restart does not clear it, and a paused network is never spawned at boot, per the pause handler’s own doc comment (crates/blockwatcher-api/src/routes/networks_ops.rs). pause cancels and drains the running pipeline as part of the same request; resume restarts it. Both answer 204 on success; a restart failure inside resume, after the persisted pause has already been cleared, is logged rather than returned as an error.

Skip

POST /networks/{id}/skip

Moves a paused network’s checkpoint and source start_block forward, without replaying the gap in between: catch-up while the operator has decided it isn’t worth reading.

{ "to": "tip" }

{ "to": { "block": 11424100 } }

Response 200 (blockwatcher_core::SkipReport):

{ "network": "sepolia", "cursor": { "primary": 11424100, "secondary": 0 }, "start_block": 11424100, "previous_checkpoint": { "cursor": { "primary": 11423000, "secondary": 12 }, "module": "evm-rpc" } }

Refusal cases, all named explicitly rather than left to fall through a generic status:

• Not paused: 409 conflict: "cannot skip network '{id}' while it is not paused; pause it first".
• Rewind: 422 invalid_resource when the target is behind the current checkpoint: "cannot skip to block {N}: checkpoint is already at {M}".
• Unsupported tip: 422 invalid_resource, "this source cannot report a confirmed tip", when "to": "tip" targets a source module that has no way to answer it (e.g. evm-mempool).
• A tip lookup that is merely unreachable right now (a transient RPC failure) is 503 unavailable instead, distinct from the source genuinely not supporting the concept at all.

Checkpoint delete

DELETE /networks/{id}/checkpoint

An explicit reset for an orphaned checkpoint (one whose network resource has already been deleted). Deleting a network deliberately leaves its checkpoint behind so a network re-created under the same id resumes where the old one stopped; this route is how an operator clears that row for good once they know it won’t be reused. The checkpoint is the only row a network delete leaves waiting: pending gate emissions are dead-lettered by the delete itself (see the gate delivery guarantee), so they show up in this network’s dead letters rather than sitting in an outbox nothing reads.

Response: 204 on success.

Errors: 409 conflict ("cannot reset checkpoint for network '{id}' while the network resource still exists") while the network resource is still there; 404 not_found if there’s no checkpoint for the id at all.

Dead letters

GET    /networks/{id}/dead-letters
DELETE /networks/{id}/dead-letters
POST   /networks/{id}/dead-letters/{match_id}/replay
DELETE /networks/{id}/dead-letters/{match_id}

A dead letter can be listed, replayed, or discarded without an attempt at delivery, singly or in bulk. Discarding is permanent and does not consult the sink at all; replaying is the only route that resends a letter.

These routes answer for any id that still holds letters, whether or not its network resource exists: a deleted network’s letters (including the ones its own delete minted from pending gate emissions) stay listable, replayable (the sink resource outlives the network), and discardable. An id with neither a network resource nor letters is 404 not_found, so a typo’d id still refuses rather than answering an empty list.

GET takes offset and limit query parameters (limit defaults to 100) and pages the queue in insertion order:

curl -H "Authorization: Bearer $TOKEN" \
  "http://127.0.0.1:8080/networks/sepolia/dead-letters?offset=0&limit=50"

Response 200:

{
  "entries": [
    {
      "match_id": "5f1136ec114001c7f726218b59527365",
      "monitor": "usdc-sepolia-transfers",
      "sink": "ops-slack",
      "cursor": { "primary": 11424039, "secondary": 54 },
      "attempts": 5,
      "reason": "permanent: webhook host returned 500 five times",
      "payload": { "type": "match", "id": "5f1136ec...", "monitor": "usdc-sepolia-transfers", "network": "sepolia", "event": {} }
    }
  ]
}

payload is omitted for a letter recorded before replay support existed, which is exactly the case replay below refuses outright, since there is nothing to resend. A stored payload is a tagged SinkEvent (type: match or type: retracted). Legacy rows that stored a bare Match object still load as type: match.

POST /networks/{id}/dead-letters/{match_id}/replay

Rebuilds the letter’s sink from its stored config and re-enters the same retry path a live delivery uses. Success deletes the letter: 204. Exhaustion leaves it queued with attempts and reason updated in place and answers 502 replay_failed. A letter with no stored payload refuses as 422 invalid_resource with the message "dead letter has no replay payload (recorded before replay support)" rather than attempting anything. A letter whose payload is a retraction refuses as 422 invalid_resource with "dead letter payload is a retraction; only match events can be replayed".

Identity is (match_id, sink): MatchId is not unique across sinks. ?sink= may be omitted when that match id is unique on the network; otherwise the call answers 409 conflict with "match '{id}' is dead-lettered for more than one sink; pass sink to name the row". The same query applies to the single-letter DELETE below.

curl -X POST \
  "http://127.0.0.1:8080/networks/sepolia/dead-letters/5f1136ec114001c7f726218b59527365/replay?sink=ops-slack" \
  -H "Authorization: Bearer $TOKEN"

DELETE /networks/{id}/dead-letters

Discards every letter matching the optional sink and monitor query filters, without attempting delivery on any of them. Absent filters means “any”; both present combine as AND. Nothing matching discards zero, which is not an error.

Response 200:

{ "discarded": 3 }

curl -X DELETE \
  "http://127.0.0.1:8080/networks/sepolia/dead-letters?sink=ops-slack" \
  -H "Authorization: Bearer $TOKEN"

DELETE /networks/{id}/dead-letters/{match_id}

Discards one letter by its match id, without attempting delivery. The optional ?sink= query is the same disambiguation as replay: omit it only when the match id is unique.

Response: 204 on success.

Errors: 404 not_found when no letter with that match id (and sink, if given) exists — which is what lets a caller tell a race against a concurrent replay from a success. 409 conflict when the match id is queued for more than one sink and sink was omitted.

curl -X DELETE \
  "http://127.0.0.1:8080/networks/sepolia/dead-letters/5f1136ec114001c7f726218b59527365?sink=ops-slack" \
  -H "Authorization: Bearer $TOKEN"