Resources

Everything blockwatcher watches and everywhere it delivers to is one of its resource kinds, each its own Rust struct in crates/blockwatcher-types/src/resource.rs, each rejecting an unrecognized field at write time (#[serde(deny_unknown_fields)], one exception noted below). This page covers what each kind actually contains, how a monitor references the others, how a resource gets into storage in the first place, and exactly what gets checked before a write is allowed to land. The field-level tables (every key, its type, its default, and the exact refusal a bad value gets) live in the Resource reference, one page per kind.

flowchart LR
    rpc["RPC endpoints<br/>(external chains)"]
    sources["Sources<br/>evm-rpc · evm-mempool"]
    sinks["Sinks<br/>webhook · script · log"]
    storage["Storage<br/>checkpoints · dead letters · resources"]
    api["REST API"]
    metrics["Metrics"]
    engine["engine<br/>bounded channels · checkpoints"]

    subgraph pipeline["Engine pipeline"]
        direction LR
        decoder["Decoder"]
        matcher["Matcher<br/>predicates"]
        gate["Gate<br/>threshold · max_once"]
        decoder --> matcher
        matcher --> gate
    end

    rpc --> sources
    sources -->|"decode and match"| decoder
    gate --> sinks
    api -->|"manages resources"| storage
    storage <--> engine
    engine -->|"drives"| pipeline
    engine -.->|"reports"| metrics

    classDef module fill:none,stroke:#a9a3e3
    classDef core fill:none,stroke:#8a8d86,stroke-dasharray: 5 5

    class rpc,sources,sinks,decoder,matcher,gate module
    class engine,api,storage,metrics core
classDef dim fill:none,stroke:#999999,color:#999999,opacity:0.35
classDef focus fill:#ffd43b,stroke:#d9480f,stroke-width:3px,color:#1a1a1a
class rpc,sources,sinks,metrics,engine,decoder,matcher,gate dim
class storage,api focus
click sources "selectors.html"
click decoder "chain-agnosticism.html"
click matcher "predicates.html"
click gate "gates.html"
click sinks "delivery.html"
click storage "resources.html"
click api "../reference/http-api.html"
click metrics "../reference/observability.html"
click engine "pipeline.html"

Key takeaways

• blockwatcher’s resource kinds are exactly Network, Spec, SinkDef, and Monitor, each rejecting an unrecognized field except Monitor’s selectors, checked later by the decoder.
• A monitor is the only resource that references any other: it names one network, one spec per selector, and one or more sinks.
• Every resource is a versioned record, written through the REST API’s ETag/If-Match concurrency or, once, through a first-boot seed directory.
• What restarts on a write differs by kind: a monitor write hot-swaps in place, a network write always restarts, and a spec or sink write restarts every network it affects.
• Every write validates before it reaches storage, and every validation rejection surfaces on the wire as 422.

The kinds and their fields

Network (Network, crates/blockwatcher-types/src/resource.rs): an id, a chain tag, and a source module selection (ModuleSel: a module name plus that module’s own opaque config object). Nothing else. It is the only resource whose module family (Source) is configured on the resource itself rather than referenced from elsewhere.

Spec (Spec): an id, a chain tag, and a payload that core never interprets: for the evm chain family this is a Solidity JSON ABI array, meaningless to anything outside that one chain’s decoder. The write itself is what runs EvmDecoder::compile_spec (crates/blockwatcher-evm/src/decoder/mod.rs), turning the ABI into a SchemaSet and a selector-keyed dispatch table exactly once; a selector or predicate written against this spec afterward resolves against that already-built result, so re-parsing the ABI text is a write-time cost this resource pays only once, not something every later selector compile repeats.

SinkDef (SinkDef): an id, a module name, that module’s config, an optional retry policy (DeliveryRetry { max_attempts, initial_backoff_ms, max_backoff_ms }), an optional throttle policy (Throttle { max_deliveries, window_ms }), and an optional aggregate policy (Aggregate { window_ms, max_batch }). All of them are siblings of config rather than nested inside it: a sink module never sees or interprets its own retry, throttle, or aggregate policy, the engine owns retry, dead-lettering, throttling, and aggregation identically for every module. A SinkDef with no retry falls back to the engine-wide default from blockwatcher.toml; a SinkDef with no throttle is not throttled at all, since throttling is opt-in; a SinkDef whose throttle field is present fills any field left out of it with the default, 60 deliveries per 60,000ms; a SinkDef with no aggregate delivers every match on its own, unbatched. See Delivery policies for what enforcing each one means, and for a complete SinkDef JSON example carrying all three beside a webhook module config.

Monitor (Monitor): an id, the network it watches, one or more selectors, an optional predicate, an optional gate (ModuleSel: module + config; omitted means passthrough), and actions (a list of sink ids). gate is this monitor’s decision rule (like predicate), not a shared resource id (like actions: SinkId[]). Journals are per (pipeline, monitor). RawSelector is deliberately the one resource shape without deny_unknown_fields: it carries a core-readable spec field plus a #[serde(flatten)] bag of decoder-owned keys (events, functions, addresses for the evm decoder), and serde cannot combine flatten with deny_unknown_fields on the same struct. An unrecognized selector key is still rejected, just later, by the chain’s decoder at compile time rather than by serde at parse time, in compile’s own unknown-key check (crates/blockwatcher-evm/src/decoder/selector.rs).

How they reference each other

A monitor is the only resource that names any of the others, and it names all of them:

erDiagram
    Network {
        NetworkId id
        ChainKind chain
        ModuleSel source
    }
    Spec {
        SpecId id
        ChainKind chain
        Json payload
    }
    SinkDef {
        SinkId id
        string module
        DeliveryRetry retry
        Throttle throttle
        Aggregate aggregate
    }
    Monitor {
        MonitorId id
        NetworkId network
        RawSelector[] selectors
        string predicate
        ModuleSel gate
        SinkId[] actions
    }
    Monitor }o--|| Network : "network"
    Monitor }o--o{ Spec : "selectors[].spec"
    Monitor }o--o{ SinkDef : "actions"

Monitor.gate is optional; mermaid cannot show Option cleanly, so omitted means passthrough, as Gates describes. Monitor.network names exactly one Network; each entry in Monitor.selectors names exactly one Spec through its spec field; and Monitor.actions names one or more SinkDefs. Network, Spec, and SinkDef never reference each other or point back at a monitor: the reference graph is a star with Monitor at the center, one level deep. A Spec and the Networks that use it are connected only indirectly, through whichever monitors select that spec on that network; spec_set_for_chain (crates/blockwatcher-core/src/compile.rs) compiles every spec sharing a network’s chain together, which is why a spec edit can affect a network that no monitor explicitly ties to it by name (see the write-time checks below).

Lifecycle: how a record is created, versioned, and reloaded by kind

Every resource is a VersionedRecord in storage: an id, a version for optimistic concurrency, and the JSON value (crates/blockwatcher-types/src/resource.rs). A record gets there by exactly the routes below, and by no other:

• The REST API, PUT /{kind}/{id}. Without an If-Match header the write is a create: it fails with 409 Conflict if the id already exists. With If-Match: "<version>" it is an update: it fails with 412 Precondition Failed (naming the actual current version) if the version doesn’t match, or 404 if the record is gone, per Storage::put’s contract and the write handler’s call to if_match (crates/blockwatcher-ports/src/storage.rs, crates/blockwatcher-api/src/routes/resources.rs). A successful write’s response carries the resulting version in its own ETag, which is what the next conditional write is expected to send back as If-Match.
• A seed directory, read once at boot via the binary’s --seed <dir> flag. It expects exactly one subdirectory per kind (networks/, specs/, sinks/, monitors/), one JSON file per resource, and it is strictly first-boot: seeding runs the whole bundle through the same validation Engine::start itself would run, then writes it with create-only semantics, and once storage holds any resource of any kind, seeding is refused as a no-op on every later boot, per validate and store_is_empty (crates/blockwatcher/src/seed.rs). After that first boot, every resource in that deployment is managed exclusively through the API.

What happens to a running pipeline differs by kind, and this is the one place “hot-reloaded” oversimplifies:

• Writing a Monitor usually restarts nothing. It recompiles that monitor’s network’s whole MonitorSet from storage and publishes it to the already-running pipeline through a watch channel: the same source instance keeps running, its checkpoint keeps advancing straight through the swap, in ControlHandle::put_monitor (crates/blockwatcher-core/src/control/writes.rs). Two things turn that swap into a restart of the one network:

  • Changing gate (module or config) is a new compiled artifact and drops that monitor’s gate journal. Holds belong to one envelope, and a processor keeps the journal of every gated monitor it is running in memory, so the drop may only happen with nothing processing for that network: the pipeline is stopped first, the holds dropped, and the pipeline brought back, in wipe_gate_state_while_stopped (crates/blockwatcher-core/src/control/mod.rs). Deleting a gated monitor takes the same path, per delete_monitor (crates/blockwatcher-core/src/control/deletes.rs). The wipe drops holds (undecided accumulation), never committed emissions: an outbox row the old envelope emitted but had not yet delivered survives and is delivered (or dead-lettered) when the pipeline comes back, per the gate delivery guarantee.
  • If the recompiled set now names a sink the running pipeline never spawned a worker for, the write escalates to a full restart, because a sink worker cannot be added to a pipeline after the fact, per hot_swap_monitors’s own doc comment (crates/blockwatcher-core/src/engine/control.rs).

  Both cost only what any restart costs: the source resumes from the checkpoint the drain left behind. A network with no running pipeline is never brought up by either: there is nothing to protect, and starting one would undo whatever stopped it.

• Writing a Network always restarts that network’s pipeline: the old one is drained, a fresh one is spawned, and it resumes from whatever checkpoint the drain left behind, in put_network (crates/blockwatcher-core/src/control/writes.rs). A restart is cheap specifically because the checkpoint survives it; it is not a re-scan from the beginning.

• Writing a SinkDef restarts every network whose stored monitors currently name that sink id, not every network in the deployment, per put_sink (crates/blockwatcher-core/src/control/writes.rs).

• Writing a Spec restarts every network on either the spec’s new chain or (on a reassignment) its prior chain, because spec_set_for_chain compiles every spec sharing a chain together: a network that never named this spec by id can still be running a compiled set built partly from it, per put_spec (crates/blockwatcher-core/src/control/writes.rs).

A restart that fails part-way through a multi-network fan-out (a SinkDef or Spec write affecting several networks) does not fail the request or stop the ones after it: the resource is already durably stored either way, and each network’s own restart failure is only logged, in restart_affected (crates/blockwatcher-core/src/control/writes.rs).

Whatever triggers it, a restart has the same observable side effects: a network write, a spec or sink write, a gate-envelope change, a gated monitor’s delete, and reorg recovery all go through this same replacement. Each one resets the /status counter snapshot for that network and rebuilds its sink workers, so in-memory throttle and aggregate windows start fresh; the Prometheus counters, checkpoints, journals, and outbox are unaffected, since none of them live in the pipeline instance a restart replaces.

Write-time validation

Every write validates before it ever reaches storage, and every rejection is 422 Unprocessable Entity on the wire: classify, in crates/blockwatcher-api/src/error.rs, classifies exactly this set of engine refusals that way, distinguishing them from a version conflict (412), a missing reference lookup (404), or a genuine server fault (500, and never with internal detail on the wire). What gets checked differs by kind:

• Monitor: its network must exist; every id in actions must name an existing sink; every selector’s spec must exist and share the network’s chain; and the whole monitor (selectors, predicate, and gate) must compile against the live decoder, matcher, and gate catalog before the write is persisted, in put_monitor (crates/blockwatcher-core/src/control/writes.rs). An unknown gate module, unknown config keys, out-of-range window_ms/count, or missing block.timestamp is 422: gate requires 'block.timestamp'; this monitor's selectors do not expose it.
• Network: its chain must have a loaded decoder, and its source module must actually construct with the given config, not merely be a name the catalog recognizes. Updating an existing network additionally recompiles every monitor already stored for it against the incoming chain, refusing a chain reassignment that would strand them, in put_network (writes.rs).
• SinkDef: its module must actually construct with the given config. Nothing about existing monitors is re-validated: a sink id, once it exists, is a stable dependency by name, per put_sink (writes.rs).
• Spec: its chain must have a loaded decoder, and the payload must compile standalone; then every monitor on every network that would be affected (per the chain-sharing rule above) is recompiled against a hypothetical spec set with this write already applied, so a reassignment that would break one of them is refused before it is stored, in put_spec (writes.rs).
• Delete, for Network/Spec/SinkDef, refuses outright while any stored monitor still references the id (checked before storage is touched at all), because a monitor left pointing at nothing would not fail on its own; it would fail the next boot for the whole deployment, per refuse_if_referenced (crates/blockwatcher-core/src/control/deletes.rs).

The did-you-mean suggestion

A 422 for an unresolvable name (an event or function a selector names that the spec doesn’t declare) carries a suggestion when one is available. For a selector’s events/functions list, the suggestion is simply the first event or function of that kind the spec declares, kind_suggestion (crates/blockwatcher-evm/src/decoder/selector.rs); it is not an edit-distance match. A predicate’s unknown field or namespace, covered on Predicates, gets a real bounded edit-distance suggestion instead: the mechanisms are deliberately different, one per crate that owns the vocabulary being checked. Either way the message shape is the same: SelectorError::UnknownField and PredicateError::UnknownField both render as unknown field '{field}', followed by a did you mean '{suggestion}'? suffix when one exists (crates/blockwatcher-ports/src/error.rs); for example, naming "Transfr" in a selector against a spec that declares Transfer and Approval rejects with unknown field 'Transfr' plus did you mean 'Approval'? (the suggestion is the spec’s first declared event of that kind, not the closest by spelling).