Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

How it works in one picture

Every network you register runs its own pipeline: a chain-specific source feeding a chain-agnostic decode-and-match stage, fanning out to whichever sinks the matching monitors name. A separate engine wires those pipelines up from resources managed over the REST API, and owns the bounded queues and checkpoint bookkeeping that make delivery safe to resume after a restart.

flowchart LR
    subgraph pipeline["one pipeline per network"]
        direction LR
        rpc("RPC source"):::module
        mem("Mempool source"):::module
        dec("Decoder"):::module
        mat("Matcher"):::module
        gate("Gate"):::core
        sw("Sink worker<br/>throttle · aggregate"):::core
        wh("Webhook sink"):::module
        scr("Script sink"):::module
        lg("Log sink"):::module
        rpc --> dec
        mem --> dec
        dec --> mat
        mat --> gate
        gate --> sw
        sw --> wh
        sw --> scr
        sw --> lg
    end

    api("REST API"):::core -->|writes resources| eng
    sto("Storage port<br/>checkpoints · dead letters"):::module <--> eng
    eng("engine<br/>bounded channels · checkpoint tracker"):::core --> pipeline
    eng -. interest hints .-> rpc
    eng -. interest hints .-> mem

    classDef module fill:none,stroke:#a9a3e3
    classDef core fill:none,stroke:#8a8d86,stroke-dasharray: 5 5

Boxes drawn with a solid outline are swappable modules: an operator picks evm-rpc or evm-mempool for the source, webhook, script, or log for each sink, and so on. The engine and the REST API, drawn with a dashed outline, are fixed: every pipeline is built the same way regardless of which modules fill its module slots.

One event’s journey

  1. A source pulls activity. The RPC source (module name evm-rpc) polls its configured endpoints for new blocks and the logs and transactions in them; the Mempool source (evm-mempool) instead subscribes to one node’s pending-transaction feed. Either way, what reaches the next stage is a raw, chain-native payload tagged with a cursor, the source’s own position marker.

  2. The decoder normalizes it. Decoder takes that raw payload and the compiled schema from the relevant contract spec, and turns it into blockwatcher’s canonical value model, the same shape of data regardless of which chain family produced it. Nothing past this point knows or cares that the source was EVM-specific.

  3. The matcher evaluates every monitor. For each of the network’s active monitors whose selectors apply to this occurrence, Matcher runs the monitor’s compiled predicate (if it has one) against the decoded fields. A monitor with no predicate accepts everything its selectors already narrowed down to.

  4. A gate may stay quiet. If the monitor names a gate, the engine offers this hit and the persisted journal to that module. Retain or Discard mints nothing and does not stall the checkpoint. Emit mints a match or digest and continues as today.

  5. A match is fanned out. Every monitor that accepted the occurrence (Emit / passthrough) produces one match with a deterministic id, and the engine routes it to a Sink worker for every sink that monitor’s actions name. A sink with no throttle or aggregate configured delivers that match immediately; a sink with one of those set instead holds it in a window, dropping or bundling matches per the policy, and eventually delivers a plain match or a digest of everything the window collected. Either way delivery is one attempt per sink, independently retried.

  6. The checkpoint waits for the whole event. The engine only advances the network’s checkpoint past this event once every match it produced (across every monitor and every sink) has either been delivered or exhausted its retries and become a dead letter. Quiet gate hits are already done (outstanding 0). An event that produced no matches at all completes immediately; an event with five dispatched deliveries waits for all five.

If the process crashes anywhere in that sequence, it resumes on restart from the last checkpoint that step 6 actually persisted. Anything from an event at or before that checkpoint is done: matches were delivered or dead-lettered before the checkpoint advanced. Anything after it is redone from scratch, meaning a match already delivered right before the crash can be delivered again, but nothing already accepted onto a queue is ever skipped. A duplicate is possible; a gap is not.