Modules and trade-offs

Every other concept page treats one specific behavior as swappable: a source in The pipeline, a matcher in Predicates. This page steps back and treats “swappable” itself as the organizing idea: what a module actually is, how one gets into a running binary, and what it means for an operator that every axis of behavior in blockwatcher is one.

flowchart LR
    rpc["RPC endpoints<br/>(external chains)"]
    sources["Sources<br/>evm-rpc · evm-mempool"]
    sinks["Sinks<br/>webhook · script · log"]
    storage["Storage<br/>checkpoints · dead letters · resources"]
    api["REST API"]
    metrics["Metrics"]
    engine["engine<br/>bounded channels · checkpoints"]

    subgraph pipeline["Engine pipeline"]
        direction LR
        decoder["Decoder"]
        matcher["Matcher<br/>predicates"]
        gate["Gate<br/>threshold · max_once"]
        decoder --> matcher
        matcher --> gate
    end

    rpc --> sources
    sources -->|"decode and match"| decoder
    gate --> sinks
    api -->|"manages resources"| storage
    storage <--> engine
    engine -->|"drives"| pipeline
    engine -.->|"reports"| metrics

    classDef module fill:none,stroke:#a9a3e3
    classDef core fill:none,stroke:#8a8d86,stroke-dasharray: 5 5

    class rpc,sources,sinks,decoder,matcher,gate module
    class engine,api,storage,metrics core
classDef dim fill:none,stroke:#999999,color:#999999,opacity:0.35
classDef focus fill:#ffd43b,stroke:#d9480f,stroke-width:3px,color:#1a1a1a
class rpc,api,metrics,engine dim
class sources,decoder,matcher,gate,sinks,storage focus
click sources "selectors.html"
click decoder "chain-agnosticism.html"
click matcher "predicates.html"
click gate "gates.html"
click sinks "delivery.html"
click storage "resources.html"
click api "../reference/http-api.html"
click metrics "../reference/observability.html"
click engine "pipeline.html"

Key takeaways

• blockwatcher has exactly six ports (Source, Decoder, Matcher, Gate, Sink, Storage), and a module is one concrete implementation of exactly one of them.
• A module never straddles two ports, even when several modules live in the same crate, and configuration resolves a name against the running binary’s own module catalog.
• Two modules behind the same port can trade off completeness against something else, most visibly evm-rpc (at-least-once delivery, higher latency) against evm-mempool (pending visibility, no restart recovery).
• Switching modules is the operator’s actual lever for a trade-off: changing one module leaves every other port’s behavior untouched, since none of them are written against one module’s assumptions.

Everything an operator composes is a module

An blockwatcher deployment has exactly six axes along which behavior can differ: how it reads a chain, how it turns raw bytes into blockwatcher’s canonical shape, how it decides whether a decoded occurrence counts, when a predicate-true hit becomes a delivery, where a match goes, and where resources and operational state persist. Those six axes are the six ports blockwatcher defines (Source, Decoder, Matcher, Gate, Sink, Storage), and a module is, by definition, one concrete implementation of exactly one of them. Nothing in blockwatcher is configured by writing code against a bespoke integration point; every one of these six choices is made the same way, by naming a module in configuration and giving it whatever config object that module expects.

The six ports and every module registered behind each one, in this build, look like this:

flowchart TD
    source["Source"] --> rpc["evm-rpc"]
    source --> mempool["evm-mempool"]
    decoder["Decoder"] --> evm["evm"]
    matcher["Matcher"] --> expr["expr"]
    gate["Gate"] --> threshold["threshold"]
    gate --> maxonce["max_once"]
    sink["Sink"] --> webhook["webhook"]
    sink --> script["script"]
    sink --> log["log"]
    storage["Storage"] --> memory["memory"]
    storage --> sqlite["sqlite"]

    %% Layout only, no meaning: these invisible edges wrap the six ports
    %% into bands. Without them every port sits on one row, which renders
    %% too wide for the content column and shrinks the labels. Keep them.
    rpc ~~~ sink
    evm ~~~ storage
    expr ~~~ maxonce

ModuleCatalog (crates/blockwatcher-core/src/catalog.rs) is where that naming resolves: one name-keyed map per port family, each holding factory functions folded in from every compiled-in module’s own get_all() enumeration. A config that names a module absent from the catalog (a typo, or a module simply not compiled into this build) fails at boot or at write time with EngineError::UnknownModule, listing every alternative that actually is registered, via unknown and the family! macro’s lookup arm (catalog.rs), rather than panicking or silently no-op’ing that pipeline stage. Because the list comes from the catalog itself, the message always names what this particular binary carries, never a superset the workspace merely contains somewhere.

One module, one port, one name

A module never straddles two ports. crates/blockwatcher-evm, for instance, ships two Source implementations (evm-rpc, evm-mempool) and one Decoder (evm): three separate modules living in one crate, each registered under its own name and each satisfying exactly one port’s trait, never blending source and decode logic into a single type, in build_catalog (crates/blockwatcher-embed/src/catalog.rs, folding blockwatcher_evm::sources::get_all() and blockwatcher_evm::decoders::get_all() separately into the catalog). Which module handles which resource is itself fixed by the port: a network names its Source module, a spec’s chain determines its Decoder, a sink resource names its Sink module, blockwatcher.toml names the one Matcher and one Storage backend for the whole process. An operator never picks a module without also picking, structurally, which port it fills.

Trade-offs are a property of the module, not a setting

Because two modules behind the same port are interchangeable at the trait level, they are free to differ arbitrarily in the trade-offs they make. blockwatcher leans on that rather than trying to expose every axis as a tunable knob on one do-everything implementation. The clearest example ships on the Source port, between the EVM sources:

• evm-rpc scans confirmed blocks and logs over RPC. It only ever reports something once that something has a fixed position in the chain, which is exactly what lets it make at-least-once delivery’s promise: its checkpoint names a real block to resume from, so a crash costs at worst a duplicate, never a gap. The cost is latency and RPC quota: an event is only visible once it is mined and this source has polled far enough to see it.
• evm-mempool watches a node’s pending-transaction feed instead. It reports a transaction before it is ever mined, which is strictly faster, but at the cost of the one guarantee evm-rpc can make: a pending transaction may never be mined at all, and this source’s cursor is a per-run arrival counter rather than a chain position, so a restart cannot resume it: whatever was pending while the process was down is simply gone. Delivery guarantees covers exactly what that costs a consumer.

Choosing between them is choosing a point on a latency-vs-completeness trade-off, and blockwatcher does not try to collapse that choice into a single configurable source with a “mode” flag: the two behaviors are different enough, and the guarantee difference consequential enough, that they are two separate modules an operator picks between by name, each documenting its own side of the trade-off rather than one module documenting a matrix of settings. The same pattern of “trade-off lives in which module you picked, not in a shared config surface” carries across the other five ports as well: the webhook sink trades a network dependency for delivering anywhere HTTP reaches; the log sink trades reach for having none; sqlite storage trades a single-writer constraint for surviving a restart, memory storage trades restart survival for zero setup; threshold trades per-hit delivery for a session digest that does not stall the checkpoint; max_once trades later in-window hits for a single alert.

The module catalog

Every row below is a module registered into ModuleCatalog by one of the get_all() calls build_catalog (crates/blockwatcher-embed/src/catalog.rs) folds together. The table is exhaustive in both directions: every module compiled into this binary appears here, and every row names something a real registration call, cross-checked against each family’s own enumeration, actually produces.

Omit-gate is engine passthrough, not a catalog row an operator must name. Matcher is still one module for the whole process. Gate is per monitor, like sink.

Module selection is by name in configuration; a name the running binary never registered fails loudly, with the list of names it actually did register, rather than silently doing nothing.

Why this is the operator’s actual lever

Because a trade-off lives inside the module rather than in a shared knob, “choose your trade-offs” and “choose your modules” are the same action for an operator. Wanting faster visibility into pending activity, accepting that some of it will never be confirmed, means switching a network’s source from evm-rpc to evm-mempool, not flipping a setting on one source that tries to serve both goals at once. The six-port boundary is what makes that swap safe to make at all: switching sources changes nothing about how the decoder, the matcher, a gate, or any configured sink behaves, because none of them were ever written against one source’s assumptions to begin with.