Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

blockwatcher-ports

blockwatcher-ports is the trait boundary that makes blockwatcher chain-agnostic: it defines the port traits every module implements (Source, Decoder, Matcher, Gate, Sink, Storage), their write-time and runtime error types, the registration contract every module family enumerates itself through, and, behind two features, the in-memory test doubles that let consumers exercise a port’s contract without linking any real module at all.

Its production dependencies are blockwatcher-types plus async plumbing (tokio, tokio-util, futures, async-trait) and (de)serialization glue (serde, serde_json, thiserror). No chain SDK, no HTTP client, no storage driver appears anywhere in its allowlist or its transitive tree; that is what lets blockwatcher-core, which depends on this crate directly, drive a pipeline without ever importing one.

Key takeaways

  • blockwatcher-ports is the trait boundary that makes blockwatcher chain-agnostic: it defines the port traits (Source, Decoder, Matcher, Gate, Sink, Storage), their error types, and the module registration contract.
  • Behind the fakes and testing features it also provides in-memory test doubles for every port, so a consumer can exercise a port’s contract without linking any real module.
  • No chain SDK, HTTP client, or storage driver appears anywhere in its allowlist or transitive tree, which is what lets blockwatcher-core depend on it directly without importing one.

Responsibilities

  • Define the port traits and the associated types each one needs: Source (SourceCtx, SourceStatus, SourceOutcome, InterestSet, ScanRange), Decoder (CompiledSpec, CompiledSelector, DecodeOutcome, SpecSet), Matcher (CompiledPredicate, Explanation, ExplanationNode), Gate (CompiledGate, GateHit, GateDecision, GateAux, GateCtx), Sink, and Storage (BatchEntry).
  • Define the write-time and runtime error enum for each port (SourceError, SpecError, SelectorError, PredicateError, SinkError, GateError, StorageError) and ErrorClass, the retry classification every one of them reports through the Classify trait.
  • Define ModuleRegistry, the trait a module family implements once per module to name itself and hand back a constructor, plus the *Factory type alias beside each port trait that its Factory associated type resolves to.
  • (fakes feature) Provide one in-memory implementation per port: FakeSource, FakeDecoder, FakeMatcher, PassthroughGate, FakeSink, MemoryStorage, and FlakyStorage (a fault-injecting wrapper around MemoryStorage), each registered through the same ModuleRegistry contract a real module uses, so they are executable proof that a port hides no backend-specific detail, not just a convenience.
  • (testing feature) Re-export mockall-generated mocks: one per port trait (MockSource, MockDecoder, MockMatcher, MockGate, MockSink), and one per storage facet (MockResourceStore, MockCheckpointStore, MockDeadLetterStore, MockPauseStore, MockDeliveryJournal, MockGateStore) for unit-level substitution.

Not this crate’s job: implementing any real module: blockwatcher-evm, blockwatcher-rpc, blockwatcher-storage, blockwatcher-sinks, and blockwatcher-expr each implement exactly one port over real chain, network, or storage logic; blockwatcher-gates implements Gate the same way; running a pipeline, retrying a failed delivery, or deciding when a checkpoint advances (blockwatcher-core owns all of that: a Sink or Source implementation never retries internally); the predicate language’s grammar and evaluator (blockwatcher-expr defines those against the Matcher trait this crate only declares).

Key types and traits

NameKindRole
SourcetraitPulls raw activity into a pipeline; owns its own cursor semantics and run loop
DecodertraitCompiles a chain artifact into chain-agnostic schemas once, then decodes raw payloads against the compiled result
MatchertraitCompiles predicate source against a SchemaSet, then evaluates it against decoded events
GatetraitCompiles a monitor’s gate.config, then decides Retain/Discard/Emit over an engine-owned journal
SinktraitDelivers one SinkEvent (Match or Retracted); the engine owns all retry/backoff/dead-letter policy
StoragetraitResource CRUD, checkpoints, dead letters, and the bounded delivery journal, with optimistic concurrency on every write
ModuleRegistrytraitThe NAME + Factory + factory() contract every module declares itself through
ClassifytraitReports one of four ErrorClass values for any port error
ErrorClassenumTransient / Permanent / RateLimited / RetryNarrower: the whole retry-policy vocabulary
SourceError, SpecError, SelectorError, PredicateError, SinkError, GateError, StorageErrorenumOne thiserror enum per port, each with its own Classify impl
CompiledSpec, CompiledSelector, CompiledPredicate, CompiledGatestructType-erased compiled artifacts (downcast::<T>()), built once at write time and read on the hot path
InterestSet, ScanRangestructHints a source may use to narrow fetching, and the bounded range for a one-shot history scan
SourceCtx, SourceStatus, SourceOutcomestruct/enumEverything the engine hands a running source, its typed liveness status, and how run returns without a SourceError (Ended or Invalidated { from })
DecodeOutcomestructOne decode() call’s result: decoded events plus an undecodable payload count
Explanation, ExplanationNodeenum/structThe dry-run “why did or didn’t this match” tree a Matcher::explain returns
BatchEntrystructOne create in Storage::put_batch’s create-only batch
FakeSource, FakeDecoder, FakeMatcher, PassthroughGate, FakeSink, MemoryStorage, FlakyStoragestruct(fakes feature) In-memory, minimal implementations of every port

Object safety

Every port trait is consumed as a trait object, never through its concrete type: each *Factory alias resolves to Arc<dyn Source>, Arc<dyn Decoder>, Arc<dyn Matcher>, Arc<dyn Gate>, Arc<dyn Sink>, or Arc<dyn Storage> (source.rs, decoder.rs, matcher.rs, gate.rs, sink.rs, storage.rs), and every module’s registered factory constructs and returns exactly one of those. Staying dyn-compatible is load-bearing for that reason, and every trait keeps it this way:

  • No generic methods. None of them declares a method with its own type parameter: every method’s arguments and return type are concrete port types (&RawEvent, &CompiledSpec, &SchemaSet, and so on). The generic constructors this crate does define, CompiledSpec::new::<T> and CompiledSelector::new::<T> (decoder.rs) and CompiledPredicate::new::<T> (matcher.rs) and CompiledGate::new::<T> (gate.rs), are inherent methods on plain structs, not methods on a port trait, so they never have to satisfy object safety at all.
  • async fn only behind the macro that makes it dyn-compatible. Source, Sink, and Storage each declare async fn methods and are annotated #[async_trait] (source.rs, sink.rs, storage.rs), which rewrites every async fn into a plain fn returning a boxed, pinned future (the shape a trait object can actually hold), since a bare async fn in a trait is not itself object-safe. Decoder and Matcher have no async methods at all: compile_spec, compile, decode, interest, and merge_interest on Decoder, and compile, matches, explain, and referenced_fields on Matcher, all run synchronously, so both stay a plain pub trait X: Send + Sync with no macro needed (decoder.rs, matcher.rs). Gate is the same shape: compile, on_hit, and defaulted on_invalidate are synchronous (gate.rs).

The Send + Sync bound itself sits once, on the trait declaration, for the same reason: every one is declared pub trait X: Send + Sync as a supertrait bound (source.rs, decoder.rs, matcher.rs, gate.rs, sink.rs, storage.rs), so every dyn X is already Send + Sync by construction. None of the *Factory aliases repeats the bound on its own Arc<dyn X>: there is no Arc<dyn X + Send + Sync> anywhere in this crate, because the supertrait already settled it once, at the trait itself.

How data flows through it

The crate’s defining shape is the split between write time, when a monitor or spec is compiled into an opaque artifact, and the hot path, which only ever reads that artifact back through a typed downcast:

flowchart TD
    subgraph wt["Write time: once per monitor/spec"]
        Spec -->|"Decoder::compile_spec"| CS["CompiledSpec"]
        CS -->|"Decoder::compile(selectors, specs)"| CSel["CompiledSelector"]
        Text["predicate source"] -->|"Matcher::compile(text, schemas)"| CP["CompiledPredicate"]
    end
    subgraph hp["Hot path: once per event"]
        Raw["RawEvent"] -->|"Decoder::decode(raw, selector)"| DE["DecodedEvent"]
        DE -->|"Matcher::matches(predicate, event)"| Bool["bool"]
        Bool -->|"Sink::deliver(event)"| Result["Ok / SinkError"]
    end
    CSel -.->|"read via downcast"| DE
    CP -.->|"read via downcast"| Bool

Every port besides Storage follows this same compile-once, evaluate-many shape; Storage is the exception, since every one of its operations is already a runtime call with no separate write-time compilation step.

Neighbours

blockwatcher-ports depends on the following in production:

  • blockwatcher-types: vocabulary crate
  • tokio: async plumbing
  • tokio-util: async utilities
  • futures: future combinators
  • async-trait: trait async support
  • serde: (de)serialization derive
  • serde_json: JSON values for module configuration and factories
  • thiserror: error derive

Optionally in production (behind testing and fakes features):

  • mockall: mockall-generated trait mocks (testing feature)
  • num-bigint: arbitrary-precision integers (fakes feature)

In [dev-dependencies] only:

  • tokio (test-util feature): for fakes module async tests

The following crates depend on it directly (per the dependency table):

  • blockwatcher-core: engine pipeline
  • blockwatcher-api: HTTP API
  • blockwatcher-expr: predicate language and matcher module
  • blockwatcher-rpc: RPC chain module
  • blockwatcher-evm: EVM chain module
  • blockwatcher-testkit: workspace test fixtures
  • blockwatcher-storage: storage module
  • blockwatcher-sinks: sink modules
  • blockwatcher-gates: gate modules
  • blockwatcher (binary): process binary

blockwatcher-evm-testkit reaches it only transitively, through blockwatcher-evm and blockwatcher-rpc.

Reading the source

  1. Start at lib.rs: the module list, the full set of re-exports, and which of them are feature-gated.
  2. error.rs: ErrorClass, Classify, and the seven port error enums; read this before any port trait, since every fallible method returns one of these.
  3. registry.rs: ModuleRegistry and BoxFuture, the shared contract every module family’s own registration builds on.
  4. source.rs, decoder.rs, matcher.rs, gate.rs, sink.rs, storage.rs: the port traits, each beside the associated types and factory alias it needs; read in this order, since Source’s InterestSet and Decoder’s interest/merge_interest defaults are easiest to follow before Matcher, Gate, and Sink, which are simpler traits.
  5. fakes/mod.rs, then fakes/source.rs, fakes/decoder.rs, fakes/matcher.rs, fakes/gate.rs, fakes/sink.rs, fakes/storage.rs, and fakes/flaky_storage.rs: one minimal implementation per port, each registered through ModuleRegistry exactly as a real module would be.