Testing strategy

blockwatcher’s tests are layered by what they run against, and each layer answers a question the layers around it cannot: a unit layer proves a crate’s own logic against an in-memory stand-in for whatever port it depends on; a module layer proves a real module’s wire-level or storage-level behavior against a real (if local and disposable) backend; and a black-box end-to-end layer proves the assembled binary against a real chain. None of the three substitutes for either of the others, and none of them is optional: Architecture decisions § A trait is only trustworthy if something fake proves it states the rule the first layer exists to satisfy, and the crates this page is about (blockwatcher-testkit, blockwatcher-evm-testkit, blockwatcher-e2e) are the shared scaffolding the second and third layers are built on.

This page does not restate what a fake, a mock, or a harness actually does: blockwatcher-ports documents the fakes and testing features, and the three crate pages above document their own harnesses in full. This page’s job is to say which layer each of them belongs to, and how CI reaches every layer.

Key takeaways

• Tests are layered by what they run against: unit tests prove a crate’s own logic against an in-memory fake, module tests prove a real module against a real local backend, and end-to-end tests prove the assembled binary against a real chain.
• None of the three layers substitutes for the others, and none is optional.
• blockwatcher-testkit, blockwatcher-evm-testkit, and blockwatcher-e2e are the shared scaffolding the second and third layers are built on.

The three layers

Each layer proves something the layers around it cannot:

flowchart TD
    unit["unit tests<br/>fake per port:<br/>FakeSource, MemoryStorage..."] --> module["module tests<br/>real backend, run locally:<br/>mock rpc node, sqlite file"]
    module --> e2e["end to end tests<br/>real binary, real chain:<br/>anvil, blockwatcher-e2e"]

Unit tests: one fake per port

Every port trait blockwatcher-ports declares ships with an in-memory implementation behind its fakes feature: FakeSource, FakeDecoder, FakeMatcher, FakeSink, MemoryStorage, and FlakyStorage (a fault-injecting wrapper around MemoryStorage), each registered through the same ModuleRegistry contract a real module uses. A crate that needs to drive a port without linking any real module enables this feature on its blockwatcher-ports dev-dependency: blockwatcher-api, blockwatcher-core, blockwatcher-evm, blockwatcher-expr, blockwatcher-sinks, blockwatcher-storage, and the blockwatcher binary itself all do, each in [dev-dependencies] only. This is the fakes-per-port rule in its mechanical form: a change to a port’s shape that the matching fake cannot satisfy is a signal the port’s own design needs another look before anything downstream of it does.

blockwatcher-ports also declares a testing feature, gated behind which mockall::automock generates mocks for every port trait (MockSource, MockDecoder, MockMatcher, MockSink) and every storage facet (MockResourceStore, MockCheckpointStore, MockDeadLetterStore, MockPauseStore, MockDeliveryJournal, MockGateStore) (crates/blockwatcher-ports/src/{decoder,matcher,sink,source,storage}.rs, each carrying #[cfg_attr(feature = "testing", mockall::automock)] on its trait declaration). No workspace crate’s Cargo.toml enables that feature on its own blockwatcher-ports dependency: every port substitution anywhere in this workspace’s tests goes through a hand-written fakes implementation instead. The testing feature and its generated mocks still compile, and still get exercised, under the check job’s cargo test --workspace --all-features and the feature-powerset job’s combinatorial build (see How CI runs each layer below); they exist as available unit-level substitution infrastructure that this workspace’s own test suites have not, so far, needed to reach for over a hand-written fake.

Module tests: real backends, run locally

One level up from a port fake, a module’s own tests prove its wire-level or storage-level behavior against something that behaves like the real backend it talks to, not against another abstraction:

• blockwatcher-storage’s tests/contract.rs runs blockwatcher-testkit’s exercise_storage_contract against the ports fake, its own in-memory module, and its sqlite module over a real file on disk, in turn, so all three are proven equivalent by one shared behavioral contract rather than by shared code. See blockwatcher-testkit § The storage contract.
• blockwatcher-evm‘s own tests/*.rs and several of its src/ modules’ #[cfg(test)] blocks run the evm-rpc and evm-mempool sources against blockwatcher-evm-testkit’s scripted JSON-RPC and WebSocket mock node and its SimChain fixture, which answers real wire shapes (including reorgs) rather than a chain-agnostic port’s in-memory stand-in. The point of this layer is proving the decode and retry logic against realistic responses, which a Source-level fake has no wire format to get wrong in the first place.

End-to-end: the real binary, against a real chain

blockwatcher-e2e is the one layer that drives the assembled blockwatcher binary itself, as a spawned process, and asserts only on what an operator can observe from outside it. The scenarios in tests/e2e/anvil.rs need a real anvil node rather than any mock, because their assertions are about what a real node does with a transaction, a block, and a confirmation barrier, not about what a scripted handler was told to answer.

How CI runs each layer

.github/workflows/ci.yml declares these jobs, and each one exercises a different slice of the layers above:

A vX.Y.Z tag runs .github/workflows/release.yml instead of this table. Its preflight job re-runs the check job’s overlapping steps (including the foundry pin and BLOCKWATCHER_E2E_REQUIRE_ANVIL) plus scripts/check-release-version.sh; it does not repeat windows-check or feature-powerset. The publish job then reflows the matching changelog section into GitHub Release notes, attaches a Linux archive, and pushes both Docker images to GHCR. See CONTRIBUTING.md.

Only the check job actually runs tests, and it runs all of them in one cargo test --workspace --all-features invocation: there is no separate CI job per layer, because the layering above is a property of what each test is written to run against, not of how CI schedules them. The BLOCKWATCHER_E2E_REQUIRE_ANVIL: "1" environment variable on that one step is what turns blockwatcher-e2e’s anvil-backed scenarios from a locally-friendly skip (an absent anvil prints a marker and returns early, so a contributor without foundry installed stays unblocked on everything else) into a hard failure in CI, where anvil was just provisioned and its absence would otherwise be silent. See blockwatcher-e2e § Staying out of the default build for why blockwatcher-e2e is still a full participant in cargo test --workspace despite shipping no production code at all.

Consistency with the rest of the wiki

This page intentionally does not restate what already has a canonical home elsewhere in the wiki:

• Exactly what blockwatcher-ports’ fakes and testing features provide is documented once, in blockwatcher-ports § Responsibilities.
• The mechanical rule that nothing outside [dev-dependencies] may depend on blockwatcher-testkit or blockwatcher-evm-testkit, and that blockwatcher-e2e carries a deliberately empty allowlist entry, is documented once, in Architecture decisions § How the dependency gate turns rules into a mechanical check and Workspace map § Verifying the rings.