Benchmarks

scripts/bench/ times blockwatcher end to end against a cached upstream RPC, so a real change in decode, predicate, or delivery cost shows up as a real number rather than noise from repeated network calls. This page states exactly what a number here measures, how to reproduce it, and the baseline every later run diffs against.

The harness itself is chain-agnostic: bench.sh and run-one.sh boot blockwatcher --config <name>/blockwatcher.toml --seed <name>/seed and poll GET /status regardless of which source module the scenario’s network resource names. What a scenario measures, and what its numbers mean, is specific to one chain family, so this page has one section per family. Today that is EVM alone; a second chain family adds its own section here with its own scenarios and its own result columns, rather than being folded into EVM’s table under a different name.

Methodology

Applies to every scenario regardless of chain family.

What every run measures

Each scripts/bench/scenarios/<name>/ is a full blockwatcher instance, booted fresh with storage.module = "memory", so every run starts from the seed alone with nothing carried over from a previous one. A run is timed from process start until GET /status’s checkpoint cursor for the scenario’s one network reaches a fixed end position, matching every occurrence the scenario’s monitor selects along the way and delivering each match to that monitor’s log sink. The clock stops at the checkpoint, not at process exit, but delivery to the sink is inside that window: a monitor whose matches never finish delivering would keep the checkpoint from advancing. The runner polls the status endpoint every 0.2 seconds, so each wall-clock number carries up to about 0.2 seconds of quantization on top of the streamed work it measures.

Out of scope

• Webhook delivery latency: every scenario’s sink is log, not webhook, so no network hop to a receiving service is timed.
• Any storage backend other than memory: a scenario’s checkpoint and resources live in process memory for the run’s duration and are discarded when it exits, so SQLite’s or any other backend’s write cost is not represented here.
• Cold-cache latency against the real upstream: bench.sh warms each scenario’s cache with one discarded run before timing begins, so the timed rows measure blockwatcher’s own cost, not the upstream’s.

Tooling

• Docker, for the caching proxy each family’s scenarios sit behind.
• curl, jq, and bc, used by scripts/bench/run-one.sh for status polling, checkpoint extraction, and elapsed-time arithmetic.
• hyperfine.
• A release build at target/release/blockwatcher (cargo build --release -p blockwatcher).
• A real upstream endpoint reachable from this machine for whichever family’s scenarios are being run; see that family’s section for the exact form it takes and how to point the cache at it.

How bench.sh runs a scenario

bench.sh warms each scenario’s cache with one discarded run, times five runs per scenario with hyperfine (exported to scripts/bench/results-<scenario>.md, gitignored as a machine-local artifact), then prints one more counted run per scenario as a scenario,seconds,<count-column> CSV row on its own stdout. The timed rows and the counted row together are what each family’s baseline table transcribes; what the count column measures (RPC attempts, for EVM) is specific to the family.

Rows in a family’s baseline measure the engine as of this page’s own revision: the harness (this page, bench.sh, and each scenario’s seed) ships alongside the code it measures, so the commit and date a row was recorded at are in this file’s git history, and a re-run that changes a number updates the row in the same commit as the change that moved it. Changes that affect decode, predicate, or delivery cost re-run scripts/bench/bench.sh and append their own rows below a family’s baseline rather than replacing it, so a regression or an improvement is visible against a fixed starting point.

EVM

Every scenario in this section runs the evm-rpc source against Ethereum mainnet (chain 1) through a caching eRPC proxy: the numbers describe EVM-family behavior at mainnet density, and carry to neither another chain family nor another EVM source module (evm-mempool is not benchmarked here: it has no historical range to replay against a cache).

Scenarios

Each scenario’s network seed pins the evm-rpc source’s logs_window to { initial: 50, max: 50 }, sized to the baseline upstream’s own eth_getLogs range cap (see Reproducing an EVM run). Without that pin the source’s own default window (initial: 512) is wider than that cap, so its first eth_getLogs call for the range is refused every run and the source narrows and retries; with the pin, no call is ever refused, so what’s measured is blockwatcher’s own decode/predicate/delivery cost against a fixed, upstream-independent request pattern, not the shape of any one provider’s range limit. The request pattern is therefore fixed and upstream-independent (no refusal, no mid-scan narrowing), which is also what makes it fully cacheable: a warm run touches the eRPC cache for every historical call and the real upstream only for head probes.

The rpc-attempts column comes from a separate counted run, not from hyperfine’s five timed runs: hyperfine owns the timed runs’ stdout, so counting is a sixth pass over the same already-warm cache. That count also includes head probes made while the run is in flight, so it varies slightly run to run and between scenarios whose historical request pattern is otherwise identical.

Reproducing an EVM run

BENCH_UPSTREAM_RPC=https://your-mainnet-endpoint.example \
  docker compose -f scripts/bench/compose.yaml up -d
export BLOCKWATCHER_BENCH_RPC=http://127.0.0.1:4000/main/evm/1
scripts/bench/bench.sh

scripts/bench/compose.yaml pins the eRPC image to ghcr.io/erpc/erpc:0.1.2; bumping the pin invalidates recorded rows until the baseline below is re-run.

The baseline below ran with BENCH_UPSTREAM_RPC=https://eth-pokt.nodies.app (a shared, free-tier mainnet endpoint), which is also where the 50-block eth_getLogs cap each seed’s logs_window is pinned to comes from; a different upstream may need a different logs_window value to stay unrefused, per its own advertised range cap. A heavily rate-limited upstream may also need its cache pre-warmed before the first timed run: replaying the range’s own requests (the same methods and params a warm run would make) paced below the provider’s limit fills the eRPC cache without ever tripping it, which a live engine process retrying on its own schedule cannot reliably do against a tight limiter.

Baseline

s1-broad-transfers restricts nothing, so it decodes and delivers every ERC-20 Transfer in the range; its wall time reflects the log sink’s per-match cost multiplied across every one of them, not just RPC or decode cost. s2-selective-weth decodes the same range but keeps almost everything out of its predicate, so its wall time sits far closer to pure RPC-and-decode cost. s3-function-transfer turns on the full-block-with-transactions path (a functions selector fetches every block in a window via eth_getBlockByNumber rather than one ranged eth_getLogs call, plus one eth_getTransactionReceipt per matching call), so its rpc-attempts count reflects that access pattern rather than logs_window’s cap. All three rows use a fully warmed cache, so none of them include upstream latency.

Rows below this baseline carry no date of their own, on the same terms as the baseline itself: the commit and date each was recorded at are in this file’s git history.

Concurrent receipts and batched headers

The evm-rpc source’s receipt_concurrency and header_batch, both at their defaults (4 receipts in flight per leaf, 20 headers per JSON-RPC batch), against the same fixed range, the same seeds, and the same machine as the baseline above.

Request count is where the change lands hardest: batching turns a 50-block window’s 50 eth_getBlockByNumber round trips into three, which is what separates 12 and 13 attempts from the baseline’s 153 and 177, and 34 from its 130. Wall time moves only where those round trips dominated it. s3-function-transfer roughly halves, fetching its receipts four at a time across a window whose headers also arrive in one request rather than eight; s2-selective-weth drops about a quarter; s1-broad-transfers stays inside its own spread, since its wall time is the log sink’s per-match cost across every Transfer in the range rather than anything the request count governs.

Logs-bloom pre-screen

The evm-rpc source’s bloom_screen setting, at its default (true), against the same fixed range, the same seeds, and the same machine as the baseline above.

Request counts hold steady against the previous section’s rows: 12 and 13 attempts match exactly, and 34 moving to 36 reflects how far past the end block each run got before the runner stopped it, described below, rather than a change in the historical request pattern. Wall time for each scenario sits inside its own spread from the previous section, so nothing here reads as a regression or an improvement on its own.

The column’s own arithmetic caps how far this setting could ever move it. A screened window skips exactly one call, its eth_getLogs, and it cannot skip any of the header requests that came before: those headers are the screen’s own input, since a window’s blooms are read out of them, so every one is already fetched by the time the decision exists. One window’s own cost is ceil(logs_window / header_batch) header requests plus that single log call, which is three and one at a 50-block window and a 20-header batch. Reading one s2-selective-weth run’s requests out of the eRPC instance’s own log shows exactly that shape, three eth_getBlockByNumber for every eth_getLogs. So a window spends three header requests per log call, and a screen that fired on every window in the range would still leave those three standing. Set against the previous section’s rows, which already cut 153, 177, and 130 attempts down to 12, 13, and 34 by batching those same headers, the log call is the smaller remaining share rather than the dominant one, so a large drop in this column was never available here at any fill rate.

That same log also shows why a row’s total is not the benchmarked range divided by the window width. The run issued four log calls, covering 22000000 to 22000049 and 22000050 to 22000099 inside the range, and 22000100 to 22000149 and 22000150 to 22000199 beyond it. The runner stops a scenario once the pipeline’s checkpoint reaches the end block, and by that point the source has already started the windows after it, so a row’s total carries however many of those a given run reached before it was stopped. That is why these totals move between runs in steps of roughly one window’s cost rather than a request at a time, and why a row is an upper bound on the requests the benchmarked range itself needs rather than an exact count of them.

That also makes this column the wrong instrument for the saving the setting is named for. A skipped eth_getLogs is one fewer log call, and a run that skipped every log call it could would still report most of the attempts it reports now, because the header requests and the head probes are counted here too. blockwatcher_evm_bloom_skips_total counts the windows that skipped, which is the quantity the setting acts on directly.

Holding steady is the expected result on this fixture, not evidence that the screen does nothing. The screen skips a window’s eth_getLogs call only when every fetched header’s bloom refutes every monitored address, or every monitored topic0, in that window, since eth_getLogs requires both to hold and either dimension refuted alone already rules out a match; a refutation itself is certain only when the m3 membership test returns a negative for an item genuinely absent from that block. Sampling every seventh block of the benchmarked range (landing on 22000000 through 22000098) through the same eRPC cache and counting set bits in each logsBloom puts the average fill across this range at about 45% of its 2048 bits, so even for an item absent from a given block, bit collisions alone let the test falsely report it present in roughly 0.45^3 (about 9%) of blocks. A 50-block window only skips when all 50 of its headers refute, so the probability that every one of them does compounds to under 1%, even for an address that appears nowhere on chain; at that rate a skip is a rare event across a handful of windows, not a routine one. On top of that baseline rarity, s2-selective-weth monitors an address and topic0 that genuinely occur inside the range, so its windows could never skip regardless of fill rate, and s1-broad-transfers restricts nothing at all, so a filter with no address or topic to refute against is never screened by design.

Where the setting earns its keep is sparser blocks and narrower windows: a chain or range whose headers carry a lower average bloom fill pushes the false-admission rate down for every block in a window, and a smaller logs_window needs fewer of those blocks to agree before it can skip. Re-running the s2-selective-weth scenario against this same cached range, with its seed’s logs_window pinned to { initial: 10, max: 10 } and its monitored address replaced by one that appears nowhere in the range, then reading blockwatcher_evm_bloom_skips_total off the metrics endpoint confirms the mechanism directly: those narrower windows do skip, and the series appears and grows as the run proceeds. The rows above are the honest result for a dense mainnet range at the seeds’ default window, not the feature’s best case.