Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Sink

A sink is a delivery destination: an id, a module name, that module’s config, and optional engine-owned policies (SinkDef, crates/blockwatcher-types/src/resource.rs). It is written through PUT /sinks/{id} on the HTTP API or as one JSON file under a seed directory’s sinks/ subdirectory. Writing a sink restarts every network whose stored monitors name that sink id, per Resources § Lifecycle.

A complete sink carrying every policy field beside a webhook module config, the same example Delivery guarantees § Delivery policies walks through (the policy values shown are each field’s defaults):

{
  "id": "alerts",
  "module": "webhook",
  "config": {
    "url_secret": "env:ALERTS_WEBHOOK_URL",
    "headers": { "x-source": "blockwatcher" },
    "timeout_ms": 10000,
    "body_template": "{% if type == \"digest\" %}{{ matches | length }} matches{% else %}{{ type }}: {{ monitor }}{% endif %}"
  },
  "retry": { "max_attempts": 8, "initial_backoff_ms": 200, "max_backoff_ms": 30000 },
  "throttle": { "max_deliveries": 60, "window_ms": 60000 },
  "aggregate": { "window_ms": 30000, "max_batch": 100 }
}

Fields

SinkDef rejects an unrecognized field (#[serde(deny_unknown_fields)]). retry, throttle, and aggregate are siblings of config, never nested inside it: a sink module never sees or interprets its own policies, so enforcement is identical for every module. These policies run after a match exists. Counting hits or capping alerts before a match is minted is a monitor gate, not a sink field. throttle dead-letters refused matches (replayable); max_once discards them. aggregate stalls the checkpoint for the window; threshold does not. What enforcing each policy means at runtime is documented on Delivery guarantees § Delivery policies; this page owns the fields and their constraints.

KeyTypeDefaultMeaning
idstringnone (required)The sink’s name, referenced by a monitor’s actions list.
modulestringnone (required)Which sink module delivers: webhook, script, or log in shipped builds.
configobjectnone (required)The named module’s own config, documented per module below. Write-time validation constructs the module with it, so a bad config refuses the write as a 422.
retryobjectabsentDelivery retry policy. Absent falls back to the engine-wide [engine].default_retry from the instance configuration.
throttleobjectabsentDelivery admission cap. Absent means not throttled at all: throttling is opt-in.
aggregateobjectabsentDigest batching. Absent delivers every match on its own, unbatched.

retry, throttle, and aggregate are each validated by the same validate call at two points: put_sink (crates/blockwatcher-core/src/control/writes.rs) refuses the write, and validate_and_build (crates/blockwatcher-core/src/engine/boot.rs) runs the identical check against rows already in storage at boot, refusing to start the engine instead. A policy is either absent or fully valid, never partially so.

retry (DeliveryRetry)

KeyTypeDefaultMeaning
max_attemptsu328Total delivery attempts, not retries after the first: 1 delivers once and dead-letters on failure, and 0 is treated as 1, because a delivery that was never attempted cannot be honestly recorded as given up on.
initial_backoff_msu64200Delay before the second attempt. Backoff doubles per attempt; the multiplier is fixed at 2. Above max_backoff_ms refuses: retry.initial_backoff_ms (30001) must be at most retry.max_backoff_ms (30000).
max_backoff_msu6430000Cap on the doubling backoff between attempts.

throttle (Throttle)

KeyTypeDefaultMeaning
max_deliveriesu3260Successful deliveries the window admits before suppressing more. A zero refuses: throttle.max_deliveries (0) must be at least 1.
window_msu6460000The window’s length. A zero refuses: throttle.window_ms (0) must be at least 1. Above one day refuses: throttle.window_ms (86400001) must be at most 86400000; the cap itself is a legal window.

aggregate (Aggregate)

KeyTypeDefaultMeaning
window_msu6430000How long a batch stays open once the first match joins it. A zero refuses: aggregate.window_ms (0) must be at least 1. Above one day refuses: aggregate.window_ms (86400001) must be at most 86400000.
max_batchu32100The batch size that closes the batch immediately. A zero refuses: aggregate.max_batch (0) must be at least 1. Above the cap of 10,000 refuses: aggregate.max_batch (20000) must be at most 10000; the cap itself is a legal batch size.

The caps bound what one open window can cost: every buffered match is held in memory and the checkpoint stays behind all of them until the window closes.

Sink modules

Each module’s example below is copied verbatim from that crate’s registry_examples/*.json file, the same file the crate’s family-completeness test constructs; any change to those files updates these examples in the same change, per the wiki-parity rule.

webhook

POSTs each sink event as JSON to a secret-referenced URL (crates/blockwatcher-sinks/src/webhook.rs). Redirects are never followed (a 3xx fails permanently). Header values come from two maps: headers is plain configuration, and header_secrets names secret references (env:NAME) resolved fresh at each delivery, the same way url_secret is — so a secret Authorization header is supported without ever holding a resolved copy past the request that carries it. A header named in both maps has no defined winner, so the write refuses rather than guessing one. The example config, registry_examples/webhook.json:

{
  "url_secret": "env:BLOCKWATCHER_EXAMPLE_WEBHOOK_URL",
  "headers": { "x-blockwatcher-monitor": "treasury" },
  "timeout_ms": 10000
}
KeyTypeDefaultMeaning
url_secretstringnone (required)An env:NAME reference to where the URL lives, resolved at each delivery. The write refuses a value that is not a reference, and refuses a reference whose variable is absent or does not hold a URL: 'env:NAME' resolves to a value that is not a URL.
headersobject of string to string{}Extra request headers, plain values. A name or value that does not parse refuses with invalid header name '<name>': ... or invalid value for header '<name>': .... A configured content-type replaces the default application/json.
header_secretsobject of string to string{}Extra request headers whose values are env:NAME secret references, resolved at each delivery like url_secret — a resolved value is held no longer than the request that carries it. A name that does not parse refuses the same way headers does. A name that also appears in headers refuses: header '<name>' is set in both 'headers' and 'header_secrets'; ....
timeout_msu6410000Bound on one delivery attempt. A zero refuses: 'timeout_ms' must be greater than 0; 0 makes every attempt time out immediately and dead-letter.
body_templatestringabsentA minijinja template for the request body; absent keeps the canonical sink-event JSON unchanged. Validated at write time by rendering against synthetic match, retracted, and digest events, so a syntax error or a missing filter refuses with invalid webhook body_template: ... rather than dead-lettering the first real delivery. A field the template reads that a real event lacks renders empty. The template semantics, context shape, and the digest type guard are on blockwatcher-sinks § webhook.

script

Runs an operator-authored program per event with the canonical sink-event JSON on stdin (crates/blockwatcher-sinks/src/script.rs). The example config, registry_examples/script.json:

{
  "command": "/usr/local/bin/blockwatcher-notify",
  "args": ["--channel", "ops"],
  "timeout_ms": 30000
}
KeyTypeDefaultMeaning
commandstringnone (required)The program to run. An empty string refuses: script command must not be empty. The path’s existence is checked only at delivery, not at the write: the file may legitimately appear after boot.
argsarray of strings[]Arguments, passed to the program verbatim.
timeout_msu6430000Bounds the stdin write and the exit wait together. A zero refuses with the same message as the webhook’s timeout_ms.
body_templatestringabsentA minijinja template for the bytes piped to the script’s stdin; absent keeps the canonical sink-event JSON unchanged. Validated at write time by rendering against synthetic match, retracted, and digest events, so a syntax error or a missing filter refuses with invalid script body_template: ... rather than dead-lettering the first real delivery. A field the template reads that a real event lacks renders empty. Shares its template semantics, context shape, and the digest type guard with webhook’s body_template, documented on blockwatcher-sinks § script.

The exit-status contract (sysexits):

  • 0: delivered. A script that exits 0 without reading stdin has still delivered; the exit status is the acknowledgement.
  • 75 (sysexits EX_TEMPFAIL): transient, retried by the engine under the sink’s retry policy.
  • Any other exit code: permanent, straight to the dead-letter queue.
  • Death by signal, and the module’s own timeout: transient.

The last 4 KiB of the script’s stderr ride the error message; stdout is ignored.

log

Writes one line of canonical sink-event JSON to the process’s stdout per delivery (crates/blockwatcher-sinks/src/log.rs). It takes no required configuration; the example config, registry_examples/log.json, is the empty object, and any key other than body_template refuses the write:

{}
KeyTypeDefaultMeaning
body_templatestringabsentA minijinja template for the emitted line, rendered before the trailing newline; absent keeps the canonical sink-event JSON unchanged. Validated at write time by rendering against synthetic match, retracted, and digest events, so a syntax error or a missing filter refuses with invalid log body_template: ... rather than dead-lettering the first real delivery. A field the template reads that a real event lacks renders empty. Shares its template semantics, context shape, and the digest type guard with webhook’s body_template, documented on blockwatcher-sinks § webhook.