A Solana WebSocket can disconnect for ten seconds and reconnect cleanly.
Your process looks healthy. Your transaction history now has a hole.
Reconnect logic restores the connection. It does not restore events that arrived while the client was offline.
A production monitor needs two paths:
live stream for speed + RPC backfill for completenessThis guide builds that loop with a durable cursor, overlap window, idempotent writes, commitment policy, and a gap detector.
Kill the default approach
Default: subscribe, reconnect on close, and assume the stream is complete.
What breaks first:
- deploys create invisible holes
- provider incidents lose events
- duplicate notifications create double processing
- processed updates are stored as final truth
- the monitor cannot prove where recovery resumed
By the end you will have
- A live-plus-backfill architecture
- A durable cursor schema
- A reconnect and overlap algorithm
- An idempotency key
- A production monitoring checklist
A subscription is a notification channel
Solana WebSocket methods deliver notifications for matching account, log, signature, slot, or block activity. Each subscription has its own filters and commitment behavior. For example, accountSubscribe supports processed, confirmed, and finalized, with finalized as the documented default. Solana: accountSubscribe
That contract does not provide an application-level replay cursor.
Provider products may add replay or persistence, but your consumer still needs to know:
- the last durable slot
- which signatures were processed
- whether writes are idempotent
- which commitment is stored
- how recovery finds missing history
Treat the stream as the fast path. Treat RPC history as the repair path.
Store a cursor you can explain
Use a cursor per workload or filter:
type MonitorCursor = {
streamId: string;
lastObservedSlot: number;
lastDurableSlot: number;
lastSignature?: string;
commitment: "processed" | "confirmed" | "finalized";
updatedAt: string;
};lastObservedSlot helps detect live progress.
lastDurableSlot advances only after the event has been written and its required side effects complete.
Do not advance the durable cursor when the callback starts. Advance it after the transaction commits.
For address-based recovery, getSignaturesForAddress returns confirmed transaction signatures newest-first and supports before, until, limit, and minContextSlot. Solana: getSignaturesForAddress
Keep the last durable signature when your filter maps cleanly to an address. Keep slot checkpoints for broader program or block workloads.
Backfill with an overlap window
Recovery should intentionally re-read some history.
disconnect detected
-> stop cursor advancement
-> reconnect live subscription
-> backfill from last durable checkpoint with overlap
-> deduplicate
-> process oldest to newest
-> advance durable cursorThe overlap handles races between the restored subscription and the backfill query.
Do not try to calculate a perfect boundary. Make duplicate processing safe.
Example recovery loop:
async function recoverAddress(
connection: Connection,
address: PublicKey,
lastSignature?: string,
) {
const recovered = [];
let before: string | undefined;
while (true) {
const page = await connection.getSignaturesForAddress(address, {
before,
until: lastSignature,
limit: 1000,
});
if (page.length === 0) break;
recovered.push(...page);
if (page.length < 1000) break;
before = page.at(-1)?.signature;
}
return recovered.reverse();
}The exact query depends on the workload. A block consumer may backfill slots and call getBlock. A transaction consumer may recover signatures, then call getTransaction.
Make every write idempotent
Use an event key that represents the chain event, not the delivery attempt.
Common keys:
transaction: <signature>
instruction: <signature>:<instruction_index>
log event: <signature>:<log_index>
account version: <pubkey>:<slot>Then enforce uniqueness in storage:
create unique index monitor_events_event_key_idx
on monitor_events (event_key);The live path and backfill path should call the same processor.
async function processEvent(event: MonitorEvent) {
await db.transaction(async (tx) => {
const inserted = await tx.events.insertIfAbsent(event.eventKey, event);
if (!inserted) return;
await applyDomainEffects(tx, event);
await tx.cursors.advance(event.streamId, event.slot, event.signature);
});
}If duplicate delivery changes state twice, the consumer is not recovery-safe.
Separate signal from truth
Low-latency systems may consume processed events because they need an early signal.
User balances, accounting records, and irreversible actions need a stronger confirmation policy.
Use two states:
observed_processed
confirmed
finalized
revertedStore the commitment with the event. Promote state when later evidence arrives. Do not overwrite provenance.
Helius documents transaction monitoring over Yellowstone gRPC with filters and execution details, while its streaming overview separates standard WebSockets, enhanced WebSockets, and gRPC-class delivery. The provider choice changes latency and operational features, but it does not remove the need for consumer-side state and idempotency. Helius: Transaction monitoring with Yellowstone gRPC, Helius: Solana data streaming
Magnet: No-Missed-Events Checklist
NO-MISSED-EVENTS CHECKLIST
STREAM
[ ] Filter and commitment are explicit.
[ ] Connection health and last observed slot are measured.
[ ] Reconnect uses bounded exponential backoff with jitter.
CURSOR
[ ] Last durable slot is stored outside process memory.
[ ] Cursor advances only after durable processing.
[ ] Signature checkpoint is stored when the workload supports it.
RECOVERY
[ ] Every reconnect triggers a backfill check.
[ ] Backfill overlaps the live boundary.
[ ] Recovered events run oldest to newest.
IDEMPOTENCY
[ ] Event keys represent chain events.
[ ] Storage rejects duplicates.
[ ] Live and backfill use the same processor.
TRUTH
[ ] Commitment is stored with each event.
[ ] Processed signals are not labeled finalized.
[ ] Reverted or missing events have a repair path.
OPERATIONS
[ ] Gap count and recovery lag are observable.
[ ] A forced disconnect test passes.
[ ] A deploy during active traffic creates no permanent hole.You should see: after killing the monitor for a controlled interval, it reconnects, backfills the interval, reports duplicates safely, and returns the durable cursor to the current slot.
Detect gaps instead of assuming completeness
Track:
- seconds since last notification
- observed slot versus cluster slot
- last durable slot
- reconnect count
- recovered event count
- duplicate count
- backfill duration
- failed processing count
Alert on a stalled cursor, not only a closed socket.
A connection can remain open while the consumer is stuck, the subscription is invalid, or downstream storage is failing.
FAQ
Does WebSocket reconnect recover missed Solana events?
No. Reconnect restores future notifications. Your consumer needs RPC backfill or a provider replay feature for the disconnected interval.
How much overlap should backfill use?
Use enough overlap to cover boundary races, then rely on idempotent event keys. Measure the duplicate count and recovery time for the chosen workload.
Should a monitor store processed or finalized events?
It may store both if their states remain explicit. Early signals should be promoted or reverted as stronger commitment evidence arrives.
Failure modes
| Failure | Symptom | Fix |
|---|---|---|
| Reconnect without backfill | Clean uptime, missing history | Recover from durable checkpoint |
| Cursor in memory | Every restart forgets its boundary | Store it transactionally |
| Cursor advances early | Failed side effect is skipped forever | Advance after durable processing |
| No overlap | Race loses events at reconnect boundary | Re-read history and deduplicate |
| Non-idempotent handler | Recovery doubles balances or alerts | Add event keys and unique constraints |
| Processed equals final | UI or accounting shows reverted state | Store and promote commitment explicitly |
| Socket-only health check | Open connection, frozen consumer | Measure slot and durable progress |
When not to build this yourself
Use a managed persistence or replay product when:
- you cannot operate an always-on consumer
- the workload needs long historical replay
- filters cover a large share of chain traffic
- recovery time must have a contractual bound
- your team cannot test and monitor gap repair
Even then, keep idempotent writes and an application cursor. Provider replay protects delivery. It cannot guarantee your database side effects completed.
Your next action: run the No-Missed-Events Checklist, then kill one devnet monitor for 60 seconds and prove the recovery path.
Related: Solana streaming paths decision matrix and Solana agent ops devnet checklist.