Skip to main content
The SDK provides two buffer implementations depending on your deployment:

EventBuffer β€” in-memory default

Collects events in memory so you can read them at any time instead of processing them immediately in a callback.

Setup

Constructor

max_size
int
default:"1000"
Maximum events to keep in memory. When full, the oldest event is dropped and on_drop is called. Set to 0 for unlimited (not recommended in production β€” use RedisEventBuffer instead).
on_drop
Callable[[AnyPayload], None]
default:"None"
Called with the dropped payload when the buffer is full. Use for metrics or alerting. If None, drops are logged as warnings only.

Drop handling example


Reading events

drain() β€” get all events and clear the buffer

peek(n) β€” inspect without clearing

size β€” how many events are waiting

Drain by type

Process actions and summaries at different cadences:

Periodic indexing example


RedisEventBuffer β€” production, multi-pod

Uses a Redis sorted set (ZSET) per session with a sliding timestamp window β€” the same architecture as the event connector’s internal buffer. Designed for high-throughput deployments where multiple service instances share state.

Constructor

redis_url
string
required
Redis connection URL, e.g. "redis://localhost:6379/0".
key_prefix
string
default:"\"default\""
Namespace prefix for all Redis keys. Use a unique value per service or environment to avoid key collisions.
window_seconds
float
default:"120.0"
Sliding window size in seconds. Events older than this are automatically evicted on the next write. Default matches the connector’s 2-minute window.
max_concurrent
int
default:"10"
Maximum concurrent Redis write operations. When exceeded, the payload is dropped and on_drop is called instead of queuing unboundedly.
on_drop
Callable[[AnyPayload], None]
default:"None"
Called when a payload is dropped due to backpressure or Redis being unavailable. Use for metrics or alerting.

Graceful degradation

If Redis is unavailable at startup or during operation, add() calls on_drop and returns β€” your application keeps running without buffering. drain() returns an empty list. No exceptions are raised.

API reference


BufferBackend protocol

Implement BufferBackend to plug in any durable store (Kafka, Postgres, etc.) as a custom async backend:

EventBuffer API reference