Elasticity vs. State? Exploring Kafka Streams Cassandra State Store

Hartmut Armbruster23 minutes

kafka-streams-cassandra-state-store' is a drop-in Kafka Streams State Store implementation that persists data to Apache Cassandra.

By moving the state to an external datastore the stateful streams app (from a deployment point of view) effectively becomes stateless. This greatly improves elasticity and allows for fluent CI/CD (rolling upgrades, security patching, pod eviction, ...).
It also can also help to reduce failure recovery and rebalancing downtimes, with demos showing sporty 100ms rebalancing downtimes for your stateful Kafka Streams application, no matter the size of the application’s state.

As a bonus accessing Cassandra State Stores via 'Interactive Queries' (e.g. exposing via REST API) is simple and efficient since there's no need for an RPC layer proxying and fanning out requests to all instances of your streams application.

Share this

Video Slides

Hartmut Armbruster, Software Architect, Developer, Thriving.dev

Hartmut Armbruster is a senior software engineer with many years of experience designing and building solutions for high-load projects using distributed technologies. Over the past 6 years he has been working in real-time data processing at HSBC, NEX Group plc, Deutschebahn on mission critical platforms. Striving to see the bigger picture, passionate about architecture, combining, integrating, and bringing all things together.