Show HN: DuckDB for Kafka Stream Processing
dm03514 Monday, December 08, 2025
Hello Everyone! We built SQLFlow as a lightweight stream processing engine.
We leverage DuckDB as the stream processing engine, which gives SQLFlow the ability to process 10's of thousands of messages a second using ~250MiB of memory!
DuckDB also supports a rich ecosystem of sinks and connectors!
https://sql-flow.com/docs/category/tutorials/
https://github.com/turbolytics/sql-flow
We were tired of running JVM's for simple stream processing, and also of bespoke one off stream processors
I would love your feedback, criticisms and/or experiences!
Thank you
Summary
The article provides an introduction to SQL Flow, a tool that helps data analysts and developers create and manage complex SQL workflows. It covers the key features of SQL Flow, including its ability to handle data transformations, scheduling, and monitoring, making it easier to manage SQL-based data pipelines.
34
11
Summary
sql-flow.com