Streaming data pipelines

Streaming data pipelines are a data pipeline architecture that handle millions of events at scale, in real time.

Streaming data pipelines are executed continuously, all the time. They consume streams of messages(events), apply operations, such as transformations, filters, aggregations to the messages(events), and publish the processed messages to another stream.