Prepare data pipelines for the streaming age

Confluent introduces Stream Designer
Prepare data pipelines for the streaming age

providers on the subject

Confluent has announced Stream Designer, a visual interface that lets you create and deploy streaming data pipelines in minutes. It runs natively on Apache Kafka.

Confluent brings the creation of data pipelines up to speed with Stream Designer.

(Image: Confluent)

The visual point-and-click builder aims to promote the democratization of data streams, as they are now accessible to developers without special Kafka knowledge. This should enable companies to quickly create and iterate streaming pipelines to quickly connect data and promote agile development and decision making.

“We are in the midst of a profound technology shift where real-time data streaming is becoming the new normal, enabling new business models, better customer experiences and more efficient operations,” said Confluent CEO Jay Kreps. The stream designer should support this development and make real-time streaming of the entire data stream standard in companies.

Native usage on Kafka

Running natively on Kafka, Stream Designer provides developers with a flexible point-and-click canvas. Additionally, they can create pipelines and describe data flows and business logic in the Confluent Cloud UI in minutes. The solution uses a developer-centric approach that allows users with different skills and needs to seamlessly switch between the user interface, the code editor, and the command-line interface when creating data flow logic declaratively.

Among the benefits, Confluent cites increased developer productivity, as they no longer have to spend days or even weeks managing individual open source Kafka components, and the use of separate solutions such as Kafka Streams or Kafka Connect is no longer necessary. The unified end-to-end view also simplifies maintaining and updating pipelines throughout their lifecycle.

Pipelines created with Stream Designer can also be exported as SQL source code to be shared, deployed in other environments, or integrated into existing CI/CD workflows. Multiple users can make and work on live changes to the same pipeline, which Confluent says allows for seamless collaboration and knowledge transfer.


Leave a Comment