When(why) do we use akka streams instead of kafka streams or spark or storm or flink?

If we need to stream data from “kafka” and perform some transformations and persist the results on AWS S3 or Azure Blob storage or Google cloud storage or kafka or hdfs we have wide range of options to choose from like spark structured streaming, kafka streams, storm, flink, akka streams.

Akka streams gaining traction in streaming world from past couple of years, mainly from early 2019. It sets its own use cases and scenarios to become a alternative to its famous competitors like spark, storm, kafka streams, flink e.t.c.

Technology should be chosen based on the use case at hand, as a famous saying we can use axe to sharpen pencil as well to cut trees.

When(Why) AKKA streams ?→ Apache spark or storm or flink need a separate cluster to be maintained for streaming. Akka streams don’t need a separate cluster(means a group of nodes chained or offshelf cluster like AWS EMR), it can be run as a java/scala application on any server.

If the transformations are not that complex and if we need functionalities like batching over a period of time, perform transformations on the incoming messages and persist them on kafka or s3 or other storages, then akka streams may just fit your bill.

But if the usecase demands moving windows, water marking and other complex event processing mechanics then flink or spark makes moresense.