Spark streaming vs Spark structured streaming

To process continuous streams of data from sources HDFS directories, TCP sockets, Kafka, Flume, Twitter e.t.c spark has two methodologies.

Spark streaming in general works on something called as a micro batch. The stream pipeline is registered with some operations and the Spark polls the source after every batch duration (defined in the application) .

Spark streaming uses DStream api and spark structured streaming uses dataset/dataframe — short answer :-) .