Inscrivez-vous ou connectez-vous pour rejoindre votre communauté professionnelle.
Hi Partha,
A Spark Streaming Context is the entry point of streaming program. Spark Streaming Context can be created only after Spark Configuration or SparkConf is created. SparkConf provides Configuration for a Spark application. Used to set various Spark parameters as key-value pairs. A Streaming Context object can be created from a SparkConf object.
We have to import spark.streaming package for the same.
An example:
import org.apache.spark._
import org.apache.spark.streaming._
val conf = new SparkConf().setAppName(appName).setMaster(master) val streamcontent = new StreamingContext(conf, Seconds())The appName parameter is a name for your application to show on the cluster UI and master is a Spark, Mesos or YARN cluster URL, or a special “local[*]” string to run in local mode.
Seconds() is the timeframe( seconds in this example) for which data stream is ingested from source.
A StreamingContext object can also be created from an existing SparkContext object.
An example:
import org.apache.spark.streaming._
import org.apache.spark.streaming._
val conf = new SparkConf().setAppName(appName).setMaster(master) val streamcontent = new StreamingContext(conf, Seconds(1)) val newstreamcontent = new StreamingContext(streamcontent,Seconds())Hope this will help.
