DataStreamReader.csv(path[, schema, sep, …])
DataStreamReader.csv
Loads a CSV file stream and returns the result as a DataFrame.
DataFrame
DataStreamReader.format(source)
DataStreamReader.format
Specifies the input data source format.
DataStreamReader.json(path[, schema, …])
DataStreamReader.json
Loads a JSON file stream and returns the results as a DataFrame.
DataStreamReader.load([path, format, schema])
DataStreamReader.load
Loads a data stream from a data source and returns it as a DataFrame.
DataStreamReader.option(key, value)
DataStreamReader.option
Adds an input option for the underlying data source.
DataStreamReader.options(**options)
DataStreamReader.options
Adds input options for the underlying data source.
DataStreamReader.orc(path[, mergeSchema, …])
DataStreamReader.orc
Loads a ORC file stream, returning the result as a DataFrame.
DataStreamReader.parquet(path[, …])
DataStreamReader.parquet
Loads a Parquet file stream, returning the result as a DataFrame.
DataStreamReader.schema(schema)
DataStreamReader.schema
Specifies the input schema.
DataStreamReader.table(tableName)
DataStreamReader.table
Define a Streaming DataFrame on a Table.
DataStreamReader.text(path[, wholetext, …])
DataStreamReader.text
Loads a text file stream and returns a DataFrame whose schema starts with a string column named “value”, and followed by partitioned columns if there are any.
DataStreamWriter.foreach(f)
DataStreamWriter.foreach
Sets the output of the streaming query to be processed using the provided writer f.
f
DataStreamWriter.foreachBatch(func)
DataStreamWriter.foreachBatch
Sets the output of the streaming query to be processed using the provided function.
DataStreamWriter.format(source)
DataStreamWriter.format
Specifies the underlying output data source.
DataStreamWriter.option(key, value)
DataStreamWriter.option
Adds an output option for the underlying data source.
DataStreamWriter.options(**options)
DataStreamWriter.options
Adds output options for the underlying data source.
DataStreamWriter.outputMode(outputMode)
DataStreamWriter.outputMode
Specifies how data of a streaming DataFrame/Dataset is written to a streaming sink.
DataStreamWriter.partitionBy(*cols)
DataStreamWriter.partitionBy
Partitions the output by the given columns on the file system.
DataStreamWriter.queryName(queryName)
DataStreamWriter.queryName
Specifies the name of the StreamingQuery that can be started with start().
StreamingQuery
start()
DataStreamWriter.start([path, format, …])
DataStreamWriter.start
Streams the contents of the DataFrame to a data source.
DataStreamWriter.toTable(tableName[, …])
DataStreamWriter.toTable
Starts the execution of the streaming query, which will continually output results to the given table as new data arrives.
DataStreamWriter.trigger(*[, …])
DataStreamWriter.trigger
Set the trigger for the stream query.