DataStreamReader.
parquet
Loads a Parquet file stream, returning the result as a DataFrame.
DataFrame
New in version 2.0.0.
the path in any Hadoop supported file system
For the extra options, refer to Data Source Option. in the version you use.
Examples
>>> parquet_sdf = spark.readStream.schema(sdf_schema).parquet(tempfile.mkdtemp()) >>> parquet_sdf.isStreaming True >>> parquet_sdf.schema == sdf_schema True