DataFrameReader.csv(path[, schema, sep, …])
DataFrameReader.csv
Loads a CSV file and returns the result as a DataFrame.
DataFrame
DataFrameReader.format(source)
DataFrameReader.format
Specifies the input data source format.
DataFrameReader.jdbc(url, table[, column, …])
DataFrameReader.jdbc
Construct a DataFrame representing the database table named table accessible via JDBC URL url and connection properties.
table
url
properties
DataFrameReader.json(path[, schema, …])
DataFrameReader.json
Loads JSON files and returns the results as a DataFrame.
DataFrameReader.load([path, format, schema])
DataFrameReader.load
Loads data from a data source and returns it as a DataFrame.
DataFrameReader.option(key, value)
DataFrameReader.option
Adds an input option for the underlying data source.
DataFrameReader.options(**options)
DataFrameReader.options
Adds input options for the underlying data source.
DataFrameReader.orc(path[, mergeSchema, …])
DataFrameReader.orc
Loads ORC files, returning the result as a DataFrame.
DataFrameReader.parquet(*paths, **options)
DataFrameReader.parquet
Loads Parquet files, returning the result as a DataFrame.
DataFrameReader.schema(schema)
DataFrameReader.schema
Specifies the input schema.
DataFrameReader.table(tableName)
DataFrameReader.table
Returns the specified table as a DataFrame.
DataFrameReader.text(paths[, wholetext, …])
DataFrameReader.text
Loads text files and returns a DataFrame whose schema starts with a string column named “value”, and followed by partitioned columns if there are any.
DataFrameWriter.bucketBy(numBuckets, col, *cols)
DataFrameWriter.bucketBy
Buckets the output by the given columns.
DataFrameWriter.csv(path[, mode, …])
DataFrameWriter.csv
Saves the content of the DataFrame in CSV format at the specified path.
DataFrameWriter.format(source)
DataFrameWriter.format
Specifies the underlying output data source.
DataFrameWriter.insertInto(tableName[, …])
DataFrameWriter.insertInto
Inserts the content of the DataFrame to the specified table.
DataFrameWriter.jdbc(url, table[, mode, …])
DataFrameWriter.jdbc
Saves the content of the DataFrame to an external database table via JDBC.
DataFrameWriter.json(path[, mode, …])
DataFrameWriter.json
Saves the content of the DataFrame in JSON format (JSON Lines text format or newline-delimited JSON) at the specified path.
DataFrameWriter.mode(saveMode)
DataFrameWriter.mode
Specifies the behavior when data or table already exists.
DataFrameWriter.option(key, value)
DataFrameWriter.option
Adds an output option for the underlying data source.
DataFrameWriter.options(**options)
DataFrameWriter.options
Adds output options for the underlying data source.
DataFrameWriter.orc(path[, mode, …])
DataFrameWriter.orc
Saves the content of the DataFrame in ORC format at the specified path.
DataFrameWriter.parquet(path[, mode, …])
DataFrameWriter.parquet
Saves the content of the DataFrame in Parquet format at the specified path.
DataFrameWriter.partitionBy(*cols)
DataFrameWriter.partitionBy
Partitions the output by the given columns on the file system.
DataFrameWriter.save([path, format, mode, …])
DataFrameWriter.save
Saves the contents of the DataFrame to a data source.
DataFrameWriter.saveAsTable(name[, format, …])
DataFrameWriter.saveAsTable
Saves the content of the DataFrame as the specified table.
DataFrameWriter.sortBy(col, *cols)
DataFrameWriter.sortBy
Sorts the output in each bucket by the given columns on the file system.
DataFrameWriter.text(path[, compression, …])
DataFrameWriter.text
Saves the content of the DataFrame in a text file at the specified path.