DataFrame.
describe
Computes basic statistics for numeric and string columns.
New in version 1.3.1.
This include count, mean, stddev, min, and max. If no columns are given, this function computes statistics for all numerical or string columns.
See also
DataFrame.summary
Notes
This function is meant for exploratory data analysis, as we make no guarantee about the backward compatibility of the schema of the resulting DataFrame.
DataFrame
Use summary for expanded statistics and control over which statistics to compute.
Examples
>>> df.describe(['age']).show() +-------+------------------+ |summary| age| +-------+------------------+ | count| 2| | mean| 3.5| | stddev|2.1213203435596424| | min| 2| | max| 5| +-------+------------------+ >>> df.describe().show() +-------+------------------+-----+ |summary| age| name| +-------+------------------+-----+ | count| 2| 2| | mean| 3.5| null| | stddev|2.1213203435596424| null| | min| 2|Alice| | max| 5| Bob| +-------+------------------+-----+