pyspark.sql.
Column
A column in a DataFrame.
Column instances can be created by:
# 1. Select a column out of a DataFrame df.colName df["colName"] # 2. Create from an expression df.colName + 1 1 / df.colName
New in version 1.3.0.
Methods
alias(*alias, **kwargs)
alias
Returns this column aliased with a new name or names (in the case of expressions that return more than one column, such as explode).
asc()
asc
Returns a sort expression based on ascending order of the column.
asc_nulls_first()
asc_nulls_first
Returns a sort expression based on ascending order of the column, and null values return before non-null values.
asc_nulls_last()
asc_nulls_last
Returns a sort expression based on ascending order of the column, and null values appear after non-null values.
astype(dataType)
astype
astype() is an alias for cast().
astype()
cast()
between(lowerBound, upperBound)
between
True if the current column is between the lower bound and upper bound, inclusive.
bitwiseAND(other)
bitwiseAND
Compute bitwise AND of this expression with another expression.
bitwiseOR(other)
bitwiseOR
Compute bitwise OR of this expression with another expression.
bitwiseXOR(other)
bitwiseXOR
Compute bitwise XOR of this expression with another expression.
cast(dataType)
cast
Casts the column into type dataType.
dataType
contains(other)
contains
Contains the other element.
desc()
desc
Returns a sort expression based on the descending order of the column.
desc_nulls_first()
desc_nulls_first
Returns a sort expression based on the descending order of the column, and null values appear before non-null values.
desc_nulls_last()
desc_nulls_last
Returns a sort expression based on the descending order of the column, and null values appear after non-null values.
dropFields(*fieldNames)
dropFields
An expression that drops fields in StructType by name.
StructType
endswith(other)
endswith
String ends with.
eqNullSafe(other)
eqNullSafe
Equality test that is safe for null values.
getField(name)
getField
An expression that gets a field by name in a StructType.
getItem(key)
getItem
An expression that gets an item at position ordinal out of a list, or gets an item by key out of a dict.
ordinal
ilike(other)
ilike
SQL ILIKE expression (case insensitive LIKE).
isNotNull()
isNotNull
True if the current expression is NOT null.
isNull()
isNull
True if the current expression is null.
isin(*cols)
isin
A boolean expression that is evaluated to true if the value of this expression is contained by the evaluated values of the arguments.
like(other)
like
SQL like expression.
name(*alias, **kwargs)
name
name() is an alias for alias().
name()
alias()
otherwise(value)
otherwise
Evaluates a list of conditions and returns one of multiple possible result expressions.
over(window)
over
Define a windowing column.
rlike(other)
rlike
SQL RLIKE expression (LIKE with Regex).
startswith(other)
startswith
String starts with.
substr(startPos, length)
substr
Return a Column which is a substring of the column.
when(condition, value)
when
withField(fieldName, col)
withField
An expression that adds/replaces a field in StructType by name.