pyspark.sql.functions.
exists
Returns whether a predicate holds for one or more elements in the array.
New in version 3.1.0.
Column
name of column or expression
(x: Column) -> Column: ... returning the Boolean expression. Can use methods of Column, functions defined in pyspark.sql.functions and Scala UserDefinedFunctions. Python UserDefinedFunctions are not supported (SPARK-27052).
(x: Column) -> Column: ...
pyspark.sql.functions
UserDefinedFunctions
Examples
>>> df = spark.createDataFrame([(1, [1, 2, 3, 4]), (2, [3, -1, 0])],("key", "values")) >>> df.select(exists("values", lambda x: x < 0).alias("any_negative")).show() +------------+ |any_negative| +------------+ | false| | true| +------------+