pyspark.sql.functions.
forall
Returns whether a predicate holds for every element in the array.
New in version 3.1.0.
Column
name of column or expression
(x: Column) -> Column: ... returning the Boolean expression. Can use methods of Column, functions defined in pyspark.sql.functions and Scala UserDefinedFunctions. Python UserDefinedFunctions are not supported (SPARK-27052).
(x: Column) -> Column: ...
pyspark.sql.functions
UserDefinedFunctions
Examples
>>> df = spark.createDataFrame( ... [(1, ["bar"]), (2, ["foo", "bar"]), (3, ["foobar", "foo"])], ... ("key", "values") ... ) >>> df.select(forall("values", lambda x: x.rlike("foo")).alias("all_foo")).show() +-------+ |all_foo| +-------+ | false| | false| | true| +-------+