pyspark.sql.functions.
transform_keys
Applies a function to every key-value pair in a map and returns a map with the results of those applications as the new keys for the pairs.
New in version 3.1.0.
Column
name of column or expression
a binary function (k: Column, v: Column) -> Column... Can use methods of Column, functions defined in pyspark.sql.functions and Scala UserDefinedFunctions. Python UserDefinedFunctions are not supported (SPARK-27052).
(k: Column, v: Column) -> Column...
pyspark.sql.functions
UserDefinedFunctions
Examples
>>> df = spark.createDataFrame([(1, {"foo": -2.0, "bar": 2.0})], ("id", "data")) >>> df.select(transform_keys( ... "data", lambda k, _: upper(k)).alias("data_upper") ... ).show(truncate=False) +-------------------------+ |data_upper | +-------------------------+ |{BAR -> 2.0, FOO -> -2.0}| +-------------------------+