Skip to content

Commit

Permalink
Update dataframe.py
Browse files Browse the repository at this point in the history
removed space from empty lines
  • Loading branch information
chanansh committed Dec 31, 2018
1 parent def5b2c commit b370363
Showing 1 changed file with 6 additions and 7 deletions.
13 changes: 6 additions & 7 deletions python/pyspark/sql/dataframe.py
Expand Up @@ -2050,18 +2050,18 @@ def toDF(self, *cols):
def transform(self, func, *args, **kwargs):
"""Returns a new class:`DataFrame` according to a user-defined custom transform method.
This allows chaining transformations rather than using nested or temporary variables.
:param func: a user-defined custom transform function
:param *args: optional positional arguments to pass to `func`
:param **kwargs: optional keywarded arguments to pass to `func`
A more concrete example::
def with_greeting(df):
def with_greeting(df):
return df.withColumn("greeting", lit("hi"))
def with_something(df, something):
return df.withColumn("something", lit(something))
data = [("jose", 1), ("li", 2), ("liz", 3)]
source_df = spark.createDataFrame(data, ["name", "age"])
Expand All @@ -2076,15 +2076,14 @@ def with_something(df, something):
| li| 2| hi| crazy|
| liz| 3| hi| crazy|
+----+---+--------+---------+
This is equiavalent to a nested:
actual_df = with_something(with_greeting(source_df), "crazy"))
credit to: https://medium.com/@mrpowers/chaining-custom-pyspark-transformations-4f38a8c7ae55
"""
return func(self, *args, **kwargs)

@since(1.3)
def toPandas(self):
"""
Expand Down

0 comments on commit b370363

Please sign in to comment.