New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-46767][PYTHON][DOCS] Refine docstring of abs/acos/acosh
#44794
Conversation
+------------------+ | ||
| ACOS(value)| | ||
+------------------+ | ||
|1.0471975511965979| |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
for trigonometric functions' example, what about making them use or return some special values? e.g.
In [26]: df = spark.createDataFrame([(-1.0, 0.5), (-0.5, -0.5), (0.0, 1.0), (0.5, 0.0), (1.0, 1.0)], ["v1", "v2"])
In [27]: df.select("v1", sf.acos("v1"), sf.degrees(sf.acos("v1"))).show()
+----+------------------+------------------+
| v1| ACOS(v1)| DEGREES(ACOS(v1))|
+----+------------------+------------------+
|-1.0| 3.141592653589...| 180.0|
|-0.5|2.0943951023931...|120.00000000000...|
| 0.0|1.5707963267948...| 90.0|
| 0.5|1.0471975511965...| 60.00000000000...|
| 1.0| 0.0| 0.0|
+----+------------------+------------------+
and we'd better use ...
at the last of numbers, since the computation result may vary slightly on different envs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK
>>> df = spark.createDataFrame([(-1.0,), (-0.5,), (0.0,), (0.5,), (1.0,)], ["value"]) | ||
>>> df.select("value", | ||
... sf.substring(sf.acos("value"), 0, 15).alias("ACOS(value)"), | ||
... sf.substring(sf.degrees(sf.acos("value")), 0, 5).alias("DEGREES(ACOS(value))") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks, this looks pretty good.
btw, I guess we can use sf.round
to make the result more stable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
or let's simply remove the degrees column
+-----+---------------+--------------------+ | ||
| -1.0|3.1415926535897| 180.0| | ||
| -0.5|2.0943951023931| 120.0| | ||
| 0.0|1.5707963267948| 90.0| |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| 0.0|1.5707963267948| 90.0| | |
| 0.0|1.5707963267...| 90.0| |
and other similar places.
To make the result more stable: e.g. the doctest won't fail when we change jdk version/python version/underlying engineer/etc.
Merged into master. Thanks @zhengruifeng |
What changes were proposed in this pull request?
This pr refine docstring of
abs/acos/acosh
and add some new examples.Why are the changes needed?
To improve PySpark documentation
Does this PR introduce any user-facing change?
No
How was this patch tested?
Pass Github Actions
Was this patch authored or co-authored using generative AI tooling?
No