Skip to content

[SPARK-52050][DOCS] Refine PySpark profiler user guide for clarity #50850

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
xinrong-meng wants to merge 5 commits into from
Closed

[SPARK-52050][DOCS] Refine PySpark profiler user guide for clarity #50850

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
37c3d2e
impr
xinrong-meng May 9, 2025
77beae6
impr
xinrong-meng May 9, 2025
968e34b
empty commit
xinrong-meng May 12, 2025
29c595a
Update python/docs/source/user_guide/bugbusting.ipynb
xinrong-meng May 13, 2025
141ee89
Update python/docs/source/user_guide/bugbusting.ipynb
xinrong-meng May 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions python/docs/source/user_guide/bugbusting.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -458,7 +458,7 @@
"id": "7f01c836",
"metadata": {},
"source": [
"#### Python and Pandas UDF"
"#### Python/Pandas UDF"
Copy link
Copy Markdown
Member Author

@xinrong-meng xinrong-meng May 9, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be consistent with the section below, see here.

]
},
{
Expand Down Expand Up @@ -532,7 +532,7 @@
"id": "abaf0439-43de-4482-8a80-258be3d98366",
"metadata": {},
"source": [
"The UDF IDs can be seen in the query plan, for example, `add1(...)#16L` in\n",
"The UDF IDs (e.g., 16) can be seen in the query plan, for example, `add1(...)#16L` in\n",
"`ArrowEvalPython` as shown below."
]
},
Expand Down Expand Up @@ -586,7 +586,7 @@
"metadata": {},
"source": [
"[Python Profilers](https://docs.python.org/3/library/profile.html) are useful built-in\n",
"features in Python itself. To use this on driver side, you can use it as you would\n",
"features in Python itself for profiling performance. To use this on driver side, you can use it as you would\n",
"do for regular Python programs because PySpark on driver side is a regular Python\n",
"process unless you are running your driver program in another machine\n",
"(e.g., YARN cluster mode)."
Expand Down Expand Up @@ -731,7 +731,7 @@
"id": "b35fb6c4-c074-4866-abd6-b1b435721b67",
"metadata": {},
"source": [
"The UDF IDs can be seen in the query plan, for example, `add1(...)#22L` in\n",
"The UDF IDs (e.g., 22) can be seen in the query plan, for example, `add1(...)#22L` in\n",
"`ArrowEvalPython` below."
]
},
Expand Down