[SPARK-46282][PYTHON][DOCS] Rename "Spark SQL" page to "DataFrame API"

[itholic]

What changes were proposed in this pull request?

This PR proposes to rename "Spark SQL" page to "DataFrame API".

This includes updating the title and any relevant references within the documentation to better highlight the centrality of the DataFrame API in PySpark.

This also includes adding SQL Reference related descriptions and links to the pyspark.sql.SparkSession.sql API to make it easier for users to use SQL supported by Spark.

Why are the changes needed?

The change is needed to more accurately represent the content of the page, which primarily focuses on the DataFrame API – a key aspect of PySpark. Renaming the page to "DataFrame API" emphasizes its importance and utility in the PySpark ecosystem, making the documentation more intuitive and aligned with the common usage and understanding of PySpark's core functionalities.

Does this PR introduce any user-facing change?

No API changes, but user-facing documentation will be updated:

Main

API Reference

SparkSession.sql

How was this patch tested?

Manually build docs.

Was this patch authored or co-authored using generative AI tooling?

No.

[itholic]

Let me close this ticket for now as it seems to be need further discussion.