[SPARK-59136][CONNECT] Make the active Spark Connect client identifiable (pyspark-client-rust vs the reference client)

[HyukjinKwon]

What changes were proposed in this pull request?

pyspark-client-rust installs the same pyspark API surface as the reference
pyspark-client, backed by the native Rust engine (tonic). Because it is a drop-in
under the pyspark import name, this PR makes it easy to tell which client is in use

• at runtime, on PyPI, and in the docs:

• Runtime markers: pyspark.__rust_client__ (True) and pyspark.__engine__
  ("rust").

• One-time connect log: the first time a session connects, an INFO line is
  emitted on the pyspark logger naming the Rust-backed client and linking the docs
  (best-effort; never fails the connect).

• New docs page "Which client am I using?": pyspark vs pyspark-client vs
  pyspark-client-rust, how to tell them apart, where to file issues, and server
  compatibility - linked from the docs home and the Compatibility page.

• Install/uninstall guidance: because all three packages share the pyspark
  import directory, the Installation docs now spell out that you must uninstall the
  existing client before installing (not on top of it), and that uninstalling
  pyspark-client-rust alone leaves no working pyspark - you must reinstall
  pyspark-client / pyspark to switch back.

• PyPI metadata: clearer description, more keywords, and a Documentation URL.

No change to the API surface or to the behavior of existing Spark Connect code.

Why are the changes needed?

Follow-up from the dev@ discussion on the Python wrapper of the Rust Connect client.
Because the package installs under the pyspark name as a drop-in, it should be
unmistakable which client is active - so behavior and bug reports are attributed to
the right project (apache/spark-connect-rust) rather than the reference client, and
so the two clients' relationship is documented. The install/uninstall guidance
addresses review feedback that installing over an existing pyspark, or later
uninstalling the Rust client, can otherwise leave a broken pyspark. Intended for RC2.

Does this PR introduce any user-facing change?

Yes, additive only: new pyspark.__rust_client__ / pyspark.__engine__ attributes, a
one-time INFO log on first connect, new/updated documentation, and updated PyPI
package metadata. No existing API or result changes.

How was this patch tested?

• cargo check -p pyspark-rs and cargo fmt -p pyspark-rs -- --check pass.
• python -m py_compile on pyspark/__init__.py; verified pyspark.__rust_client__
  / pyspark.__engine__ are present.
• pyproject.toml validated with tomllib.
• Docs built with mkdocs build --strict (mkdocs-material 9.x); the new page, its
  admonitions/tabs, the nav entry, and all internal links render.

This pull request and its description were written by Isaac.

[HyukjinKwon]

@holdenk I believe this clarifies enough. Basically one time log first, and routing people to the correct repo and docs.

[holdenk]

So if someone installs regular pyspark and then the rust client what state will they be in? Similarily if they then uninstall the rust client (say they see the flag) I assume we might leave them with a broken pyspark then? We should probably at least document that uninstalling the rust client is insufficient and they'll need to re-install PySpark.

[HyukjinKwon]

I actually documented that but yeah let me make it a bit more better.

[HyukjinKwon]

This should be ready for a look @holdenk 👍

[holdenk]

LGTM