feat: generalize storage_options for GCS/Azure and any object_store backend

[dcfocus]

Summary

Closes #45.

The Rust + PyO3 layer already forwards an arbitrary storage_options dict through to lance's DatasetBuilder::with_storage_options, but the Python constructor only exposed ergonomic shortcuts for AWS keys, which made it look like GCS/Azure weren't supported and blocked tiered-memory work on gs:// URIs.

This PR:

• Promotes storage_options as the canonical, backend-agnostic way to configure remote stores — aligned with how lance and lance-graph handle backends.
• Keeps aws_access_key_id, aws_secret_access_key, aws_session_token, region, endpoint_url, allow_http as backwards-compatible shortcuts that now emit a single DeprecationWarning pointing callers at storage_options.
• Documents S3, GCS, and Azure usage in the Python docstring and the README.
• Adds python/tests/test_storage_options.py with 8 unit tests covering merge / pass-through / precedence / deprecation semantics.
• Adds python/tests/test_gcs_persistence.py, an opt-in real-GCS integration test gated on LANCE_CONTEXT_GCS_BUCKET + one of LANCE_CONTEXT_GCS_SERVICE_ACCOUNT_KEY / GOOGLE_APPLICATION_CREDENTIALS / LANCE_CONTEXT_GCS_ENDPOINT (for `fake-gcs-server`-style emulators).
• Rewrites the existing S3 integration test to use the canonical `storage_options=` path and adds a companion back-compat test that asserts the deprecated AWS kwargs still work and emit the warning.
• Fixes the `moto.server` subprocess invocation for moto >= 5 (dropped the positional service arg) and adds `moto[s3,server]` so `flask` / `flask-cors` are available; without this the S3 suite was being silently skipped on modern environments.

No behavior change for local or S3 callers that already pass `storage_options=`.

Notes

An emulator-based GCS test was explored with `gcp-storage-emulator` but has two compat issues outside this repo's scope: (1) it imports `fs` (pyfilesystem2) which calls `pkg_resources.declare_namespace`, broken under setuptools >= 81 (Python 3.12+); (2) its JSON responses fail OpenDAL's deserializer with `invalid type: null, expected a string`. The opt-in integration test works against `fake-gcs-server` (Go binary) and real GCS, which is what the UP+GCS tiered-memory rollout will actually use.

Test plan

• `cargo fmt --all -- --check`
• `cargo clippy --all-targets -- -D warnings`
• `cargo test --workspace` (6 passed)
• `uv run pytest` (39 passed, 2 skipped — PIL image test + opt-in GCS)
• `uv run ruff format --check python/`
• `uv run ruff check python/`
• `uv run pyright` (0 errors)
• Run `test_gcs_persistence.py` against real GCS with `LANCE_CONTEXT_GCS_BUCKET=... GOOGLE_APPLICATION_CREDENTIALS=...` from the UP+GCS environment before/after merge to confirm end-to-end for the tiered-memory work

Made with Cursor