[#2230] feat(client-python): Add doc and support publishing Gravitino Python client to Pypi #2932

SophieTech88 · 2024-04-14T02:16:27Z

What changes were proposed in this pull request?

We published the first version of python-client package of Gravitino to Pypi.

Add how to use Python Client document
add some tools to requirements.txt
./gradlew :clients:client-python:distribution
./gradlew :clients:client-python:deploy

Why are the changes needed?

We need the python client package for user to use Gravitino in Python project.

#2230

Does this PR introduce any user-facing change?

Users can use pip install gravitino to install the Gravitino python-client package directly.

How was this patch tested?

CI Passed

xunliu · 2024-04-22T16:56:40Z

I will improve this PR.

SophieTech88 · 2024-04-28T03:55:09Z

I will improve this PR.

Many thanks.

xunliu · 2024-05-12T14:54:59Z

hi @SophieTech88 I help you improvement this PR,
@SophieTech88 @coolderli @noidname01 @zhaoyongjie Please help me review this PR, Thanks!

noidname01 · 2024-05-13T11:40:55Z

docs/how-to-use-python-client.md

+5. Distribute the Gravitino Python client module
+
+    ```shell
+    ./gradlew :clients:client-python:distribute


There is no distribute task in build.gradle.kts, only distribution task.

OK, I fixed it.

noidname01 · 2024-05-13T12:32:10Z

docs/how-to-use-the-playground.md

@@ -8,7 +8,7 @@ This software is licensed under the Apache License version 2."

 ## Playground introduction

-The playground is a complete Gravitino Docker runtime environment with `Hive`, `HDFS`, `Trino`, `MySQL`, `PostgreSQL`, and a `Gravitino` server.
+The playground is a complete Gravitino Docker runtime environment with `Hive`, `HDFS`, `Trino`, `MySQL`, `PostgreSQL`, `Jupter`, and a `Gravitino` server.


I think it's Jupyter?

OK, I fixed it.

Publish the first version of Gravitino Pyhton client to Pypi Ticket: 2230

…ient to Pypi

coolderli · 2024-05-14T03:38:36Z

docs/how-to-use-the-playground.md

+```
+
+### Experiencing Gravitino Fileset with Jupyter
+We proved a Fileset playground environment to help you quickly understand how to use Gravitino Python client to manage HDFS in Gravitino service.


proved -> provided?

coolderli · 2024-05-14T03:40:57Z

docs/how-to-use-the-playground.md

+3. Install Gravitino Python client
+4. Initialize Gravitino admin client and create a Gravitino metalake
+5. Initialize Gravitino client and list metalakes
+6. Create a Gravitino `Catalog` and special `type` is `Catalog.Type.FILESET` and `proved` is [hadoop](https://datastrato.ai/docs/latest/hadoop-catalog)


proved -> provider

Let me rewrite this doc.

coolderli · 2024-05-14T03:51:29Z

clients/client-python/gravitino/catalog/fileset_catalog.py

@@ -200,15 +200,14 @@ def format_fileset_request_path(namespace: Namespace) -> str:
    def to_fileset_update_request(change: FilesetChange):
        if isinstance(change, FilesetChange.RenameFileset):
            return FilesetUpdateRequest.RenameFilesetRequest(change.new_name())
-        elif isinstance(change, FilesetChange.UpdateFilesetComment):
+        if isinstance(change, FilesetChange.UpdateFilesetComment):


Why do we change the elif to if? Does that mean we have to continue the remaining if judgment even though the first if is matched?

logically it is the same because of the return, right? Let me check if it is the code style of python.

Yeah, actually it's no-else-return in Pylint, it's not necessary to use elif in this case.

This is suggested by pylint, the no-else-return rule
https://pylint.pycqa.org/en/latest/user_guide/messages/refactor/no-else-return.html

jerryshao · 2024-05-14T06:00:26Z

clients/client-python/setup.py

 setup(
    name="gravitino",
    description="Python lib/client for Gravitino",
-    version="0.5.0",
-    long_description=open("README.md").read(),
+    version="0.5.0.dev12",


How do we define this version 0.5.0.dev12?

Here covers all versioning Python packages.
https://packaging.python.org/en/latest/discussions/versioning/

The version number need to be changed each commit Pypi.org.

jerryshao · 2024-05-14T06:09:53Z

clients/client-python/setup.py

-    packages=find_packages(include=["gravitino", ".*"]),
+    packages=find_packages(exclude=["tests*"]),
+    classifiers=[
+        'Development Status :: 3 - Alpha',


What is the meaning of 3 - Alpha, does it have special purpose?

This is Pypi.org's development status classify, https://pypi.org/search/?c=Development+Status+%3A%3A+3+-+Alpha

jerryshao · 2024-05-14T06:16:53Z

docs/how-to-use-python-client.md

+First of all, You must have a Gravitino Service, You can refer document of [How to install Gravitino](https://datastrato.ai/docs/latest/how-to-install)
+to build Gravitino service from source code and install it in your local.
+
+### Gravitino Python client API


You'd better add a chapter/paragraph here about how to install or use python client, like pip install or something else.

jerryshao · 2024-05-14T07:27:36Z

@xunliu please review again, I did some changes in doc.

xunliu · 2024-05-14T10:20:24Z

@xunliu please review again, I did some changes in doc.

The doc context very good!

But If use relative path in the doc, We need convert url to absolute path,

Because when deploy Python client to Pypi.org, Gradle script will copy how-to-use-python-client.md to clients-client-python/README.md and Pypi.org will use this file generation home web page.
otherwise Pypi.org don't support upload image file, so we must link external image.

So I add a commit to support convert url.

@jerryshao Please help me review this PR, Thanks.

jerryshao · 2024-05-14T14:19:25Z

Thanks @SophieTech88 @xunliu for your contributions. I'm going to merge this.

… Python client to Pypi (#2932) ### What changes were proposed in this pull request? We published the first version of python-client package of Gravitino to Pypi. 1. Add how to use Python Client document 2. add some tools to requirements.txt 3. `./gradlew :clients:client-python:distribution` 4. `./gradlew :clients:client-python:deploy` ### Why are the changes needed? We need the python client package for user to use Gravitino in Python project. #2230 ### Does this PR introduce _any_ user-facing change? Users can use `pip install gravitino` to install the Gravitino python-client package directly. ### How was this patch tested? CI Passed --------- Co-authored-by: xunliu <xun@datastrato.com> Co-authored-by: Jerry Shao <jerryshao@datastrato.com>

…vitino Python client to Pypi (apache#2932) ### What changes were proposed in this pull request? We published the first version of python-client package of Gravitino to Pypi. 1. Add how to use Python Client document 2. add some tools to requirements.txt 3. `./gradlew :clients:client-python:distribution` 4. `./gradlew :clients:client-python:deploy` ### Why are the changes needed? We need the python client package for user to use Gravitino in Python project. apache#2230 ### Does this PR introduce _any_ user-facing change? Users can use `pip install gravitino` to install the Gravitino python-client package directly. ### How was this patch tested? CI Passed --------- Co-authored-by: xunliu <xun@datastrato.com> Co-authored-by: Jerry Shao <jerryshao@datastrato.com>

qqqttt123 changed the title ~~[#2230]Publish the first version of Gravitino Pyhton client to Pypi~~ [#2230] build: Publish the first version of Gravitino Pyhton client to Pypi Apr 14, 2024

xunliu mentioned this pull request Apr 22, 2024

[#2929] improvement(pyClient): Optimize Python client code and add integration test #2930

Merged

mchades assigned xunliu and SophieTech88 Apr 26, 2024

mchades requested a review from xunliu April 26, 2024 07:38

xunliu mentioned this pull request May 6, 2024

[#36] Add Jupyter container into playground apache/gravitino-playground#37

Merged

xunliu added the 0.5.1 Release v0.5.1 label May 6, 2024

xunliu added this to the Gravitino June Release milestone May 6, 2024

xunliu force-pushed the sophies-2230 branch from 04f5d5f to a2c7c17 Compare May 10, 2024 13:01

jerryshao added branch-0.5 and removed 0.5.1 Release v0.5.1 labels May 11, 2024

xunliu force-pushed the sophies-2230 branch 2 times, most recently from bc039e3 to 11b01c1 Compare May 12, 2024 14:18

xunliu changed the title ~~[#2230] build: Publish the first version of Gravitino Pyhton client to Pypi~~ [#2230] feat(PyClient): Publish Gravitino Python client to Pypi May 12, 2024

noidname01 reviewed May 13, 2024

View reviewed changes

SophieTech88 and others added 4 commits May 14, 2024 11:08

Publish the first version of Gravitino Pyhton client to Pypi

ee0c6b6

Publish the first version of Gravitino Pyhton client to Pypi Ticket: 2230

[apache#2230] build: Publish the first version of Gravitino Pyhton cl…

f62c95e

…ient to Pypi

update Setup.py

f4bbcd0

update doc

938e6a8

xunliu force-pushed the sophies-2230 branch from b7c6954 to 938e6a8 Compare May 14, 2024 03:08

xunliu and others added 2 commits May 14, 2024 11:19

snake_case naming style

03940ea

Merge branch 'main' into sophies-2230

4e99c78

coolderli reviewed May 14, 2024

View reviewed changes

typo

40f07b1

jerryshao reviewed May 14, 2024

View reviewed changes

xunliu and others added 3 commits May 14, 2024 14:24

pip

0b5fa99

Improve the docs

7b5824a

Improve the docs

f40b24f

xunliu added 2 commits May 14, 2024 17:52

convert url

e7351c4

delete README.md of client-python

c745b17

delete README.md

ccc80ca

jerryshao changed the title ~~[#2230] feat(PyClient): Publish Gravitino Python client to Pypi~~ [#2230] feat(client-python): Add doc and support publishing Gravitino Python client to Pypi May 14, 2024

jerryshao approved these changes May 14, 2024

View reviewed changes

jerryshao merged commit 12c32c7 into apache:main May 14, 2024
9 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[#2230] feat(client-python): Add doc and support publishing Gravitino Python client to Pypi #2932

[#2230] feat(client-python): Add doc and support publishing Gravitino Python client to Pypi #2932

SophieTech88 commented Apr 14, 2024 •

edited by xunliu

Loading

xunliu commented Apr 22, 2024

SophieTech88 commented Apr 28, 2024

xunliu commented May 12, 2024 •

edited

Loading

noidname01 May 13, 2024

xunliu May 14, 2024

noidname01 May 13, 2024

xunliu May 14, 2024

coolderli May 14, 2024

xunliu May 14, 2024

coolderli May 14, 2024

xunliu May 14, 2024

jerryshao May 14, 2024

coolderli May 14, 2024

jerryshao May 14, 2024

noidname01 May 14, 2024

xunliu May 14, 2024

coolderli May 14, 2024

jerryshao May 14, 2024

xunliu May 14, 2024

jerryshao May 14, 2024

xunliu May 14, 2024

jerryshao May 14, 2024

xunliu May 14, 2024

jerryshao commented May 14, 2024

xunliu commented May 14, 2024

jerryshao commented May 14, 2024

[#2230] feat(client-python): Add doc and support publishing Gravitino Python client to Pypi #2932

[#2230] feat(client-python): Add doc and support publishing Gravitino Python client to Pypi #2932

Conversation

SophieTech88 commented Apr 14, 2024 • edited by xunliu Loading

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

xunliu commented Apr 22, 2024

SophieTech88 commented Apr 28, 2024

xunliu commented May 12, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jerryshao commented May 14, 2024

xunliu commented May 14, 2024

jerryshao commented May 14, 2024

SophieTech88 commented Apr 14, 2024 •

edited by xunliu

Loading

xunliu commented May 12, 2024 •

edited

Loading