[WIP] Language Agnostic Typing in /info route

[freddyaboulton]

Description

Approach

• The /info route now returns language agnostic typing (specified via json-schema) in the type field as opposed to python_type. The json-schemas are stored in python-client/types.json.
• In order to power the View Api page, we also store a python_type field in the /info/ route.
• The clients responsibility is to turn the language agnostic type in the type field into a language-specific type for the client's language. This is the info for serialize=False.
• For python, the data format for the serialize=True case is specified in the serialized_data() method of the serializer class. This is because it's up to the client to decide what data format to accept for the serialized case.
• For custom components, this means devs should use one of the pre-built serializers in their component if they want client users to benefit from a "serialized" data format. Otherwise, the "raw" data should be expected for both the serialize=True and serialize=False cases.

/info route api changes

This is the new data format

{"named_endpoints": {<dependency-name: {
                    "label": <not-changed>,
                    "type": <lang-agnostic-type-info>,
                    "python_type": {
                        "type": <string with the python type>,
                        "description":<description of this python type>,
                    },
                    "component": <not-changed>,
                    "example_input": <not-changed>,
                    "serializer": <name-of-serializer>
              }
       }
}   

Example Output

Stable Diffusion

Client.predict() Usage Info
---------------------------
Named API endpoints: 0

Unnamed API endpoints: 4

 - predict(parameter19, fn_index=0) -> (enter_your_prompt, enter_your_negative_prompt, guidance_scale)
    Parameters:
     - [Dataset] parameter19: str
    Returns:
     - [Textbox] enter_your_prompt: str
     - [Textbox] enter_your_negative_prompt: str
     - [Slider] guidance_scale: Union[int, float]

 - predict(enter_your_prompt, enter_your_negative_prompt, guidance_scale, fn_index=1) -> generated_images
    Parameters:
     - [Textbox] enter_your_prompt: str
     - [Textbox] enter_your_negative_prompt: str
     - [Slider] guidance_scale: Union[int, float]
    Returns:
     - [Gallery] generated_images: str (path to directory with images and a file associating images with captions called captions.json)

 - predict(enter_your_prompt, enter_your_negative_prompt, guidance_scale, fn_index=2) -> generated_images
    Parameters:
     - [Textbox] enter_your_prompt: str
     - [Textbox] enter_your_negative_prompt: str
     - [Slider] guidance_scale: Union[int, float]
    Returns:
     - [Gallery] generated_images: str (path to directory with images and a file associating images with captions called captions.json)

 - predict(enter_your_prompt, enter_your_negative_prompt, guidance_scale, fn_index=3) -> generated_images
    Parameters:
     - [Textbox] enter_your_prompt: str
     - [Textbox] enter_your_negative_prompt: str
     - [Slider] guidance_scale: Union[int, float]
    Returns:
     - [Gallery] generated_images: str (path to directory with images and a file associating images with captions called captions.json)

Checklist:

• /info route returns language agnostic typing
• Python client takes language agnostic types and converts to python types (serialize=False)
• Python client takes language agnostic types and converts to python types (serialize=True)
• JS client takes language agnostic types and coverts to js types
• Install this PR in space fetcher server

Checklist:

• I have performed a self-review of my own code
• I have added a short summary of my change to the CHANGELOG.md
• My code follows the style guidelines of this project
• I have commented my code in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

A note about the CHANGELOG

Hello 👋 and thank you for contributing to Gradio!

All pull requests must update the change log located in CHANGELOG.md, unless the pull request is labeled with the "no-changelog-update" label.

Please add a brief summary of the change to the Upcoming Release > Full Changelog section of the CHANGELOG.md file and include
a link to the PR (formatted in markdown) and a link to your github profile (if you like). For example, "* Added a cool new feature by [@myusername](link-to-your-github-profile) in [PR 11111](https://github.com/gradio-app/gradio/pull/11111)".

If you would like to elaborate on your change further, feel free to include a longer explanation in the other sections.
If you would like an image/gif/video showcasing your feature, it may be best to edit the CHANGELOG file using the
GitHub web UI since that lets you upload files directly via drag-and-drop.

[gradio-pr-bot]

All the demos for this PR have been deployed at https://huggingface.co/spaces/gradio-pr-deploys/pr-4039-all-demos

[freddyaboulton]

Ignore this comment - will delete

[freddyaboulton]

This docstring is out of date

[freddyaboulton]

I think we can get rid of serialized_info

[abidlabs]

Cool I was wondering about that as well. Why some components have it and why others don't

[abidlabs]

Now

Suggested change

	version.parse(self.config.get("version", "2.0")) > version.Version("3.28.1")
	version.parse(self.config.get("version", "2.0")) > version.Version("3.28.3")

[abidlabs]

Type here and below

Suggested change

	serialzier = serializer_class()
	serializer = serializer_class()

[abidlabs]

Should this be:

Suggested change

	if "serialized_info" in info:
	if info["serialized_info"]:

[abidlabs]

To make sure I understand, this basically tests to make sure all of the serializer classes have their info defined in the types.json file, correct?

[freddyaboulton]

Yes!

[abidlabs]

Ah nice thanks for bringing this out. @pngwn will be happy. Maybe we even more it two directories up to the client folder since relevant to all of the client

[abidlabs]

The FileSerializable is a doozy. Thanks for writing it all out!

[freddyaboulton]

There was a bug where this function would return Serializable for Number when the more appropriate value is NumberSerializable.

[abidlabs]

Really awesome PR @freddyaboulton! I tested by connecting to a variety of Gradio apps, both running locally and on Spaces and everything seems to be working fantastic ✅

[abidlabs]

I was testing out the File component after @pngwn's comment, and I don't think the info is correct about the File component here's what I see when I look at the API info in File:

import gradio as gr
from gradio_client import Client

io = gr.Interface(lambda x:x.name, gr.File(file_count="multiple"), "file")
_, url, _ = io.launch()

Client(url)

Client.predict() Usage Info
---------------------------
Named API endpoints: 1

 - predict(x, api_name="/predict") -> output
    Parameters:
     - [File] x: str (filepath or URL to file) 
    Returns:
     - [File] output: str (filepath or URL to file) 

To be clear, the issue is not the /info endpoint, which displays the information correctly, but the conversion of the schema into the python is incorrect.

[freddyaboulton]

Should be fixed now @abidlabs

[pngwn]

Just left one comment to make my life easier, but everything else looks great! Thanks @freddyaboulton

[pngwn]

Could we put the serializer name on the JSON object.

eg:

"serializer": "ListStringSerializable"

[abidlabs]

Nice! Fix works for me @freddyaboulton

[abidlabs]

Looking great! Released gradio_client==0.2.0 and merging this in as discussed