Bug: incorrectly validate the rest api raw input data

[liblaf]

• githubkit version: 0.11.13
• pydantic version: 2.9.2

Reproduction Code:

import asyncio

import githubkit
import githubkit.versions.latest.models as m
from icecream import ic


async def main() -> None:
    gh: githubkit.GitHub = githubkit.GitHub(githubkit.ActionAuthStrategy())
    resp = await gh.rest.repos.async_get_release_by_tag(
        "liblaf", "hello-githubkit", "test"
    )
    release: m.Release = resp.parsed_data
    resp = await gh.rest.repos.async_upload_release_asset(
        "liblaf", "hello-githubkit", release.id, name="test.txt", data=b"Hello, world!"
    )
    ic(resp.parsed_data)


if __name__ == "__main__":
    asyncio.run(main())

Error:

Traceback (most recent call last):
  File "**/.venv/lib/python3.12/site-packages/pydantic/type_adapter.py", line 270, in _init_core_attrs
    self._core_schema = _getattr_no_parents(self._type, '__pydantic_core_schema__')
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "**/.venv/lib/python3.12/site-packages/pydantic/type_adapter.py", line 112, in _getattr_no_parents
    raise AttributeError(attribute)
AttributeError: __pydantic_core_schema__

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "**/hello.py", line 21, in <module>
    asyncio.run(main())
  File "/usr/lib/python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "**/hello.py", line 14, in main
    resp = await gh.rest.repos.async_upload_release_asset(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "**/.venv/lib/python3.12/site-packages/githubkit/versions/v2022_11_28/rest/repos.py", line 13894, in async_upload_release_asset
    content = type_validate_python(FileTypes, content)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "**/.venv/lib/python3.12/site-packages/githubkit/compat.py", line 48, in type_validate_python
    return TypeAdapter(type_).validate_python(data)
           ^^^^^^^^^^^^^^^^^^
  File "**/.venv/lib/python3.12/site-packages/pydantic/type_adapter.py", line 257, in __init__
    self._init_core_attrs(rebuild_mocks=False)
  File "**/.venv/lib/python3.12/site-packages/pydantic/type_adapter.py", line 135, in wrapped
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "**/.venv/lib/python3.12/site-packages/pydantic/type_adapter.py", line 277, in _init_core_attrs
    self._core_schema = _get_schema(self._type, config_wrapper, parent_depth=self._parent_depth)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "**/.venv/lib/python3.12/site-packages/pydantic/type_adapter.py", line 95, in _get_schema
    schema = gen.generate_schema(type_)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "**/.venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 655, in generate_schema
    schema = self._generate_schema_inner(obj)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "**/.venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 929, in _generate_schema_inner
    return self.match_type(obj)
           ^^^^^^^^^^^^^^^^^^^^
  File "**/.venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 1029, in match_type
    return self._match_generic_type(obj, origin)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "**/.venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 1058, in _match_generic_type
    return self._union_schema(obj)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "**/.venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 1378, in _union_schema
    choices.append(self.generate_schema(arg))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "**/.venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 655, in generate_schema
    schema = self._generate_schema_inner(obj)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "**/.venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 929, in _generate_schema_inner
    return self.match_type(obj)
           ^^^^^^^^^^^^^^^^^^^^
  File "**/.venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 1029, in match_type
    return self._match_generic_type(obj, origin)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "**/.venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 1087, in _match_generic_type
    return self._unknown_type_schema(obj)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "**/.venv/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 558, in _unknown_type_schema
    raise PydanticSchemaGenerationError(
pydantic.errors.PydanticSchemaGenerationError: Unable to generate pydantic-core schema for typing.IO[bytes]. Set `arbitrary_types_allowed=True` in the model_config to ignore this error or implement `__get_pydantic_core_schema__` on your type to fully support it.

If you got this error by calling handler(<some type>) within `__get_pydantic_core_schema__` then you likely need to call `handler.generate_schema(<some type>)` since we do not call `__get_pydantic_core_schema__` on `<some type>` otherwise to avoid infinite recursion.

For further information visit https://errors.pydantic.dev/2.9/u/schema-for-unknown-type

[yanyongyu]

It seems the raw content is passed to pydantic validation. Input data validation is only needed when request body is application/json. I will fix this in the codegen.

[yanyongyu]

The master branch fixed this error. You may test it by installing the master version.

[liblaf]

The master branch fixed this error. You may test it by installing the master version.

@yanyongyu The validation error has been resolved. However, async_upload_release_asset() still throws a 404 Not Found error. This occurs because the base_url for uploading release assets is https://uploads.github.com, which differs from https://api.github.com, the base URL used for the standard REST API.

For reference, see the Upload a release asset documentation. The discrepancy in base_url is not clearly stated in the documentation, so you need to examine the request example using cURL to understand this difference.

The PyGithub library addresses this issue by utilizing the upload_url from the release object (see the relevant code here). I'm curious if there is an elegant way to handle this in githubkit, as githubkit does not currently provide a configuration option for upload_url.

I believe this should be addressed as a separate issue. Should I open a new issue for tracking purposes?

[yanyongyu]

This is because the base_url for uploading release assets is https://uploads.github.com/

githubkit currently does not support specify base_url for single api request. This will make maintenance difficult.

As described in the github docs,

Use the upload_url returned in the response of the Create a release endpoint to upload a release asset.

You can call the url directly by using github.arequest.

release = xxx  # get release first

resp = await g.arequest(
    "POST",
    release.upload_url.split("{?")[0],
    content=b"xxx",
    headers={"Content-Type": "xxx"}
)

Otherwise, you can copy a github client with base_url modified.

config = g.config.copy()
config.base_url = "https://uploads.github.com/"
upload_g = GitHub(g.auth, config=config)

[yanyongyu]

The PyGithub library addresses this issue by utilizing the upload_url from the release object (see the relevant code here).

Different from PyGithub, githubkit is API oriented, not object oriented. So, githubkit will not extract the upload url from release object. You may need to extract it by yourself, as described above.

Also found another example, from octokit: https://github.com/octokit/rest.js/blob/94443df1a616182af063a08e2fabe6af0c6f6b0b/test/scenarios/release-assets.test.ts#L19-L39

[liblaf]

@yanyongyu thx! I will use arequest() instead.