Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BatchPrediction 400: Invalid JSON payload received. Unknown name \"type_\" at 'generation_config.response_schema #4333

Open
orenmiaraswish opened this issue Sep 2, 2024 · 6 comments
Open

BatchPrediction 400: Invalid JSON payload received. Unknown name \"type_\" at 'generation_config.response_schema #4333

orenmiaraswish opened this issue Sep 2, 2024 · 6 comments
Assignees
@jaycee-li
Labels
api: vertex-ai Issues related to the googleapis/python-aiplatform API.

Comments

@orenmiaraswish
Copy link

orenmiaraswish commented Sep 2, 2024 •
edited
Loading

Problem:

400 Error message With Vertex BatchPrediction API with gemini-1.5-flash-001 and gemini-1.5-pro-001

I am getting similar issues using Batch Prediction with gemini-1.5-flash-001 and gemini-1.5-pro-001.
When not using Vertex BatchPrediction API the code at the (Expected output) works.

image

followed the documentation here:

Environment details

  • Windows 10 Pro 22H2
  • python 3.11.8 / pip 24.2
  • google-cloud-aiplatform==1.64.0
  • google-cloud-bigquery==3.18.0
  • google-cloud-core==2.4.1
  • google-cloud-kms==2.21.1
  • google-api-core==2.19.1
  • google-resumable-media==2.7.2
  • googleapis-common-protos==1.63.2
  • Google Cloud SDK 482.0.0 (beta 2024.06.24, core 2024.06.24, gcloud-crc32c 1.0.0, gke-gcloud-auth-plugin 0.5.8 gsutil 5.30)
  • LOCATION : europe-west4

Steps to reproduce:

  1. Use Vertex BatchPrediction API (europe-west4)
  2. Use BigQuery
  3. Run the Job with Input payload as below (Taken from BigQuery Input):
{
    "contents":
    [
        {
            "role": "user",
            "parts":
            {
                "text": "TEXT GOES HERE"
            }
        }
    ],
    "system_instruction":
    {
        "parts":
        [
            {
                "text": "MY SYSTEM PROMPT"
            }
        ]
    },
    "safety_settings":
    [
        {
            "category": 1,
            "threshold": 4
        },
        {
            "category": 3,
            "threshold": 4
        },
        {
            "category": 4,
            "threshold": 4
        },
        {
            "category": 2,
            "threshold": 4
        },
        {
            "category": 0,
            "threshold": 4
        }
    ],
    "generation_config":
    {
        "temperature": 0.1,
        "response_mime_type": "application/json",
        "response_schema":
        {
            "type_": "OBJECT",
            "properties":
            {
                "description":
                {
                    "type_": "STRING"
                },
                "steps":
                {
                    "type_": "BOOLEAN"
                },
                "title":
                {
                    "type_": "STRING"
                },
                "questions":
                {
                    "type_": "ARRAY",
                    "items":
                    {
                        "type_": "STRING"
                    }
                }
            },
            "required":
            [
                "title",
                "steps",
                "description",
                "questions"
            ]
        }
    }
}

Code example

generation_config = GenerationConfig(
    temperature=0.1,
    response_mime_type="application/json",
    response_schema={
        "type": "object",
        "properties": {
            "title": {"type": "string"},
            "steps": {"type": "boolean"},
            "description": {"type": "string"},
            "questions": {"type": "array", "items": {"type": "string"}}
        },
        "required": [
            "title",
            "steps",
            "description",
            "questions"
        ]
    }
)
...

Stack trace: (BigQuery - Output)

{
    "error":
    {
        "code": 400,
        "message": "Invalid JSON payload received. Unknown name \"type_\" at 'generation_config.response_schema.properties[0].value': Cannot find field.\nInvalid JSON payload received. Unknown name \"type_\" at 'generation_config.response_schema.properties[1].value.items': Cannot find field.\nInvalid JSON payload received. Unknown name \"type_\" at 'generation_config.response_schema.properties[1].value': Cannot find field.\nInvalid JSON payload received. Unknown name \"type_\" at 'generation_config.response_schema.properties[2].value': Cannot find field.\nInvalid JSON payload received. Unknown name \"type_\" at 'generation_config.response_schema.properties[3].value': Cannot find field.\nInvalid JSON payload received. Unknown name \"type_\" at 'generation_config.response_schema': Cannot find field.",
        "status": "INVALID_ARGUMENT",
        "details":
        [
            {
                "@type": "type.googleapis.com/google.rpc.BadRequest",
                "fieldViolations":
                [
                    {
                        "field": "generation_config.response_schema.properties[0].value",
                        "description": "Invalid JSON payload received. Unknown name \"type_\" at 'generation_config.response_schema.properties[0].value': Cannot find field."
                    },
                    {
                        "field": "generation_config.response_schema.properties[1].value.items",
                        "description": "Invalid JSON payload received. Unknown name \"type_\" at 'generation_config.response_schema.properties[1].value.items': Cannot find field."
                    },
                    {
                        "field": "generation_config.response_schema.properties[1].value",
                        "description": "Invalid JSON payload received. Unknown name \"type_\" at 'generation_config.response_schema.properties[1].value': Cannot find field."
                    },
                    {
                        "field": "generation_config.response_schema.properties[2].value",
                        "description": "Invalid JSON payload received. Unknown name \"type_\" at 'generation_config.response_schema.properties[2].value': Cannot find field."
                    },
                    {
                        "field": "generation_config.response_schema.properties[3].value",
                        "description": "Invalid JSON payload received. Unknown name \"type_\" at 'generation_config.response_schema.properties[3].value': Cannot find field."
                    },
                    {
                        "field": "generation_config.response_schema",
                        "description": "Invalid JSON payload received. Unknown name \"type_\" at 'generation_config.response_schema': Cannot find field."
                    }
                ]
            }
        ]
    }
}

Expected Results Example:

BigQuery Output table Response column filled with the adequate JSON output.

model = GenerativeModel(
    model_name='gemini-1.5-pro-001',
    generation_config=GenerationConfig(
        response_mime_type="application/json",
        response_schema={
            "type": "object",
            "properties": {
                "title": {"type": "STRING"},
                "steps": {"type": "BOOLEAN"},
                "description": {"type": "STRING"},
                "questions": {"type": "ARRAY", "items": {"type": "STRING"}}
            },
            "required": [
                "title",
                "steps",
                "description",
                "questions",
            ],
        }
    )
)

response = await model.generate_content_async("Is this sentce possitive negative or neutral? I am angry and happier")
{
    "description": "This sentence expresses conflicting emotions, making it difficult to categorize as strictly positive, negative, or neutral. It suggests a mix of anger and happiness, which could be interpreted as a complex emotional state.",
    "questions":
    [
        "Is the sentence expressing a positive, negative, or neutral sentiment?"
    ],
    "steps": false,
    "title": "Analyzing Sentence Sentiment"
}
@product-auto-label product-auto-label bot added the api: vertex-ai Issues related to the googleapis/python-aiplatform API. label Sep 2, 2024
@Ark-kun
Copy link
Contributor

Ark-kun commented Sep 3, 2024

@jaycee-li Can you check whether the schema proto is correctly serialized to JSON via json_format?

Or could this be caused by bugs in the code that handles proto fields renamed by the GAPIC generator?

@jaycee-li
Copy link
Contributor

Hi @orenmiaraswish,

It seems like the issue lies in the format of your Bigquery input. Bigquery expects requests in JSON format, whereas your input is currently a raw proto message.

The key difference is that Schema proto uses fields type_ and format_, while its JSON counterpart uses type and format.

To resolve your issue, you'll need to modify your Bigquery input by changing type_ to type in response_shcema.

@orenmiaraswish
Copy link
Author

@jaycee-li thanks for your replay.

The BigQuery Input is actually a JSON, though it was originated from GenerationConfig.to_dict() thus contains "type_"...
I overcome this issue by using plain JSON, to not going through the gapic validation stuff.

Original Code:

...
@staticmethod
def raw_to_json(request_id, prompt, instructions, generation_config):
    if isinstance(generation_config, GenerationConfig):
        generation_config = generation_config.to_dict()
    request = {
        "contents": [{"role": "user", "parts": {"text": prompt}}],
        "system_instruction": {"parts": [{"text": f"{instructions}"}]},
        "safety_settings": batch_request_safety_settings,
        "generation_config": generation_config,
    }
    return {"id": request_id, "request": json.dumps(request)}
...

@ItsOkayItsOfficial
Copy link

I think we are getting a variant of the same response when using the Python SDK.

Using the Vertex AI Studio with Output format JSON enabled does produced the desired response.

The Get code section provides incorrect code (below).

image

However after correcting the proffered code our generation_config looks like this:

generation_config = {
    "max_output_tokens": 8192,
    "temperature": 0.5,
    "top_p": 0.95,
    "response_mime_type": "application/json",
    "response_schema": {"type":"OBJECT","properties":{"document_type":{"type":"STRING"},"carrier":{"type":"STRING"},"shipped_date":{"type":"STRING"},"received_date":{"type":"STRING"},"purchase_order":{"type":"NUMBER"},"ship_from_complete_address":{"type":"STRING"},"ship_from_street_address":{"type":"STRING"},"ship_from_city":{"type":"STRING"},"ship_from_state":{"type":"STRING"},"ship_from_zip":{"type":"STRING"},"ship_from_name":{"type":"STRING"},"ship_to_complete_address":{"type":"STRING"},"ship_to_street_address":{"type":"STRING"},"ship_to_city":{"type":"STRING"},"ship_to_state":{"type":"STRING"},"ship_to_zip":{"type":"STRING"},"ship_to_name":{"type":"STRING"},"supplier_address":{"type":"STRING"},"supplier_name":{"type":"STRING"},"total_weight_received":{"type":"NUMBER"},"total_cases_received":{"type":"NUMBER"},"total_cube_received":{"type":"NUMBER"}},"required":["document_type","carrier","shipped_date","received_date","purchase_order","ship_from_complete_address","ship_from_street_address","ship_from_city","ship_from_state","ship_from_zip","ship_from_name","ship_to_complete_address","ship_to_street_address","ship_to_city","ship_to_state","ship_to_zip","ship_to_name","supplier_address","supplier_name","total_weight_received","total_cases_received","total_cube_received"]}
}

This produces the following error:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
File /usr/local/lib/python3.10/dist-packages/proto/message.py:729, in Message.__init__(self, mapping, ignore_unknown_fields, **kwargs)
    [728](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/message.py:728) try:
--> [729](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/message.py:729)     pb_value = marshal.to_proto(pb_type, value)
    [730](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/message.py:730) except ValueError:
    [731](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/message.py:731)     # Underscores may be appended to field names
    [732](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/message.py:732)     # that collide with python or proto-plus keywords.
   (...)
    [736](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/message.py:736)     # See related issue
    [737](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/message.py:737)     # https://github.com/googleapis/python-api-core/issues/227

File /usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:235, in BaseMarshal.to_proto(self, proto_type, value, strict)
    [233](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:233)     return {k: self.to_proto(recursive_type, v) for k, v in value.items()}
--> [235](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:235) pb_value = self.get_rule(proto_type=proto_type).to_proto(value)
    [237](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:237) # Sanity check: If we are in strict mode, did we get the value we want?

File /usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:36, in MessageRule.to_proto(self, value)
     [34](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:34) try:
     [35](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:35)     # Try the fast path first.
---> [36](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:36)     return self._descriptor(**value)
     [37](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:37) except TypeError as ex:
     [38](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:38)     # If we have a type error,
     [39](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:39)     # try the slow path in case the error
     [40](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:40)     # was an int64/string issue

ValueError: Protocol message Schema has no "type" field.

During handling of the above exception, another exception occurred:

TypeError                                 Traceback (most recent call last)
File /usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:36, in MessageRule.to_proto(self, value)
     [34](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:34) try:
     [35](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:35)     # Try the fast path first.
---> [36](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:36)     return self._descriptor(**value)
     [37](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:37) except TypeError as ex:
     [38](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:38)     # If we have a type error,
     [39](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:39)     # try the slow path in case the error
     [40](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:40)     # was an int64/string issue

TypeError: Parameter to CopyFrom() must be instance of same class: expected <class 'Schema'> got <class 'dict'>.

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
File /usr/local/lib/python3.10/dist-packages/proto/message.py:729, in Message.__init__(self, mapping, ignore_unknown_fields, **kwargs)
    [728](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/message.py:728) try:
--> [729](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/message.py:729)     pb_value = marshal.to_proto(pb_type, value)
    [730](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/message.py:730) except ValueError:
    [731](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/message.py:731)     # Underscores may be appended to field names
    [732](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/message.py:732)     # that collide with python or proto-plus keywords.
   (...)
    [736](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/message.py:736)     # See related issue
    [737](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/message.py:737)     # https://github.com/googleapis/python-api-core/issues/227

File /usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:233, in BaseMarshal.to_proto(self, proto_type, value, strict)
    [232](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:232)     recursive_type = type(proto_type().value)
--> [233](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:233)     return {k: self.to_proto(recursive_type, v) for k, v in value.items()}
    [235](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:235) pb_value = self.get_rule(proto_type=proto_type).to_proto(value)

File /usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:233, in <dictcomp>(.0)
    [232](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:232)     recursive_type = type(proto_type().value)
--> [233](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:233)     return {k: self.to_proto(recursive_type, v) for k, v in value.items()}
    [235](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:235) pb_value = self.get_rule(proto_type=proto_type).to_proto(value)

File /usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:235, in BaseMarshal.to_proto(self, proto_type, value, strict)
    [233](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:233)     return {k: self.to_proto(recursive_type, v) for k, v in value.items()}
--> [235](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:235) pb_value = self.get_rule(proto_type=proto_type).to_proto(value)
    [237](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:237) # Sanity check: If we are in strict mode, did we get the value we want?

File /usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:36, in MessageRule.to_proto(self, value)
     [34](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:34) try:
     [35](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:35)     # Try the fast path first.
---> [36](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:36)     return self._descriptor(**value)
     [37](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:37) except TypeError as ex:
     [38](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:38)     # If we have a type error,
     [39](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:39)     # try the slow path in case the error
     [40](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:40)     # was an int64/string issue

ValueError: Protocol message Schema has no "type" field.

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
Cell In[2], [line 63](vscode-notebook-cell:?execution_count=2&line=63)
     [36](vscode-notebook-cell:?execution_count=2&line=36) generation_config = {
     [37](vscode-notebook-cell:?execution_count=2&line=37)     "max_output_tokens": 8192,
     [38](vscode-notebook-cell:?execution_count=2&line=38)     "temperature": 0.5,
   (...)
     [41](vscode-notebook-cell:?execution_count=2&line=41)     "response_schema": {"type":"OBJECT","properties":{"document_type":{"type":"STRING"},"carrier":{"type":"STRING"},"shipped_date":{"type":"STRING"},"received_date":{"type":"STRING"},"purchase_order":{"type":"NUMBER"},"ship_from_complete_address":{"type":"STRING"},"ship_from_street_address":{"type":"STRING"},"ship_from_city":{"type":"STRING"},"ship_from_state":{"type":"STRING"},"ship_from_zip":{"type":"STRING"},"ship_from_name":{"type":"STRING"},"ship_to_complete_address":{"type":"STRING"},"ship_to_street_address":{"type":"STRING"},"ship_to_city":{"type":"STRING"},"ship_to_state":{"type":"STRING"},"ship_to_zip":{"type":"STRING"},"ship_to_name":{"type":"STRING"},"supplier_address":{"type":"STRING"},"supplier_name":{"type":"STRING"},"total_weight_received":{"type":"NUMBER"},"total_cases_received":{"type":"NUMBER"},"total_cube_received":{"type":"NUMBER"}},"required":["document_type","carrier","shipped_date","received_date","purchase_order","ship_from_complete_address","ship_from_street_address","ship_from_city","ship_from_state","ship_from_zip","ship_from_name","ship_to_complete_address","ship_to_street_address","ship_to_city","ship_to_state","ship_to_zip","ship_to_name","supplier_address","supplier_name","total_weight_received","total_cases_received","total_cube_received"]}
     [42](vscode-notebook-cell:?execution_count=2&line=42) }
     [44](vscode-notebook-cell:?execution_count=2&line=44) safety_settings = [
     [45](vscode-notebook-cell:?execution_count=2&line=45)     SafetySetting(
     [46](vscode-notebook-cell:?execution_count=2&line=46)         category=SafetySetting.HarmCategory.HARM_CATEGORY_HATE_SPEECH,
   (...)
     [60](vscode-notebook-cell:?execution_count=2&line=60)     )
     [61](vscode-notebook-cell:?execution_count=2&line=61) ]
---> [63](vscode-notebook-cell:?execution_count=2&line=63) generate()

Cell In[2], [line 20](vscode-notebook-cell:?execution_count=2&line=20)
      [9](vscode-notebook-cell:?execution_count=2&line=9) model = GenerativeModel(
     [10](vscode-notebook-cell:?execution_count=2&line=10)   "gemini-1.5-flash-001",
     [11](vscode-notebook-cell:?execution_count=2&line=11)   system_instruction=[textsi_1]
     [12](vscode-notebook-cell:?execution_count=2&line=12) )
     [13](vscode-notebook-cell:?execution_count=2&line=13) responses = model.generate_content(
     [14](vscode-notebook-cell:?execution_count=2&line=14)     ["""Extract the text value of the entities in this document containing a Bill of Lading.""", document1],
     [15](vscode-notebook-cell:?execution_count=2&line=15)     generation_config=generation_config,
     [16](vscode-notebook-cell:?execution_count=2&line=16)     safety_settings=safety_settings,
     [17](vscode-notebook-cell:?execution_count=2&line=17)     stream=True,
     [18](vscode-notebook-cell:?execution_count=2&line=18) )
---> [20](vscode-notebook-cell:?execution_count=2&line=20) for response in responses:
     [21](vscode-notebook-cell:?execution_count=2&line=21)   print(response.text, end="")

File ~/.local/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:769, in _GenerativeModel._generate_content_streaming(self, contents, generation_config, safety_settings, tools, tool_config)
    [742](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/home/username/genai/generate_response/~/.local/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:742) def _generate_content_streaming(
    [743](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/home/username/genai/generate_response/~/.local/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:743)     self,
    [744](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/home/username/genai/generate_response/~/.local/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:744)     contents: ContentsType,
   (...)
    [749](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/home/username/genai/generate_response/~/.local/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:749)     tool_config: Optional["ToolConfig"] = None,
    [750](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/home/username/genai/generate_response/~/.local/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:750) ) -> Iterable["GenerationResponse"]:
    [751](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/home/username/genai/generate_response/~/.local/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:751)     """Generates content.
    [752](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/home/username/genai/generate_response/~/.local/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:752) 
    [753](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/home/username/genai/generate_response/~/.local/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:753)     Args:
   (...)
    [767](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/home/username/genai/generate_response/~/.local/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:767)         A stream of GenerationResponse objects
    [768](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/home/username/genai/generate_response/~/.local/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:768)     """
--> [769](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/home/username/genai/generate_response/~/.local/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:769)     request = self._prepare_request(
    [770](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/home/username/genai/generate_response/~/.local/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:770)         contents=contents,
    [771](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/home/username/genai/generate_response/~/.local/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:771)         generation_config=generation_config,
    [772](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/home/username/genai/generate_response/~/.local/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:772)         safety_settings=safety_settings,
    [773](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/home/username/genai/generate_response/~/.local/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:773)         tools=tools,
    [774](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/home/username/genai/generate_response/~/.local/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:774)         tool_config=tool_config,
    [775](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/home/username/genai/generate_response/~/.local/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:775)     )
    [776](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/home/username/genai/generate_response/~/.local/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:776)     response_stream = self._prediction_client.stream_generate_content(
    [777](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/home/username/genai/generate_response/~/.local/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:777)         request=request
    [778](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/home/username/genai/generate_response/~/.local/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:778)     )
    [779](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/home/username/genai/generate_response/~/.local/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:779)     for chunk in response_stream:

File ~/.local/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:467, in _GenerativeModel._prepare_request(self, contents, generation_config, safety_settings, tools, tool_config, system_instruction)
    [465](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/home/username/genai/generate_response/~/.local/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:465)         gapic_generation_config = generation_config._raw_generation_config
    [466](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/home/username/genai/generate_response/~/.local/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:466)     elif isinstance(generation_config, Dict):
--> [467](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/home/username/genai/generate_response/~/.local/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:467)         gapic_generation_config = gapic_content_types.GenerationConfig(
    [468](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/home/username/genai/generate_response/~/.local/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:468)             **generation_config
    [469](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/home/username/genai/generate_response/~/.local/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:469)         )
    [471](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/home/username/genai/generate_response/~/.local/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:471) gapic_safety_settings = None
    [472](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/home/username/genai/generate_response/~/.local/lib/python3.10/site-packages/vertexai/generative_models/_generative_models.py:472) if safety_settings:

File /usr/local/lib/python3.10/dist-packages/proto/message.py:757, in Message.__init__(self, mapping, ignore_unknown_fields, **kwargs)
    [754](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/message.py:754)         for item in keys_to_update:
    [755](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/message.py:755)             value[f"{item}_"] = value.pop(item)
--> [757](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/message.py:757)     pb_value = marshal.to_proto(pb_type, value)
    [759](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/message.py:759) if pb_value is not None:
    [760](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/message.py:760)     params[key] = pb_value

File /usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:235, in BaseMarshal.to_proto(self, proto_type, value, strict)
    [232](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:232)     recursive_type = type(proto_type().value)
    [233](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:233)     return {k: self.to_proto(recursive_type, v) for k, v in value.items()}
--> [235](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:235) pb_value = self.get_rule(proto_type=proto_type).to_proto(value)
    [237](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:237) # Sanity check: If we are in strict mode, did we get the value we want?
    [238](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:238) if strict and not isinstance(pb_value, proto_type):

File /usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:41, in MessageRule.to_proto(self, value)
     [36](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:36)         return self._descriptor(**value)
     [37](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:37)     except TypeError as ex:
     [38](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:38)         # If we have a type error,
     [39](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:39)         # try the slow path in case the error
     [40](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:40)         # was an int64/string issue
---> [41](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:41)         return self._wrapper(value)._pb
     [42](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:42) return value

File /usr/local/lib/python3.10/dist-packages/proto/message.py:757, in Message.__init__(self, mapping, ignore_unknown_fields, **kwargs)
    [754](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/message.py:754)         for item in keys_to_update:
    [755](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/message.py:755)             value[f"{item}_"] = value.pop(item)
--> [757](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/message.py:757)     pb_value = marshal.to_proto(pb_type, value)
    [759](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/message.py:759) if pb_value is not None:
    [760](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/message.py:760)     params[key] = pb_value

File /usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:233, in BaseMarshal.to_proto(self, proto_type, value, strict)
    [228](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:228) if isinstance(value, dict) and (
    [229](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:229)     proto_type.DESCRIPTOR.has_options
    [230](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:230)     and proto_type.DESCRIPTOR.GetOptions().map_entry
    [231](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:231) ):
    [232](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:232)     recursive_type = type(proto_type().value)
--> [233](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:233)     return {k: self.to_proto(recursive_type, v) for k, v in value.items()}
    [235](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:235) pb_value = self.get_rule(proto_type=proto_type).to_proto(value)
    [237](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:237) # Sanity check: If we are in strict mode, did we get the value we want?

File /usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:233, in <dictcomp>(.0)
    [228](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:228) if isinstance(value, dict) and (
    [229](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:229)     proto_type.DESCRIPTOR.has_options
    [230](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:230)     and proto_type.DESCRIPTOR.GetOptions().map_entry
    [231](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:231) ):
    [232](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:232)     recursive_type = type(proto_type().value)
--> [233](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:233)     return {k: self.to_proto(recursive_type, v) for k, v in value.items()}
    [235](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:235) pb_value = self.get_rule(proto_type=proto_type).to_proto(value)
    [237](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:237) # Sanity check: If we are in strict mode, did we get the value we want?

File /usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:235, in BaseMarshal.to_proto(self, proto_type, value, strict)
    [232](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:232)     recursive_type = type(proto_type().value)
    [233](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:233)     return {k: self.to_proto(recursive_type, v) for k, v in value.items()}
--> [235](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:235) pb_value = self.get_rule(proto_type=proto_type).to_proto(value)
    [237](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:237) # Sanity check: If we are in strict mode, did we get the value we want?
    [238](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/marshal.py:238) if strict and not isinstance(pb_value, proto_type):

File /usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:36, in MessageRule.to_proto(self, value)
     [31](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:31) if isinstance(value, dict) and not self.is_map:
     [32](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:32)     # We need to use the wrapper's marshaling to handle
     [33](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:33)     # potentially problematic nested messages.
     [34](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:34)     try:
     [35](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:35)         # Try the fast path first.
---> [36](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:36)         return self._descriptor(**value)
     [37](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:37)     except TypeError as ex:
     [38](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:38)         # If we have a type error,
     [39](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:39)         # try the slow path in case the error
     [40](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:40)         # was an int64/string issue
     [41](https://vscode-remote+970-002dcs-002d233782833769-002ddefault-002ecs-002dus-002dwest1-002dwolo-002ecloudshell-002edev.vscode-resource.vscode-cdn.net/usr/local/lib/python3.10/dist-packages/proto/marshal/rules/message.py:41)         return self._wrapper(value)._pb

ValueError: Protocol message Schema has no "type" field.

@ItsOkayItsOfficial
Copy link

Following up on our previous comment above

We were able to run this successfully by replacing all references to type with type_

This generation_config produces the expected outcomes.

generation_config = {
    "max_output_tokens": 8192,
    "temperature": 0.5,
    "top_p": 0.95,
    "response_mime_type": "application/json",
    "response_schema": {"type_":"OBJECT","properties":{"document_type_":{"type_":"STRING"},"carrier":{"type_":"STRING"},"shipped_date":{"type_":"STRING"},"received_date":{"type_":"STRING"},"purchase_order":{"type_":"NUMBER"},"ship_from_complete_address":{"type_":"STRING"},"ship_from_street_address":{"type_":"STRING"},"ship_from_city":{"type_":"STRING"},"ship_from_state":{"type_":"STRING"},"ship_from_zip":{"type_":"STRING"},"ship_from_name":{"type_":"STRING"},"ship_to_complete_address":{"type_":"STRING"},"ship_to_street_address":{"type_":"STRING"},"ship_to_city":{"type_":"STRING"},"ship_to_state":{"type_":"STRING"},"ship_to_zip":{"type_":"STRING"},"ship_to_name":{"type_":"STRING"},"supplier_address":{"type_":"STRING"},"supplier_name":{"type_":"STRING"},"total_weight_received":{"type_":"NUMBER"},"total_cases_received":{"type_":"NUMBER"},"total_cube_received":{"type_":"NUMBER"}},"required":["document_type_","carrier","shipped_date","received_date","purchase_order","ship_from_complete_address","ship_from_street_address","ship_from_city","ship_from_state","ship_from_zip","ship_from_name","ship_to_complete_address","ship_to_street_address","ship_to_city","ship_to_state","ship_to_zip","ship_to_name","supplier_address","supplier_name","total_weight_received","total_cases_received","total_cube_received"]}
}

@jaycee-li
Copy link
Contributor

Hi @ItsOkayItsOfficial , thanks for raising this issue! This is a bug and I'm fixing it. Both type and type_ should work here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jaycee-li jaycee-li

Labels
api: vertex-ai Issues related to the googleapis/python-aiplatform API.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@Ark-kun @ItsOkayItsOfficial @jaycee-li @orenmiaraswish