"additionalProperties" should not be a required field

[sgsdxzy]

Describe the issue as clearly as possible:

Currently for object, "additionalProperties" is required to be a dict. However, it can be optional or false.

Steps/code to reproduce the bug:

A simple schema, sent to vLLM/Aphrodite-engine as guided_json. The properties and types of parameters cannot be known beforehand because the function signature varies for different tools. This schema works fine in lm-format-enforcer

json_template = {
    "type": "array",
    "items": {
        "type": "object",
        "properties": {
            "tool_name": {"type": "string"},
            "parameters": {"type": "object"},
        },
        "required": ["tool_name", "parameters"],
    },
    "minItems": 1,
    "maxItems": 1,
}

Expected result:

[
    {
        "tool_name": "internet_search",
        "parameters": {
            "query": "biggest penguin species",
            "provider": "Google"
        }
    }
]

Error message:

ERROR:      File "/home/sgsdxzy/Programs/aphrodite-engine/aphrodite/modeling/guided_decoding/outlines_decoding.py", line 126, in _get_cached_logits_processor
ERROR:        return JSONLogitsProcessor(guide, tokenizer)
ERROR:               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR:      File "/home/sgsdxzy/Programs/aphrodite-engine/aphrodite/modeling/guided_decoding/outlines_logits_processors.py", line 111, in __init__
ERROR:        regex_string = build_regex_from_schema(schema_str, whitespace_pattern)
ERROR:                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR:      File "/home/sgsdxzy/micromamba/envs/aphrodite/lib/python3.11/site-packages/outlines/fsm/json_schema.py", line 83, in build_regex_from_schema
ERROR:        return to_regex(resolver, content, whitespace_pattern)
ERROR:               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR:      File "/home/sgsdxzy/micromamba/envs/aphrodite/lib/python3.11/site-packages/outlines/fsm/json_schema.py", line 290, in to_regex
ERROR:        items_regex = to_regex(resolver, instance["items"], whitespace_pattern)
ERROR:                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR:      File "/home/sgsdxzy/micromamba/envs/aphrodite/lib/python3.11/site-packages/outlines/fsm/json_schema.py", line 142, in to_regex
ERROR:        subregex += to_regex(resolver, value, whitespace_pattern)
ERROR:                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR:      File "/home/sgsdxzy/micromamba/envs/aphrodite/lib/python3.11/site-packages/outlines/fsm/json_schema.py", line 320, in to_regex
ERROR:        value_pattern = to_regex(
ERROR:                        ^^^^^^^^^
ERROR:      File "/home/sgsdxzy/micromamba/envs/aphrodite/lib/python3.11/site-packages/outlines/fsm/json_schema.py", line 321, in to_regex
ERROR:        resolver, instance["additionalProperties"], whitespace_pattern
ERROR:                  ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
ERROR:    KeyError: 'additionalProperties'

Outlines/Python version information:

Version information

``` outlines: 0.0.34 python: 3.11.9 ```

Context for the issue:

Affects guided json generation for function calling in vLLM/Aphrodite-engine.

[leloykun]

@sgsdxzy, the latest version of Outlines uses the "property" field if it's present

Updating to outlines==0.0.41 should fix this issue. Oh, and make sure to install this after installing vllm; otherwise, you'll get an error

[sgsdxzy]

After upgrading to outlines==0.0.41, I am still unable to specify a unconstrained object, for example the following schema:

json_template = {
    "type": "array",
    "items": {
        "type": "object",
        "properties": {
            "tool_name": {"type": "string"},
            "parameters": {"type": "object"},
        },
        "required": ["tool_name", "parameters"],
    },
    "minItems": 1,
    "maxItems": 1,
}

is still rejected by outlines because neither "properties" nor "additionalProperties" is presented for "parameters", but I want "parameters" to be entirely controlled by the llm, as long as it's a valid json object, without any restriction on it's contents. How do I specify a free-form object?

[lapp0]

@sgsdxzy Could you please try my branch and report back whether it's working for your use case?

pip install git+https://github.com/lapp0/outlines@fix-844

Input script:

from outlines.fsm.json_schema import build_regex_from_schema
import json
import re


json_template = {
    "type": "array",
    "items": {
        "type": "object",
        "properties": {
            "tool_name": {"type": "string"},
            "parameters": {"type": "object"},
        },
        "required": ["tool_name", "parameters"],
    },
    "minItems": 1,
    "maxItems": 1,
}


pattern = build_regex_from_schema(json.dumps(json_template))


valid_result = """[
    {
        "tool_name": "internet_search",
        "parameters": {
            "query": "biggest penguin species",
            "provider": "Google"
        }
    }
]"""

match = re.match(pattern, valid_result)
assert match.string == valid_result
print("fully matched expectation!")

Output:
fully matched expectation!

[sgsdxzy]

@lapp0 Thanks, I can confirm your branch works correctly.