ParserConstraint

[tonxxd]

This seems like a very promising project!
I am trying to execute the ParserConstraint example but it get and error

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[10], line 28
     20 openai.api_key = "..."
     23 prompt = Prompt("""
     24 %system%You are a sql expert%/system%
     25 %user%Write me a query that selects the column y from table b.%/user%
     26 """)
---> 28 prompt = await prompt.complete(model=hf, constraint=constraint, name='query', max_tokens=100)
     29 # Prompt('
     30 # %system%You are a sql expert%/system%
     31 # %user%Write me a query that selects the column y from table b.%/user%
     32 # SELECT y FROM b')

File /lib/python3.10/site-packages/keymaker/prompt.py:353, in Prompt.complete(self, *completion_args, **completion_kwargs)
    341         await stream(
    342             Completion(
    343                 value=value,
   (...)
    349             ),
    350         )
    352 while token_count < token_limit:
--> 353     selected_token_ids = await constraint.constrain_tokens(text, partial_completion, model)
    355     if selected_token_ids:
    356         # if the selected tokens have the same number than the vocab size, there's no real restriction
...
    131             model.tokens.keys(),
    132         ),
    133     )

TypeError: unhashable type: 'ParserConstraint'

Any idea why thsi could happen? Thanks

[CircArgs]

@tonxxd sorry for the delay! I've been away for a few weeks. I'll get right back to you tomorrow

[CircArgs]

@tonxxd Thanks for the report! There was a small bug in the ParserConstraint trying to cache some of the work.

The ParserConstraint overall is more complicated/powerful than other constraints. Arguably, it is the most powerful and unrestricted. As such however, it can be the trickiest to get to work how you want. Some options were added between the original example and the latest code to make it slightly easier to work with (ignoring whitespace by default for example) and the caching which introduced this regression (we need tests btw which is a big todo in case you are interested in contributing 😊)

The below should work as expected now if you'd like to give it a try:

constraint = ParserConstraint(parser = parser, ignore_pattern="")

prompt = Prompt("""
%system%You are a sql expert%/system%
%user%Write me a query that selects the column y from table b.%/user%
""")

prompt = await prompt.complete(model=model, constraint=constraint, name='query', max_tokens=100)
# Prompt('
# %system%You are a sql expert%/system%
# %user%Write me a query that selects the column y from table b.%/user%
# SELECT y FROM b')