Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[WIP] agent example (w/ sandboxable Tools!) & improved OAI compatibility layer (in Python) #6389

Draft
wants to merge 73 commits into
base: master
Choose a base branch
from
Draft

[WIP] agent example (w/ sandboxable Tools!) & improved OAI compatibility layer (in Python) #6389

wants to merge 73 commits into from
+4,167 −82

Commits on Apr 27, 2024

  1. gguf: add GGUFReader.read_field(field) method + read template example

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    0d47c43 View commit details
    Browse the repository at this point in the history

  2. grammars: add troubleshooting section to readme

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    0d1d46e View commit details
    Browse the repository at this point in the history

  3. server.py: hacky code

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    63d1324 View commit details
    Browse the repository at this point in the history

  4. agents: scripts to run scripts as sandboxed fastapi servers

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    ffc7436 View commit details
    Browse the repository at this point in the history

  5. server.py: default tools work!

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    d5d9993 View commit details
    Browse the repository at this point in the history

  6. server.py: make tools work w/ mixtral-8x7b-instruct

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    8afd4de View commit details
    Browse the repository at this point in the history

  7. server.py: kinda api-compliant output, disabled grammar

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    aa9605c View commit details
    Browse the repository at this point in the history

  8. server.py: reenable grammar, accommodate mistral's escaped underscores

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    a406293 View commit details
    Browse the repository at this point in the history

  9. server.py: raise n_predict

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    63a384d View commit details
    Browse the repository at this point in the history

  10. server.py: pass all request options, comments in ts sigs, render tool… 

    … calls
    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    5f3de16 View commit details
    Browse the repository at this point in the history

  11. server.py: refactor chat handlers

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    59b4114 View commit details
    Browse the repository at this point in the history

  12. server.py: crude reactor

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    253b68d View commit details
    Browse the repository at this point in the history

  13. agent: split code from openai example

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    e874565 View commit details
    Browse the repository at this point in the history

  14. Update agent.py

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    b63f91a View commit details
    Browse the repository at this point in the history

  15. Update example_weather_tools.py

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    c340e8c View commit details
    Browse the repository at this point in the history

  16. agent: add --allow_parallel_calls

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    ce2fb01 View commit details
    Browse the repository at this point in the history

  17. agent/openai:nits

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    ea34bd3 View commit details
    Browse the repository at this point in the history

  18. openai: fix message merging for mixtral (parallel calls)

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    80c7930 View commit details
    Browse the repository at this point in the history

  19. Update prompting.py

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    9ab493f View commit details
    Browse the repository at this point in the history

  20. agent: --style

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    e0c8af4 View commit details
    Browse the repository at this point in the history

  21. Create requirements.txt

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    b4e292e View commit details
    Browse the repository at this point in the history

  22. agent: disable parallel by default

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    d1d8602 View commit details
    Browse the repository at this point in the history

  23. agent: nits

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    eb9a552 View commit details
    Browse the repository at this point in the history

  24. agent: fix functionary tool_calls templating

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    3da30ed View commit details
    Browse the repository at this point in the history

  25. Delete test.sh

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    ff6563a View commit details
    Browse the repository at this point in the history

  26. agent: format still broken

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    dd11bb6 View commit details
    Browse the repository at this point in the history

  27. agent: update readme

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    22b980f View commit details
    Browse the repository at this point in the history

  28. agent: prepare to test various templates

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    61f35e0 View commit details
    Browse the repository at this point in the history

  29. openai: test features of templates at runtime, to make sure no bits o… 

    …f intel are lost
    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    d8a53ea View commit details
    Browse the repository at this point in the history

  30. Update test_chat_handlers.py

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    ad2f4c1 View commit details
    Browse the repository at this point in the history

  31. openai: quiet + update prompt output

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    3c3eff5 View commit details
    Browse the repository at this point in the history

  32. openai: refactor chat handler vs. template

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    6935503 View commit details
    Browse the repository at this point in the history

  33. Update test_chat_handlers.md

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    d9f30f8 View commit details
    Browse the repository at this point in the history

  34. openai: only special-format assistant in thoughtful mode

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    da2067a View commit details
    Browse the repository at this point in the history

  35. openai: actually use thoughtful examples in tests

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    09de4eb View commit details
    Browse the repository at this point in the history

  36. openai: tests didn't catch output format

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    19811a4 View commit details
    Browse the repository at this point in the history

  37. openai tools: TS signatures work well too at a fraction of the eval cost

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    22fe86d View commit details
    Browse the repository at this point in the history

  38. Update test_chat_handlers.md

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    6e52a9c View commit details
    Browse the repository at this point in the history

  39. agent: fix response_format

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    701a66d View commit details
    Browse the repository at this point in the history

  40. agent: revert to json schemas (ts not ready for refs)

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    b447a74 View commit details
    Browse the repository at this point in the history

  41. agent: fix sandbox dockerfile

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    85820f4 View commit details
    Browse the repository at this point in the history

  42. agent: support basic openapi tools (incl. from fastify sandbox)

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    6880f1d View commit details
    Browse the repository at this point in the history

  43. agent: nits

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    0532680 View commit details
    Browse the repository at this point in the history

  44. agent: cache_prompt=True

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    a634e03 View commit details
    Browse the repository at this point in the history

  45. openai: nit

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    9fe269e View commit details
    Browse the repository at this point in the history

  46. agent: hint at math import in python tool

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    a61ebeb View commit details
    Browse the repository at this point in the history

  47. agent: nit

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    24e34f1 View commit details
    Browse the repository at this point in the history

  48. agent: fix killing of subprocesses 

    subprocesses again
    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    1475b1e View commit details
    Browse the repository at this point in the history

  49. agent: nits

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    6c00378 View commit details
    Browse the repository at this point in the history

  50. agent: rename fake weather tools

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    082d54d View commit details
    Browse the repository at this point in the history

  51. agent: python tool: test serializability of variables

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    f9afb04 View commit details
    Browse the repository at this point in the history

  52. agent: python tool: return errors

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    a98f483 View commit details
    Browse the repository at this point in the history

  53. agent: ensure DATA_DIR exists 

    skip-checks:true
    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    ea0c31b View commit details
    Browse the repository at this point in the history

  54. agent: mypy type fixes 

    mypy examples/agent/__main__.py
mypy examples/agent/fastify.py
mypy examples/openai/__main__.py
    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    89dcc06 View commit details
    Browse the repository at this point in the history

  55. agent: fix wait --std-tools

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    0120f7c View commit details
    Browse the repository at this point in the history

  56. grammars: early exit when no next_candidates to reject

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    09c2565 View commit details
    Browse the repository at this point in the history

  57. grammars: cache decoded tokens

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    00c709e View commit details
    Browse the repository at this point in the history

  58. grammars: faster llama_grammar_copy

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    8d503ef View commit details
    Browse the repository at this point in the history

  59. Merge branch 'gguf-read' into agent-example

    @ochafik
    ochafik committed Apr 27, 2024
    Configuration menu
    Copy the full SHA
    b4a00ce View commit details
    Browse the repository at this point in the history

Commits on Apr 30, 2024

  1. Merge remote-tracking branch 'origin/master' into agent-example

    @ochafik
    ochafik committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    7675ac6 View commit details
    Browse the repository at this point in the history

  2. openai: update after merge 

    typos
    @ochafik
    ochafik committed Apr 30, 2024
    Configuration menu
    Copy the full SHA
    312e20b View commit details
    Browse the repository at this point in the history

Commits on May 2, 2024

  1. server: tool call grammar-constraints 

    fix
    @ochafik
    ochafik committed May 2, 2024
    Configuration menu
    Copy the full SHA
    ca1a640 View commit details
    Browse the repository at this point in the history

  2. agent: url params

    @ochafik
    ochafik committed May 2, 2024
    Configuration menu
    Copy the full SHA
    2b2127c View commit details
    Browse the repository at this point in the history

  3. server: update tool calling, introduce system prompt for json schema

    @ochafik
    ochafik committed May 2, 2024
    Configuration menu
    Copy the full SHA
    e41b6ce View commit details
    Browse the repository at this point in the history

Commits on May 18, 2024

  1. openai: function call arguments must be returned stringified!

    @ochafik
    ochafik committed May 18, 2024
    Configuration menu
    Copy the full SHA
    a1d64cf View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/master' into agent-example

    @ochafik
    ochafik committed May 18, 2024
    Configuration menu
    Copy the full SHA
    3f5a25f View commit details
    Browse the repository at this point in the history

Commits on May 21, 2024

  1. openai: fix merge

    @ochafik
    ochafik committed May 21, 2024
    Configuration menu
    Copy the full SHA
    5ea637e View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/master' into agent-example

    @ochafik
    ochafik committed May 21, 2024
    Configuration menu
    Copy the full SHA
    6dadcd2 View commit details
    Browse the repository at this point in the history

Commits on May 22, 2024

  1. openai: make content optional for tool call grammar gen

    @ochafik
    ochafik committed May 22, 2024
    Configuration menu
    Copy the full SHA
    c8458fa View commit details
    Browse the repository at this point in the history

  2. openai: pretty indent json response

    @ochafik
    ochafik committed May 22, 2024
    Configuration menu
    Copy the full SHA
    a39e6e0 View commit details
    Browse the repository at this point in the history

  3. agent: support OpenAI: --endpoint https://api.openai.com --auth "Bear… 

    …er $OPENAI_API_KEY"
    @ochafik
    ochafik committed May 22, 2024
    Configuration menu
    Copy the full SHA
    793f4ff View commit details
    Browse the repository at this point in the history

  4. server: ultra basic tools, tool_choice, tool_calls support

    @ochafik
    ochafik committed May 22, 2024
    Configuration menu
    Copy the full SHA
    a1c4aac View commit details
    Browse the repository at this point in the history

Commits on Jun 9, 2024

  1. Merge remote-tracking branch 'origin/master' into agent-example

    @ochafik
    ochafik committed Jun 9, 2024
    Configuration menu
    Copy the full SHA
    298c098 View commit details
    Browse the repository at this point in the history