Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

refactor: remove User LLM/embed. defaults, add credentials file, add authentication option for custom LLM backends #835

Merged
merged 9 commits into from
Jan 19, 2024
Merged

refactor: remove User LLM/embed. defaults, add credentials file, add authentication option for custom LLM backends #835

merged 9 commits into from
Jan 19, 2024

Conversation

cpacker
Copy link
Owner

@cpacker cpacker commented Jan 17, 2024 •
edited
Loading

Closes #604

Please describe the purpose of this pull request.

  • Removes User LLM/embedding defaults, these should get pulled from the base MemGPT config now
  • Adds a ~/.memgpt/credentials file, this is where credentials sit now
  • Adds an authentication option for custom LLM backend POST requests
    • Currently adds support for bearer tokens and API keys in the HTTP headers
  • Note: renames openai_tools.py to llm_api_tools.py

How to test

  • Try using one of the completions local LLM routes to point at an API service that is compatible but requires a key

Have you tested this PR?

  • TODO

TODOs:

  • Add setting of auth_type and auth_key during memgpt configure for local/open/custom LLMs

@cpacker
renamed openai_tools.py (since it also handles non-openai calls), add…
a8f1145 
…ed auth_type and auth_key fields to LLM and embedding configs, modifies REST API calls to use auth_type+auth_key
@cpacker
missing file after rename
8d4df8f
@cpacker
merged
f94d3e1
cpacker
cpacker commented Jan 18, 2024
View reviewed changes
memgpt/data_types.py Outdated Show resolved Hide resolved
@cpacker
refactor: (1) credentials for in a ~/.memgpt/credentials file, (2) no…
2442268 
… more llm/embedding config defaults in User, these go in config instead, (3) config internal representation is not flat, it has LLMConfig and EmbeddingConfig
@cpacker cpacker changed the title feat: add authentication option for custom LLM backends refactor: remove User LLM/embed. defaults, add credentials file, add authentication option for custom LLM backends Jan 18, 2024
@sarahwooders sarahwooders self-requested a review January 19, 2024 00:11
sarahwooders
sarahwooders approved these changes Jan 19, 2024
View reviewed changes
Copy link
Collaborator

@sarahwooders sarahwooders left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@sarahwooders sarahwooders merged commit e4fab16 into main Jan 19, 2024
3 checks passed
@sarahwooders sarahwooders deleted the add-auth-to-localllm-requests branch January 19, 2024 00:11
@cpacker cpacker mentioned this pull request Jan 19, 2024
Open
@tombedor tombedor mentioned this pull request Jan 21, 2024
Closed
@cpacker cpacker mentioned this pull request Jan 22, 2024
Draft
norton120 pushed a commit to norton120/MemGPT that referenced this pull request Feb 15, 2024
@cpacker @norton120
refactor: remove User LLM/embed. defaults, add credentials file, add …
2c37dbe 
…authentication option for custom LLM backends (cpacker#835)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sarahwooders sarahwooders sarahwooders approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Allow sending keys / bearer tokens on local/custom LLM calls
2 participants
@cpacker @sarahwooders