-
-
Notifications
You must be signed in to change notification settings - Fork 2.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
🚀 Feat: Streamline File Strategies & GPT-4-Vision Settings #1535
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…stic update of filesQuery on file upload
…ble after files are fetched
- rename app.locals.configs to app.locals.paths - load custom config use fileStrategy from yaml config in app.locals
…of Objects and only their values (not keys) aside from `image_url` types
…logins or user upload
….js with changes made
7 tasks
danny-avila
added a commit
to lgtm-hbu/LibreChat
that referenced
this pull request
Jan 17, 2024
danny-avila
added a commit
to lgtm-hbu/LibreChat
that referenced
this pull request
Jan 19, 2024
danny-avila
added a commit
to lgtm-hbu/LibreChat
that referenced
this pull request
Jan 22, 2024
danny-avila
added a commit
to lgtm-hbu/LibreChat
that referenced
this pull request
Jan 23, 2024
paychex-ssmithrand
added a commit
to paychex/LibreChat
that referenced
this pull request
Jan 23, 2024
* documentation update: mongodb (danny-avila#1347) * update dotenv.md add details about the MONGO_URI connection string format * update mongodb.md add details about the MONGO_URI connection string format * chore: Get the latest of all github actions (danny-avila#1335) * refactor(api): Central Logging 📜 (danny-avila#1348) * WIP: initial logging changes add several transports in ~/config/winston omit messages in logs, truncate long strings add short blurb in dotenv for debug logging GoogleClient: using logger OpenAIClient: using logger, handleOpenAIErrors Adding typedef for payload message bumped winston and using winston-daily-rotate-file moved config for server paths to ~/config dir Added `DEBUG_LOGGING=true` to .env.example * WIP: Refactor logging statements in code * WIP: Refactor logging statements and import configurations * WIP: Refactor logging statements and import configurations * refactor: broadcast Redis initialization message with `info` not `debug` * refactor: complete Refactor logging statements and import configurations * chore: delete unused tools * fix: circular dependencies due to accessing logger * refactor(handleText): handle booleans and write tests * refactor: redact sensitive values, better formatting * chore: improve log formatting, avoid passing strings to 2nd arg * fix(ci): fix jest tests due to logger changes * refactor(getAvailablePluginsController): cache plugins as they are static and avoids async addOpenAPISpecs call every time * chore: update docs * chore: update docs * chore: create separate meiliSync logger, clean up logs to avoid being unnecessarily verbose * chore: spread objects where they are commonly logged to allow string truncation * chore: improve error log formatting * fix: Add error handling for missing role in OpenAIClient.js (danny-avila#1352) * feat: Google Gemini ❇️ (danny-avila#1355) * refactor: add gemini-pro to google Models list; use defaultModels for central model listing * refactor(SetKeyDialog): create useMultipleKeys hook to use for Azure, export `isJson` from utils, use EModelEndpoint * refactor(useUserKey): change variable names to make keyName setting more clear * refactor(FileUpload): allow passing container className string * feat(GoogleClient): Gemini support * refactor(GoogleClient): alternate stream speed for Gemini models * feat(Gemini): styling/settings configuration for Gemini * refactor(GoogleClient): substract max response tokens from max context tokens if context is above 32k (I/O max is combined between the two) * refactor(tokens): correct google max token counts and subtract max response tokens when input/output count are combined towards max context count * feat(google/initializeClient): handle both local and user_provided credentials and write tests * fix(GoogleClient): catch if credentials are undefined, handle if serviceKey is string or object correctly, handle no examples passed, throw error if not a Generative Language model and no service account JSON key is provided, throw error if it is a Generative m odel, but not google API key was provided * refactor(loadAsyncEndpoints/google): activate Google endpoint if either the service key JSON file is provided in /api/data, or a GOOGLE_KEY is defined. * docs: updated Google configuration * fix(ci): Mock import of Service Account Key JSON file (auth.json) * Update apis_and_tokens.md * feat: increase max output tokens slider for gemini pro * refactor(GoogleSettings): handle max and default maxOutputTokens on model change * chore: add sensitive redact regex * docs: add warning about data privacy * Update apis_and_tokens.md * fix: Avoid Throwing Errors for Unsupported Token Count Endpoints 🪙 (danny-avila#1356) * update: docker-compose.yaml 🐋 (danny-avila#1341) * fix: Enhance Test Coverage and Fix Compatibility Issues 👷♂️ (danny-avila#1363) * refactor: only remove conversation states from localStorage on login/logout but not on refresh * chore: add debugging log for azure completion url * chore: add api-key to redact regex * fix: do not show endpoint selector if endpoint is falsy * chore: remove logger from genAzureChatCompletion * feat(ci): mock fetchEventSource * refactor(ci): mock all model methods in BaseClient.test, as well as mock the implementation for getCompletion in FakeClient * fix(OpenAIClient): consider chatCompletion if model name includes `gpt` as opposed to `gpt-` * fix(ChatGPTClient/azureOpenAI): Remove 'model' option for Azure compatibility (cannot be sent in payload body) * feat(ci): write new test suite that significantly increase test coverage for OpenAIClient and BaseClient by covering most of the real implementation of the `sendMessage` method - test for the azure edge case where model option is appended to modelOptions, ensuring removal before sent to the azure endpoint - test for expected azure url being passed to SSE POST request - test for AZURE_OPENAI_DEFAULT_MODEL being set, but is not included in the URL deployment name as expected - test getCompletion method to have correct payload fix(ci/OpenAIClient.test.js): correctly mock hanging/async methods * refactor(addTitle): allow azure to title as it aborts signal on completion * 🔄 refactor: Consolidate Ask/Edit Controllers (danny-avila#1365) * refactor(Ask/Edit): consolidate ask/edit controllers between the main modules and openAI controllers to reduce repetition of code and increase reusability * fix(winston/logger): circular dependency issue * fix(config/scripts): fix script imports * refactor(indexSync): make not configured message an info log message * chore: create a rollup script for api/server/index.js to check circular dependencies * chore: bump @keyv/redis * Update package.json * 🍏Update mac_install.md (danny-avila#1373) The manual installation guide for mac was very outdated. This brings it up to date with the current method * 📸 feat: Gemini vision, Improved Logs and Multi-modal Handling (danny-avila#1368) * feat: add GOOGLE_MODELS env var * feat: add gemini vision support * refactor(GoogleClient): adjust clientOptions handling depending on model * fix(logger): fix redact logic and redact errors only * fix(GoogleClient): do not allow non-multiModal messages when gemini-pro-vision is selected * refactor(OpenAIClient): use `isVisionModel` client property to avoid calling validateVisionModel multiple times * refactor: better debug logging by correctly traversing, redacting sensitive info, and logging condensed versions of long values * refactor(GoogleClient): allow response errors to be thrown/caught above client handling so user receives meaningful error message debug orderedMessages, parentMessageId, and buildMessages result * refactor(AskController): use model from client.modelOptions.model when saving intermediate messages, which requires for the progress callback to be initialized after the client is initialized * feat(useSSE): revert to previous model if the model was auto-switched by backend due to message attachments * docs: update with google updates, notes about Gemini Pro Vision * fix: redis should not be initialized without USE_REDIS and increase max listeners to 20 * fix: revert `fonts` resolution path in vite.config.ts * 🚩 fix: Initialize Conversation Only when Necessary Data is Fetched (danny-avila#1379) * fix(ChatRoute): only initialize conversation after all data is fetched (models, endpoints, initialConversationQuery if not `new`) * chore: remove unnecessary packages for rolling up api * chore: bump data-provider package.json * ✨ feat: Azure Vision Support & Docs Update (danny-avila#1389) * feat(AzureOpenAI): Vision Support * chore(ci/OpenAIClient.test): update test to reflect Azure now uses chatCompletion method as opposed to getCompletion, while still testing the latter method * docs: update documentation mainly revolving around Azure setup, but also reformatting the 'Tokens and API' section completely * docs: add images and links to ai_setup.md * docs: ai setup reference * ✨ Release v0.6.5 (danny-avila#1391) * ✨ Release v0.6.5 * fix(ci): use dynamic currentDateString * Update pull_request_template.md (danny-avila#1417) * 🧹📚 docs: refactor and clean up (danny-avila#1392) * 📑 update mkdocs * rename docker override file and add to gitignore * update .env.example - GOOGLE_MODELS * update index.md * doc refactor: split installation and configuration in two sub-folders * doc update: installation guides * doc update: configuration guides * doc: new docker override guide * doc: new beginner's guide for contributions - Thanks @berry-13 * doc: update documentation_guidelines.md * doc: update testing.md * doc: update deployment guides * doc: update /dev readme * doc: update general_info * doc: add 0 value to doc weight * doc: add index.md to every doc folders * doc: add weight to index.md and move openrouter from free_ai_apis.md to ai_setup.md * doc: update toc so they display properly on the right had side in mkdocs * doc: update pandoranext.md * doc: index logging_system.md * doc: update readme.md * doc: update litellm.md * doc: update ./dev/readme.md * doc:🔖 new presets.md * doc: minor corrections * doc update: user_auth_system.md and presets.md, doc feat: add mermaid support to mkdocs * doc update: add screenshots to presets.md * doc update: add screenshots to - OpenID with AWS Cognito * doc update: BingAI cookie instruction * doc update: discord auth * doc update: facebook auth * doc: corrections to user_auth_system.md * doc update: github auth * doc update: google auth * doc update: auth clean up * doc organization: installation * doc organization: configuration * doc organization: features+plugins & update:plugins screenshots * doc organization: deploymend + general_info & update: tech_stack.md * doc organization: contributions * doc: minor fixes * doc: minor fixes * Update French Translation (danny-avila#1444) * 🪪mkdocs: social cards (danny-avila#1428) * mkdocs plugins: add plugin for social cards and plugin that allow to exclude a folder * docs: fix hyperlinks * mkdocs: social cards (descriptions) for 'contributions' and 'deployment' guides * mkdocs: social cards (descriptions) for all 'index.md' * mkdocs: social cards (descriptions) for 'features' and 'plugins' * mkdocs: social cards (descriptions) for 'general_info' * mkdocs: social cards (descriptions) for 'configuration' * mkdocs: social cards (descriptions) for 'installation' * mkdocs: minor fixes * update librechat.svg * update how_to_contribute.md add reference to the official GitHub documentation * 🌎: Italian translation update & refactor: translations (danny-avila#1414) * italian translation update * fix: removed some translations * refactor(Translation) * feat: allow FWA (danny-avila#1440) * 🌎: Update Russian translations (danny-avila#1413) * Update Russian localization * Update Ru.tsx * fix: russian translation typing errors * 🐛 fix: Prevent Unnecessary Cloning of Symbols in Log Object (danny-avila#1455) fix(api/config/parsers): prevent cloning of unnecessary symbols within log object by using `klona` instead of `klona/full`, handle symbols edge case, log parsing errors, and use spaces instead of tab for cleaner logs * 🚫🔍 feat: disallow search indexing (danny-avila#1409) * feat: disallow search indexing * Update index.js * Update .env.example * added middleware * 🔥🚀 feat: CDN (Firebase) & feat: account section (danny-avila#1438) * localization + api-endpoint * docs: added firebase documentation * chore: icons * chore: SettingsTabs * feat: account pannel; fix: gear icons * docs: position update * feat: firebase * feat: plugin support * route * fixed bugs with firebase and moved a lot of files * chore(DALLE3): using UUID v4 * feat: support for social strategies; moved '/images' path * fix: data ignored * gitignore update * docs: update firebase guide * refactor: Firebase - use singleton pattern for firebase initialization, initially on server start - reorganize imports, move firebase specific files to own service under Files - rename modules to remove 'avatar' redundancy - fix imports based on changes * ci(DALLE/DALLE3): fix tests to use logger and new expected outputs, add firebase tests * refactor(loadToolWithAuth): pass userId to tool as field * feat(images/parse): feat: Add URL Image Basename Extraction Implement a new module to extract the basename of an image from a given URL. This addition includes the function, which parses the URL and retrieves the basename using the Node.js 'url' and 'path' modules. The function is documented with JSDoc comments for better maintainability and understanding. This feature enhances the application's ability to handle and process image URLs efficiently. * refactor(addImages): function to use a more specific regular expression for observedImagePath based on the generated image markdown standard across the app * refactor(DALLE/DALLE3): utilize `getImageBasename` and `this.userId`; fix: pass correct image path to firebase url helper * fix(addImages): make more general to match any image markdown descriptor * fix(parse/getImageBasename): test result of this function for an actual image basename * ci(DALLE3): mock getImageBasename * refactor(AuthContext): use Recoil atom state for user * feat: useUploadAvatarMutation, react-query hook for avatar upload * fix(Toast): stack z-order of Toast over all components (1000) * refactor(showToast): add optional status field to avoid importing NotificationSeverity on each use of the function * refactor(routes/avatar): remove unnecessary get route, get userId from req.user.id, require auth on POST request * chore(uploadAvatar): TODO: remove direct use of Model, `User` * fix(client): fix Spinner imports * refactor(Avatar): use react-query hook, Toast, remove unnecessary states, add optimistic UI to upload * fix(avatar/localStrategy): correctly save local profile picture and cache bust for immediate rendering; fix: firebase init info message (only show once) * fix: use `includes` instead of `endsWith` for checking manual query of avatar image path in case more queries are appended (as is done in avatar/localStrategy) --------- Co-authored-by: Danny Avila <messagedaniel@protonmail.com> * 🧹fix: Handle Abort Message Edge Cases (danny-avila#1462) * chore: bump langchain to v0.0.213 from v0.0.186 * fix: handle abort edge cases: - abort message server-side if response experienced error mid-generation - attempt to recover message if aborting resulted in error - if abortKey is not provided, use conversationId if it exists - if headers were already sent, send an Event stream message - issue warning for possible Google censor/filter refactor(streamResponse): for `sendError`, allow passing overrides so that error can include partial generation, improve typing for `sendMessage` * chore(MessageContent): remove eslint warning for unused `i`, rephrase unfinished message text * fix(useSSE): avoid invoking cancelHandler if the abort response was 404 * chore(TMessage): remove unnecessary, unused legacy message property `submitting` * chore(TMessage): remove unnecessary legacy message property `cancelled` * chore(abortMiddleware): remove unused `errorText` property to avoid confusion * doc update: firebase.md (danny-avila#1456) * 📃 feat: add `list-balances`, `remove-user`, and improve User scripts (danny-avila#1418) * Refactoring opening of DB to config/helpers.js * Adding two user scripts: - 'delete-user' to remove a user definitely - 'list-balances' to show the balances of all the users * Add Russian translation for the new functionality in the settings (danny-avila#1457) * 🛠️ fix: Minor Fixes in `Message`, `Ask/EditController`, `OpenAIClient`, and `countTokens` (danny-avila#1463) * fix(Message): avoid overwriting unprovided properties * fix(OpenAIClient): return intermediateReply on user abort * fix(AskController): do not send/save final message if abort was triggered * fix(countTokens): avoid fetching remote registry and exclusively use cl100k_base or p50k_base weights for token counting * refactor(Message/messageSchema): rely on messageSchema for default values when saving messages * fix(EditController): do not send/save final message if abort was triggered * fix(config/helpers): fix module resolution error * update pull_request_template.md (danny-avila#1466) add "Translation update" the the PR type choices * 🌎: Update French Translation (danny-avila#1472) * 🌎: Update French Translation * 🌎: Update French Translation * 🌎: Update French Translation * 🌎: Update French Translation * 🌎: Update Portuguese Translation (danny-avila#1461) * 👮feat: moderation text (danny-avila#1388) * fixed some bugs and handling errors better * feat: plugins support * fix: prettier error message * moved circular-json-es6 in /api * docs: added openai moderation text * fix(gptPlugins): incorrect merge * discarding changes * removed circular-json-es6 * fix(OpenAIClient): do not invoke abortCompletion on completion error (danny-avila#1473) * Update dev-images.yml * Update backend-review.yml * remove 'MEILI_HTTP_ADDR' (danny-avila#1475) * 💫 feat: Config File & Custom Endpoints (danny-avila#1474) * WIP(backend/api): custom endpoint * WIP(frontend/client): custom endpoint * chore: adjust typedefs for configs * refactor: use data-provider for cache keys and rename enums and custom endpoint for better clarity and compatibility * feat: loadYaml utility * refactor: rename back to from and proof-of-concept for creating schemas from user-defined defaults * refactor: remove custom endpoint from default endpointsConfig as it will be exclusively managed by yaml config * refactor(EndpointController): rename variables for clarity * feat: initial load custom config * feat(server/utils): add simple `isUserProvided` helper * chore(types): update TConfig type * refactor: remove custom endpoint handling from model services as will be handled by config, modularize fetching of models * feat: loadCustomConfig, loadConfigEndpoints, loadConfigModels * chore: reorganize server init imports, invoke loadCustomConfig * refactor(loadConfigEndpoints/Models): return each custom endpoint as standalone endpoint * refactor(Endpoint/ModelController): spread config values after default (temporary) * chore(client): fix type issues * WIP: first pass for multiple custom endpoints - add endpointType to Conversation schema - add update zod schemas for both convo/presets to allow non-EModelEndpoint value as endpoint (also using type assertion) - use `endpointType` value as `endpoint` where mapping to type is necessary using this field - use custom defined `endpoint` value and not type for mapping to modelsConfig - misc: add return type to `getDefaultEndpoint` - in `useNewConvo`, add the endpointType if it wasn't already added to conversation - EndpointsMenu: use user-defined endpoint name as Title in menu - TODO: custom icon via custom config, change unknown to robot icon * refactor(parseConvo): pass args as an object and change where used accordingly; chore: comment out 'create schema' code * chore: remove unused availableModels field in TConfig type * refactor(parseCompactConvo): pass args as an object and change where used accordingly * feat: chat through custom endpoint * chore(message/convoSchemas): avoid saving empty arrays * fix(BaseClient/saveMessageToDatabase): save endpointType * refactor(ChatRoute): show Spinner if endpointsQuery or modelsQuery are still loading, which is apparent with slow fetching of models/remote config on first serve * fix(useConversation): assign endpointType if it's missing * fix(SaveAsPreset): pass real endpoint and endpointType when saving Preset) * chore: recorganize types order for TConfig, add `iconURL` * feat: custom endpoint icon support: - use UnknownIcon in all icon contexts - add mistral and openrouter as known endpoints, and add their icons - iconURL support * fix(presetSchema): move endpointType to default schema definitions shared between convoSchema and defaults * refactor(Settings/OpenAI): remove legacy `isOpenAI` flag * fix(OpenAIClient): do not invoke abortCompletion on completion error * feat: add responseSender/label support for custom endpoints: - use defaultModelLabel field in endpointOption - add model defaults for custom endpoints in `getResponseSender` - add `useGetSender` hook which uses EndpointsQuery to determine `defaultModelLabel` - include defaultModelLabel from endpointConfig in custom endpoint client options - pass `endpointType` to `getResponseSender` * feat(OpenAIClient): use custom options from config file * refactor: rename `defaultModelLabel` to `modelDisplayLabel` * refactor(data-provider): separate concerns from `schemas` into `parsers`, `config`, and fix imports elsewhere * feat: `iconURL` and extract environment variables from custom endpoint config values * feat: custom config validation via zod schema, rename and move to `./projectRoot/librechat.yaml` * docs: custom config docs and examples * fix(OpenAIClient/mistral): mistral does not allow singular system message, also add `useChatCompletion` flag to use openai-node for title completions * fix(custom/initializeClient): extract env var and use `isUserProvided` function * Update librechat.example.yaml * feat(InputWithLabel): add className props, and forwardRef * fix(streamResponse): handle error edge case where either messages or convos query throws an error * fix(useSSE): handle errorHandler edge cases where error response is and is not properly formatted from API, especially when a conversationId is not yet provided, which ensures stream is properly closed on error * feat: user_provided keys for custom endpoints * fix(config/endpointSchema): do not allow default endpoint values in custom endpoint `name` * feat(loadConfigModels): extract env variables and optimize fetching models * feat: support custom endpoint iconURL for messages and Nav * feat(OpenAIClient): add/dropParams support * docs: update docs with default params, add/dropParams, and notes to use config file instead of `OPENAI_REVERSE_PROXY` * docs: update docs with additional notes * feat(maxTokensMap): add mistral models (32k context) * docs: update openrouter notes * Update ai_setup.md * docs(custom_config): add table of contents and fix note about custom name * docs(custom_config): reorder ToC * Update custom_config.md * Add note about `max_tokens` field in custom_config.md * 🎨 style: Add Dynamic Height to Endpoint/Model Menus (danny-avila#1480) * style(EndpointsMenu): add scrolling and dynamic height * style(SelectDropDownPop): add dynamic height * 🔧 fix: langchain packages mismatch, mount config file for `deploy-compose.yaml`, silence config not found error (danny-avila#1481) * fix(api): version mismatch between langchain packages `@langchain/google-genai` & `langchain` * chore(loadYaml): silence config file not found error * chore: improve firebase init message when not configured (generalized) * fix(deploy-compose.yml): mount `librechat.yaml` config file * 🛠️ fix: Error Message Parsing and ChatOpenAI credentials (danny-avila#1482) * refactor(createLLM): ensure ChatOpenAI class always uses client-defined openAIApiKey; move typedefs to main def file * refactor(useSSE): improve error message parsing in error handler * 🧪 feat: Experimental: Enable Switching Endpoints Mid-Conversation (danny-avila#1483) * fix: load all existing conversation settings on refresh * refactor(buildDefaultConvo): use `lastConversationSetup.endpointType` before `conversation.endpointType` * refactor(TMessage/messageSchema): add `endpoint` field to messages to differentiate generation origin * feat(useNewConvo): `keepLatestMessage` param to prevent reseting the `latestMessage` mid-conversation * style(Settings): adjust height styling to allow more space in dialog for additional settings * feat: Modular Chat: experimental setting to Enable switching Endpoints mid-conversation * fix(ChatRoute): fix potential parsing issue with tPresetSchema * 🗨️ refactor: Open New Tab for Ctrl+Click or Button Combo in NewChat (danny-avila#1484) * 🔧 fix: Improve Endpoint Handling and Address Edge Cases (danny-avila#1486) * fix(TEndpointsConfig): resolve property access issues with typesafe helper function * fix: undefined or null endpoint edge case * refactor(mapEndpoints -> endpoints): renamed module to be more general for endpoint handling, wrote unit tests, export all helpers * refactor: Mount Config File in Docker and Add to .dockerignore (danny-avila#1493) * 🐳 : Improving Docker Build (danny-avila#1415) * Improving builds When adding LibreChat to ansible, it rebuilt way too often, even if I only changed the configuration. With this PR, it should build only when the files of the app change. Also removed the 'volumes' section for the 'api' in the docker-compose.yml. At least with our installation it works fine like this. * @danny-avila's comments - removed 'env_file' from docker-compose.yml - re-added link to '.env' in volumes * Adding latest changes from main * @danny-avila's comments * Updating installation instructions * @danny-avila's comments - Remove unused environment in docker-compose.yml - Re-add some steps for cleaning docker images * 🐳 : Further Docker build Cleanup & Docs Update (danny-avila#1502) * refactor: post-cleanup changes: - add more unnecessary paths to .dockerignore - remove librechat.yaml from main compose file (prevents from being required) - do not create librechat.yaml during build (does nothing) * docs: make config file instructions easier to read, more info throughout other docs * docs: add custom config to menu * Update custom_config.md * Update docker_compose_install.md * ⚖️ Update LICENSE.md Year: 2023 -> 2024 (danny-avila#1501) * docs: minor mkdocs formatting issue in custom_config.md * 🧹 docs: formatting fix for custom_config.md * ⬆️ fix(SendButton): correct dark theme bg when `enabled` (danny-avila#1503) * 🌎: Update Portuguese Translation (danny-avila#1505) * 🛠️ fix: Custom Endpoint issues, Improve SSE Response Handling (danny-avila#1510) * fix(custom): prevent presets using removed custom endpoints from causing frontend errors * refactor(abortMiddleware): send 204 status when abortController is not found/active, set expected header `application/json` when not set * fix(useSSE): general improvements: - Add endpointType to fetch URL in useSSE hook - use EndpointURLs enum - handle 204 response by setting `data` to initiated response - add better error handling UX, make clear when there is an explicit error * 🐛 fix: Load `dotenv` at Top of Call Stack (danny-avila#1512) * 🔄 refactor(config): Move `connectWithTimeout` Outside of Helpers Module (danny-avila#1513) * ✨ feat(saveImageFromUrl): Dynamic Extension Handling (danny-avila#1514) - Enhanced the `saveImageFromUrl` function to dynamically handle file extensions based on the content type of the fetched image. - Replaced the method of appending a '.png' extension with a more robust approach using regular expressions and the path module. - Allows for correct extension replacement or addition, ensuring filename consistency and compatibility with the actual image format. - Prevents issues with double extensions (e.g., 'someimage.jpg.png') and aligns saved file types with their respective content types. * 🔧 fix: Error Handling Improvements (danny-avila#1518) * style(Icon): remove error bubble from message icon * fix(custom): `initializeClient` now throws error if apiKey or baseURL are admin provided but no env var was found * refactor(tPresetSchema): match `conversationId` type to `tConversationSchema` but optional, use `extendedModelEndpointSchema` for `endpoint` * fix(useSSE): minor improvements - use `completed` set to avoid submitting unecessary abort request - set preset with `newConversation` calls using initial conversation settings to prevent default Preset override as well as default settings - return if there is a parsing error within `onerror` as expected errors from server are properly formatted * fix(getUserPluginAuthValue): throws error if no user matches (danny-avila#1522) * 🌍: Correct Localized Text of "Save & Submit" Button (danny-avila#1517) * To provide a full translation option for the button in other languages * Add a modification for the old version of the chat * 🌎: Update Portuguese Translation (danny-avila#1523) * 🌏: Update Zh.tsx (danny-avila#1527) Chinese Translation Update * 🚀 Feat: Streamline File Strategies & GPT-4-Vision Settings (danny-avila#1535) * chore: fix `endpoint` typescript issues and typo in console info message * feat(api): files GET endpoint and save only file_id references to messages * refactor(client): `useGetFiles` query hook, update file types, optimistic update of filesQuery on file upload * refactor(buildTree): update to use params object and accept fileMap * feat: map files to messages; refactor(ChatView): messages only available after files are fetched * fix: fetch files only when authenticated * feat(api): AppService - rename app.locals.configs to app.locals.paths - load custom config use fileStrategy from yaml config in app.locals * refactor: separate Firebase and Local strategies, call based on config * refactor: modularize file strategies and employ with use of DALL-E * refactor(librechat.yaml): add fileStrategy field * feat: add source to MongoFile schema, as well as BatchFile, and ExtendedFile types * feat: employ file strategies for upload/delete files * refactor(deleteFirebaseFile): add user id validation for firebase file deletion * chore(deleteFirebaseFile): update jsdocs * feat: employ strategies for vision requests * fix(client): handle messages with deleted files * fix(client): ensure `filesToDelete` always saves/sends `file.source` * feat(openAI): configurable `resendImages` and `imageDetail` * refactor(getTokenCountForMessage): recursive process only when array of Objects and only their values (not keys) aside from `image_url` types * feat(OpenAIClient): calculateImageTokenCost * chore: remove comment * refactor(uploadAvatar): employ fileStrategy for avatars, from social logins or user upload * docs: update docs on how to configure fileStrategy * fix(ci): mock winston and winston related modules, update DALLE3.spec.js with changes made * refactor(redis): change terminal message to reflect current development state * fix(DALL-E-2): pass fileStrategy to dall-e * 🚆docs: Railway support (danny-avila#1529) * Update README.md * Update README.md --------- Co-authored-by: Danny Avila <110412045+danny-avila@users.noreply.github.com> * 📄chore: Update Discord Link README.md * 🐛 fix: Remove `resendImages`, `imageDetail` from `modelOptions` for Custom Endpoints (danny-avila#1537) * 📑 docs: Update Mistral AI API example about `dropParams` (danny-avila#1538) * 🚆docs: update hetzner ubuntu example (danny-avila#1539) * 📔 docs: Update Render hosting Guide (danny-avila#1528) * add the IntelliJ Idea config file to .gitignore * Update the docs for using a user created key and restricting mongodb IP access to public IP addresses * 📑 docs: fix mistral ai api example `safe_mode` --> `safe_prompt` (danny-avila#1541) * 🏗️ fix(config): Resolve Paths Correctly for Helper Scripts on Linux and Windows (danny-avila#1542) * 🧹 docs: update docker_override.md (danny-avila#1546) minor fix * 📕 fix: Update `document` Title based on `appTitle` and on "New Chat" (danny-avila#1553) * 🐳 docs: Update dotenv.md (danny-avila#1564) docker compose is now a plugin of docker * 🎨: Updated Plugins Search Bar; 🌎: Added Translations (danny-avila#1549) * 🎨: layout search bar plugins: updated / 🌎: translation update * 🌎:Update Portuguese Translation * fix: Refactored 'pluginstoredialog' code. * chore(PopoverButtons): remove comments, re-organize imports * chore: linting and reorganize useState declarations * chore: linting and reorganize useState declarations --------- Co-authored-by: Danny Avila <110412045+danny-avila@users.noreply.github.com> Co-authored-by: Danny Avila <messagedaniel@protonmail.com> * 🌏: Add indonesia translation (danny-avila#1563) * Added Indonesian Language * added indonesian to eng.tsx and general.tsx * 🪙 refactor: Update tokens.js for Added Context Buffer from Max (danny-avila#1573) * Update tokens.js * chore: linting previous PR * chore: adjust token limits, add buffers * chore: linting * chore: adjust 32k gpt-4 limit * ✏️ feat: LaTeX parsing for Messages (danny-avila#1585) * feat: Beta features tab in Settings and LaTeX Parsing toggle * feat: LaTex parsing with spec * 🖼️ feat(DALL-E): Azure OpenAI Support & New Config Variables (danny-avila#1586) * feat(DALL-E-3/DALL-E-2): Azure OpenAI support. New Version specific environment credentials: - DALLEx_SYSTEM_PROMPT= - DALLEx_AZURE_API_VERSION= - DALLEx_BASEURL= - DALLEx_API_KEY= - replace `x` with `3` or `2` * docs: update docs based on new env vars and Azure OpenAI support for DALL-E * docs: breaking change for user provided DALLE_API_KEY: - **DALL-E Update**: user-provided keys for DALL-E are now specific to each DALL-E version, i.e.: and - Note: will work for both DALL-E-3 and DALL-E-2 when the admin provides the credential; in other words, this may only affect your users if DALLE_API_KEY is not set in the file. In this case, they will simply have to uninstall the plugin, and provide their API key again. * refactor: use process.env at runtime instead of from memory to fix testing DALLE3.spec.js, adjust test * 🎯 feat(config): Custom Endpoint Request Headers (danny-avila#1588) * 🚀 feat: Add `OPENAI_ORGANIZATION` for all OpenAI Requests (danny-avila#1590) * 📜 refactor: Enhance Auto Scroll Speed and UseEffect Cleanup (danny-avila#1591) * 🚆 docs: Add Sealos 1-Click Deployment (danny-avila#1578) Add deploy on Sealos * 🖌️ feat: Optional Display Username in Messages, Send/Stop Button Style, Localization (danny-avila#1592) * 👤add: Username instead of 'You' when sending messages. * 🌎: Added a new translation for 'You' and updated the existing translation for Spanish. * fix: remove "!" * Added: New setting Account for show username in messages chore (StopButon and SendButon): Updated to new style of ChatGPT chore Update and Added news translations: Spanish, English and Portuguese Brazilian * fix: message component definition and imports order, remove unnecessary useEffect and localStorage set, fix localStorage key in store * chore: update readme.md * chore: optimize condition for messageLabel * chore(Message.tsx): remove empty blocks --------- Co-authored-by: Raí Santos <140329135+itzraiss@users.noreply.github.com> * 🌎 : Update Portuguese Translation (danny-avila#1593) * 🌎: Update Portuguese Translation * 🌎: Update Portuguese Translation * 🪶 feat: Add Support for Azure OpenAI Base URL (danny-avila#1596) * refactor(extractBaseURL): add handling for all possible Cloudflare AI Gateway endpoints * chore: added endpointoption todo for updating type and optimizing handling app-wide * feat(azureUtils): - `genAzureChatCompletion`: allow optional client pass to update azure property - `constructAzureURL`: optionally replace placeholders for instance and deployment names of an azure baseURL - add tests for module * refactor(extractBaseURL): return entire input when cloudflare `azure-openai` suffix detected - also add more tests for both construct and extract URL * refactor(genAzureChatCompletion): only allow omitting instance name if baseURL is not set * refactor(initializeClient): determine `reverseProxyUrl` based on endpoint (azure or openai) * refactor: utitlize `constructAzureURL` when `AZURE_OPENAI_BASEURL` is set * docs: update docs on `AZURE_OPENAI_BASEURL` * fix(ci): update expected error message for `azureUtils` tests * 🌎: Update Italian Translation (danny-avila#1594) * translation update * translation updatre + new transaltions * Update language files for English and Italian * 🌍 : Translation update (Fr & It) (danny-avila#1595) * ✨ : Release v0.6.6 (danny-avila#1597) * 🧹 chore: remove deprecated model from documentation (danny-avila#1598) * 🐛 fix: Converted the getter `name` and `description` of Azure AI Search Plugin to a property (danny-avila#1599) * 📱🍎 feat: PWA on iOS (danny-avila#1602) * 🌍 : Update indonesian translation (danny-avila#1601) * 📱 fix(Root.tsx): Update styling to use `h-dvh` instead of `h-screen` (danny-avila#1608) * 🐋 feat: CI Docker builds for arm64v8 (danny-avila#1607) * 🛠️ fix: Preserve Dollar Signs in Code Blocks for LaTeX Parsing (danny-avila#1612) * 📖 docs: Update litellm.md to add Ollama (danny-avila#1616) * 🔏 feat: Nginx SSL Secure Config (danny-avila#1615) * Update nginx.conf Add Mozilla SSL Configuration Generator recommended options. * Update nginx.conf Remove Space --------- Co-authored-by: Fuegovic <32828263+fuegovic@users.noreply.github.com> Co-authored-by: Ed Burnette <ed.burnette@hiddenmind.ai> Co-authored-by: Danny Avila <110412045+danny-avila@users.noreply.github.com> Co-authored-by: Marco Beretta <81851188+Berry-13@users.noreply.github.com> Co-authored-by: Nikita Dybov <minecraft.klub@yandex.ru> Co-authored-by: Danny Avila <messagedaniel@protonmail.com> Co-authored-by: Linus Gasser <linus.gasser@epfl.ch> Co-authored-by: MACHINSOFT <110278369+machinsoft@users.noreply.github.com> Co-authored-by: Raí Santos <140329135+itzraiss@users.noreply.github.com> Co-authored-by: USAGI <124278082+snekkenull@users.noreply.github.com> Co-authored-by: Pascal Helfenstein <pascal.helfenstein@gmail.com> Co-authored-by: Tyler Mendenhall <tyler.mendenhall@e-gineering.com> Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com> Co-authored-by: Ganesh Krishnan <github@alphasigma.com.au> Co-authored-by: Wahit Fitriyanto <26300406+wahitftry@users.noreply.github.com> Co-authored-by: Carson Yang <yangchuansheng33@gmail.com> Co-authored-by: Naosuke Yokoe <ankerasoy@gmail.com> Co-authored-by: fidoriel <49869342+fidoriel@users.noreply.github.com> Co-authored-by: bsu3338 <bsu3338@users.noreply.github.com>
danny-avila
added a commit
to lgtm-hbu/LibreChat
that referenced
this pull request
Jan 31, 2024
danny-avila
added a commit
that referenced
this pull request
Feb 5, 2024
cnkang
pushed a commit
to cnkang/LibreChat
that referenced
this pull request
Feb 6, 2024
…la#1535) * chore: fix `endpoint` typescript issues and typo in console info message * feat(api): files GET endpoint and save only file_id references to messages * refactor(client): `useGetFiles` query hook, update file types, optimistic update of filesQuery on file upload * refactor(buildTree): update to use params object and accept fileMap * feat: map files to messages; refactor(ChatView): messages only available after files are fetched * fix: fetch files only when authenticated * feat(api): AppService - rename app.locals.configs to app.locals.paths - load custom config use fileStrategy from yaml config in app.locals * refactor: separate Firebase and Local strategies, call based on config * refactor: modularize file strategies and employ with use of DALL-E * refactor(librechat.yaml): add fileStrategy field * feat: add source to MongoFile schema, as well as BatchFile, and ExtendedFile types * feat: employ file strategies for upload/delete files * refactor(deleteFirebaseFile): add user id validation for firebase file deletion * chore(deleteFirebaseFile): update jsdocs * feat: employ strategies for vision requests * fix(client): handle messages with deleted files * fix(client): ensure `filesToDelete` always saves/sends `file.source` * feat(openAI): configurable `resendImages` and `imageDetail` * refactor(getTokenCountForMessage): recursive process only when array of Objects and only their values (not keys) aside from `image_url` types * feat(OpenAIClient): calculateImageTokenCost * chore: remove comment * refactor(uploadAvatar): employ fileStrategy for avatars, from social logins or user upload * docs: update docs on how to configure fileStrategy * fix(ci): mock winston and winston related modules, update DALLE3.spec.js with changes made * refactor(redis): change terminal message to reflect current development state * fix(DALL-E-2): pass fileStrategy to dall-e
danny-avila
added a commit
that referenced
this pull request
Feb 9, 2024
danny-avila
added a commit
that referenced
this pull request
Feb 11, 2024
danny-avila
added a commit
that referenced
this pull request
Feb 13, 2024
jinzishuai
pushed a commit
to aitok-ai/LibreChat
that referenced
this pull request
May 20, 2024
…la#1535) * chore: fix `endpoint` typescript issues and typo in console info message * feat(api): files GET endpoint and save only file_id references to messages * refactor(client): `useGetFiles` query hook, update file types, optimistic update of filesQuery on file upload * refactor(buildTree): update to use params object and accept fileMap * feat: map files to messages; refactor(ChatView): messages only available after files are fetched * fix: fetch files only when authenticated * feat(api): AppService - rename app.locals.configs to app.locals.paths - load custom config use fileStrategy from yaml config in app.locals * refactor: separate Firebase and Local strategies, call based on config * refactor: modularize file strategies and employ with use of DALL-E * refactor(librechat.yaml): add fileStrategy field * feat: add source to MongoFile schema, as well as BatchFile, and ExtendedFile types * feat: employ file strategies for upload/delete files * refactor(deleteFirebaseFile): add user id validation for firebase file deletion * chore(deleteFirebaseFile): update jsdocs * feat: employ strategies for vision requests * fix(client): handle messages with deleted files * fix(client): ensure `filesToDelete` always saves/sends `file.source` * feat(openAI): configurable `resendImages` and `imageDetail` * refactor(getTokenCountForMessage): recursive process only when array of Objects and only their values (not keys) aside from `image_url` types * feat(OpenAIClient): calculateImageTokenCost * chore: remove comment * refactor(uploadAvatar): employ fileStrategy for avatars, from social logins or user upload * docs: update docs on how to configure fileStrategy * fix(ci): mock winston and winston related modules, update DALLE3.spec.js with changes made * refactor(redis): change terminal message to reflect current development state * fix(DALL-E-2): pass fileStrategy to dall-e
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Users can now upload images to integrated CDN (firebase only at the moment), which allows them to send them as URLs to OpenAI Vision API. Files are no longer directly attached to messages. Code for CDN integration is now more succinct and modularized, allowing ease of development for new CDN integrations.
You now have an option to set the image detail for vision requests, as well as resending all previously attached messages to the latest message request.
More details:
New Features and Enhancements:
resendImages
andimageDetail
(26a15ac).Documentation and Refactoring:
getTokenCountForMessage
function for a more efficient recursive process, focusing on arrays of objects and their values (53eb2f8).buildTree
function to use a parameters object and accept a fileMap (e671f6f).Fixes and Improvements:
filesToDelete
always saves/sendsfile.source
(17112c3).useGetFiles
query hook and updated file types for better file management (6aeed6b).Code Maintenance:
deleteFirebaseFile
and added user ID validation for firebase file deletion (c05d3b0, f783926).librechat.yaml
to add a fileStrategy field (0f6cf2d).