Adds async processing, session management and a starter cloud setup #921
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… build directories from Docker image
🚧 feat(base.Dockerfile): create python-base stage with shared environment variables 🚧 feat(base.Dockerfile): create builder-base stage for building dependencies and creating virtual environment 🚧 feat(base.Dockerfile): create development stage for development and testing
…nd create_app function to improve modularity and maintainability
…ng Celery worker and Flower
…r bug fixes and improvements 📦 chore(pyproject.toml): add celery version 5.3.1, redis version 4.6.0, and flower version 2.0.1 to dev dependencies for task queue and monitoring
✨ feat(celery_app.py): add Celery app configuration for langflow with Redis as broker and backend ✨ feat(celeryconfig.py): add Celery configuration file with environment variable support for broker URL and result backend URL
…ngFlow backend 🔧 chore(worker.py): refactor process_graph_cached function to improve readability and maintainability
…replace it with 'process_graph_cached_worker' for processing graph data asynchronously 🔧 fix(endpoints.py): add error handling for processing tweaks and log the error message ✨ feat(endpoints.py): use 'process_graph_cached_worker' to process graph data asynchronously and return the result in 'ProcessResponse'
…ariables in cache module 🔄 refactor(cache): rename BaseCache class to BaseCacheManager for better clarity and consistency 🔄 refactor(cache): remove cache_manager from __all__ in cache module 🔄 refactor(cache): remove unused import of Service in BaseCacheManager class 🔄 refactor(cache): rename cache_manager to BaseCacheManager in factory module 🔄 refactor(cache): remove unused import of InMemoryCache in factory module 🔄 refactor(cache): remove flow module and its related code as it is no longer used 🔧 fix(manager.py): fix import statements and class inheritance in cache manager module ✨ feat(manager.py): add InMemoryCache class to implement a simple in-memory cache using an OrderedDict ✨ feat(manager.py): add RedisCache class to implement a Redis-based cache using the redis-py library
…tter organization and readability 🔀 refactor(base.py): rename CACHE_TYPE variable to REDIS_CACHE_TYPE to improve clarity 🔀 refactor(base.py): rename REDIS_HOST, REDIS_PORT, REDIS_DB, and REDIS_CACHE_EXPIRE variables to improve clarity 🔀 refactor(base.py): remove unused import statements 🔀 refactor(settings.py): remove unused import statements 🔀 refactor(settings.py): remove unused code and comments 🔀 refactor(settings.py): remove unused function `save_settings_to_yaml` 🔀 refactor(settings.py): remove unused function `update_settings` 🔀 refactor(settings.py): remove unused function `update_from_yaml` 🔀 refactor(settings.py): remove unused constant `BASE_COMPONENTS_PATH` 🔀 refactor(settings.py): remove unused class attributes 🔀 refactor(settings.py): remove unused class methods 🔀 refactor(settings.py): remove unused class `Settings`
…o improve code readability ✨ feat(chat.py): add support for cache_manager to handle chat history and cache langchain_object 🆕 feat(chat/manager.py): add ChatManager class to handle websocket connections and chat history 🆕 feat(chat/manager.py): add ChatHistory class to manage chat history for each client 🆕 feat(chat/manager.py): add methods to handle websocket connections, send messages, and process chat messages 🆕 feat(chat/manager.py): add method to set cache for a client and handle websocket communication
…n make_celery for improved modularity and readability 🔧 chore(celeryconfig.py): update broker_url and result_backend to use LANGFLOW_REDIS_HOST and LANGFLOW_REDIS_PORT environment variables if available, fallback to default values if not
…or clarity 🐛 fix(manager.py): add test connections to cache and database managers for initialization
…nt clients and notify observers on changes
…r organization and readability ✨ feat(manager.py): add support for chat cache and attach it to the chat manager for better chat history management 🔧 chore(manager.py): change client_id format in set_cache method to avoid conflicts with existing cache keys
…variables for redis cache and queue 🔧 fix(docker-compose.celery.yml): update uvicorn command to include debug log level 🔧 fix(docker-compose.celery.yml): update flower command to include redis broker URL 🔧 fix(docker-compose.celery.yml): add dependencies and environment variables for redis cache and queue in frontend service
🔒 chore(pyproject.toml): update celery, redis, and flower versions and make them optional dependencies 🔀 chore(pyproject.toml): add celery, redis, and flower to the deploy extra dependencies
…v dependencies and include deploy dependencies for production deployment
…ronment variable and its possible values
…better clarity and consistency ✨ feat(chat.py): update references to flow_data_store to cache_manager for improved semantics and readability 🐛 fix(chat.py): fix incorrect cache key in chat_manager.set_cache() method call 🐛 fix(chat.py): fix incorrect cache key in chat_manager.chat_history.empty_history() method call
…o update or insert cache value ✨ feat(manager.py): add result_dict to cache instead of directly setting langchain_object to improve cache semantics 🐛 fix(manager.py): handle case when langchain_object is not found in cache and raise ValueError
…common interface for inserting or updating cache items ✨ feat(manager.py): add upsert method to InMemoryCache and RedisCache to support inserting or updating cache items 🐛 fix(manager.py): fix client_id generation in handle_websocket method to avoid overwriting existing cache items 🔨 chore(manager.py): refactor handle_websocket method to use upsert method and improve readability
🔧 chore(manager.py): add try-except block to handle ImportError and provide clear error message when redis-py package is not installed 🔧 chore(manager.py): add import statement for redis module in the is_connected method to avoid NameError
…dicate if the vertex is a task ✨ feat(base.py): add `get_result` method to `Vertex` to retrieve the result of a built vertex 🐛 fix(types.py): pass `is_task=True` to `super().__init__` in `CustomComponentVertex` constructor ✨ feat(worker.py): add `build_vertex` task to build a vertex asynchronously
…' extras when installing dependencies with Poetry
…gle Cloud Platform 📝 docs(walkthroughtutorial.md): add tutorial for deploying Langflow on Google Cloud Platform The `deploy_langflow_gcp.sh` script is added to the `scripts/gcp` directory. This script sets up the necessary VM, image, and networking configuration for deploying Langflow on Google Cloud Platform. It also creates firewall rules, installs dependencies, and starts the Langflow server. The `walkthroughtutorial.md` file is added to the `scripts/gcp` directory. This file provides a step-by-step tutorial on how to deploy Langflow on Google Cloud Platform using the `deploy_langflow_gcp.sh` script. It includes instructions on setting up the GCP environment, running the script, and connecting to the Langflow VM. Cleanup instructions are also provided in the tutorial for removing the resources created during the deployment process. 📝 docs(walkthroughtutorial_spot.md): add walkthrough tutorial for deploying Langflow on Google Cloud Platform This commit adds a new file `walkthroughtutorial_spot.md` which contains a step-by-step tutorial for deploying Langflow on Google Cloud Platform (GCP) using Google Cloud Shell. The tutorial provides an introduction, prerequisites, instructions for setting up the GCP environment, deploying Langflow, connecting to the Langflow VM, and cleaning up resources after the deployment. The tutorial is authored by Robert Wilkins III and has a duration of 45 minutes. It assumes that the user has a GCP account and basic knowledge of Google Cloud Shell. The tutorial also includes links to relevant documentation and provides commands for configuring the GCP environment and deploying Langflow. The tutorial is comprehensive and aims to guide users through the process of deploying Langflow on GCP. It also includes cleanup instructions for removing the resources created during the tutorial if desired.
…t to result_dict["result"] to store the entire langchain_object instead
…n poetry install command to correctly install deploy extras
…o improve code cleanliness and maintainability 🔧 refactor(schemas.py): make the backend field in ProcessResponse schema optional to handle cases where backend is not provided 🔧 refactor(utils.py): add type hint ignore comment to import statement for Celery to avoid type checking error
…rove code readability and maintainability
…o provide environment variables for the build and start services step
…ile in the correct location
…end and broker services for testing environment
…ile and build the services
…mpose file before the up command
…se command to fix build and start services step in CI workflow
… inform users about potential issues and encourage reporting
…ine if new users should be active by default 🔧 fix(cache/manager.py): add warning log message to inform users that RedisCache is experimental and may not work as expected 🔧 fix(settings/auth.py): add NEW_USER_IS_ACTIVE setting to determine if new users should be active by default
…AI API key 🔥 refactor(docker-compose.celery.yml): remove unused celery service configuration
… Langflow 📝 docs(login.mdx): explain the purpose of login functionality and introduce enhanced login mechanism in Langflow 📝 docs(login.mdx): document the crucial environment variables for configuring the login settings in Langflow 📝 docs(login.mdx): provide information on how to disable automatic login and enforce user authentication in Langflow 📝 docs(login.mdx): explain the usage of LANGFLOW_SUPERUSER and LANGFLOW_SUPERUSER_PASSWORD environment variables for setting up a superuser in Langflow 📝 docs(login.mdx): document the usage of LANGFLOW_SECRET_KEY environment variable for encrypting the superuser's password in Langflow 📝 docs(login.mdx): explain the purpose of LANGFLOW_NEW_USER_IS_ACTIVE environment variable for automatically activating new users in Langflow 📝 docs(login.mdx): provide information on the command-line interface for managing superusers in Langflow 📝 docs(login.mdx): document the sign-up process in Langflow and provide an image of the sign-up page 📝 docs(login.mdx): explain how users can change their profile settings in Langflow and provide an image of the profile settings page 📝 docs(login.mdx): explain how the superuser can access the admin page in Langflow and provide an image of the admin page 📝 docs(superuser.mdx): add guide for superuser permissions in Langflow
…ettings.auth_settings.NEW_USER_IS_ACTIVE to fix incorrect attribute access
…clarity - Rename volume `./pgadmin` to `pgadmin-data` for better naming convention and clarity. - Rename volumes for RabbitMQ: - `./.docker/rabbitmq/etc/` to `rabbitmq_data` for consistency and clarity. - `./.docker/rabbitmq/data/` to `rabbitmq_data` for consistency and clarity. - `./.docker/rabbitmq/logs/` to `rabbitmq_log` for consistency and clarity. No functional changes, only volume name updates.
…ove startup order and reliability
…flicts with backend service 🔧 chore(docker-compose.override.yml): configure traefik labels for tests service to route traffic and set load balancer port
…onfiguration 🔧 chore(docker-compose.test.yml): add 'tests' service configuration to run pytest
…ers function 🔧 chore(api_key.py): add cascade delete to user-api_key relationship to ensure API keys are deleted when user is deleted
…on call to correctly pass the index parameter The getUsersPage function was being called with the incorrect index parameter. It was off by one, causing the wrong page of users to be fetched. This has been fixed by subtracting 1 from the index parameter before calling the function.
…or better flexibility in updating user information
…ion to provide more detailed error messages 🐛 fix(users.py): only hash password if user is a superuser to prevent regular users from changing their password
…ve code readability and maintainability 🔧 fix(user.py): add sa_relationship_kwargs to api_keys relationship to enable cascading delete when deleting a user
… block to ensure proper error handling ✨ feat(login.py): add endpoint for auto login to improve user experience
…event overriding the original exception
The login functionality in Langflow serves to authenticate users and protect sensitive routes in the application. This commit adds documentation for the login functionality, including information about the enhanced login mechanism introduced in version 0.5. It explains the environment variables that are crucial in configuring the login settings and provides instructions on how to set them securely. It also covers the command-line interface for managing superusers and provides details on the sign-up process and profile settings for users. Screenshots are included to illustrate the sign-up page, profile settings page, and admin page.
…login guide from step-by-step guides 📚 docs(sidebars.js): add missing login and api guidelines to the guidelines section and remove login guide from step-by-step guides
…orarily remove it from the sidebar ℹ️ The "guidelines/api" section is temporarily removed from the sidebar to prevent confusion and ensure consistency while it is being updated or revised.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds the ability to use Redis as cache, Celery for task execution, Session management, async tasks for the API and some other improvements.
Run the following command after setting up the .env file in
./deployfolder: