Skip to content

Commit

Permalink
chore: Add reload flag to run_app.py and update docs (#360)
Browse files Browse the repository at this point in the history
Several things:
1. After refactoring app.py (deleted main.py) in previous PRs, the
uvicorn hot loading command in the instruction no longer works.
Therefore I created a `--reload` flag in run_app.py that enables uvicorn
hot reloading functionality.
2. Updated this command in DEVELOPER.md.
3. Fixes grammar mistakes in docs.

---------

Co-authored-by: Yuan <45984206+Yuan325@users.noreply.github.com>
  • Loading branch information
duwenxin99 and Yuan325 committed Jun 12, 2024
1 parent de9f122 commit 123accc
Show file tree
Hide file tree
Showing 4 changed files with 17 additions and 10 deletions.
2 changes: 1 addition & 1 deletion DEVELOPER.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@
python run_app.py
```

Note: for hot reloading of the app use: `uvicorn main:app --host 0.0.0.0 --reload --port 8081`
Note: for hot reloading of the app use: `python run_app.py --reload`

1. View app at `http://localhost:8081/`

Expand Down
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ fictional passenger airline. The assistant is an AI chatbot that helps
travellers manage flights and look up information about Cymbal Air's hub
at San Francisco International Airport (SFO).

It can help answer users questions like:
It can help answer users' questions like:
* Are there any luxury shops?
* Where can I get coffee near gate A6?
* Where can I find a gift?
Expand Down Expand Up @@ -81,9 +81,9 @@ and decide when it needs to access it.
![Overview](./architecture.svg)

This demo contains 3 key parts:
1. **Application** -- The LLM-based app that acts as orchestrates layer for the
1. **Application** -- The LLM-based app that acts as the orchestrating layer for the
interaction with the LLM.
1. **Retrieval Service** -- The retrieval service provides the application
1. **Retrieval Service** -- The retrieval service provides the application with
concrete, discrete actions that allow the LLM to interact with the Database.
1. **Database** -- The database containing the data the LLM can use to answer
questions. For this application, the database used was intentionally designed
Expand All @@ -98,7 +98,7 @@ help address a number of challenges
leverage it successfully.
1. **Better scalability** - Running the retrieval as a separate service both
allows multiple different LLMs to leverage it, as well as allowing it to
scale independently. It allows allows production best practices such as
scale independently. It allows for production best practices such as
connection pooling or caching.
1. **Better security** - LLMs are susceptible to attacks such as "jailbreaking"
to circumvent safety measures that are put in place. Using an intermediary
Expand All @@ -124,7 +124,7 @@ git clone https://github.com/GoogleCloudPlatform/genai-databases-retrieval-app.g
### Setting up your Database

The retrieval service uses an interchangeable 'datastore' interface. Choose one
of any of the database's listed below to set up and initialize your database:
of the databases listed below to set up and initialize your database:

* [Set up and configure AlloyDB with public IP](./docs/datastore/alloydb.md)
* [Set up and configure Cloud SQL](./docs/datastore/cloudsql_postgres.md)
Expand All @@ -151,7 +151,7 @@ service. The directory is organized into the following folders:
| Directory | Description |
|----------------------------------------------|---------------------------------------------------------------------------------------|
| [`data`](/data) | Contains CSV files with the dataset for a working demo. |
| [`llm_demo`](/llm_demo) | Contains an LLM-based application that that uses the retrieval service via multiple orchestrator (e.g. LangChain, VertexAI). |
| [`llm_demo`](/llm_demo) | Contains an LLM-based application that uses the retrieval service via multiple orchestrator (e.g. LangChain, VertexAI). |
| [`retrieval_service`](/retrieval_service) | Contains the service for extending an LLM with information from the database. |

You can copy or fork the `retrieval_service` folder to customize it to your
Expand Down
2 changes: 1 addition & 1 deletion docs/run_llm_demo.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,6 @@
python run_app.py
```

Note: for hot reloading of the app use: `uvicorn main:app --host 0.0.0.0 --reload`
Note: for hot reloading of the app use: `python run_app.py --reload`

1. View app at `http://localhost:8081/`
11 changes: 9 additions & 2 deletions retrieval_service/run_app.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.


import argparse
import asyncio

import uvicorn
Expand All @@ -21,12 +21,19 @@


async def main():
# Set up argument parsing
parser = argparse.ArgumentParser(description="Run the FastAPI application")
parser.add_argument("--reload", action="store_true", help="Enable auto-reload")
args = parser.parse_args()

cfg = parse_config("./config.yml")
app = init_app(cfg)
if app is None:
raise TypeError("app not instantiated")
server = uvicorn.Server(
uvicorn.Config(app, host=str(cfg.host), port=cfg.port, log_level="info")
uvicorn.Config(
app, host=str(cfg.host), port=cfg.port, log_level="info", reload=args.reload
)
)
await server.serve()

Expand Down

0 comments on commit 123accc

Please sign in to comment.