Skip to content

Latest commit

 

History

History
174 lines (117 loc) · 5.34 KB

DEVELOPER.md

File metadata and controls

174 lines (117 loc) · 5.34 KB
Raw

DEVELOPER.md

Before you begin

  1. Make sure you've setup and initialized your Database.

  2. Install Python 3.11+

  3. Install dependencies. We recommend using a virtualenv:

    pip install -r retrieval_service/requirements.txt -r langchain_tools_demo/requirements.txt

  4. Install test dependencies:

    pip install -r retrieval_service/requirements-test.txt -r langchain_tools_demo/requirements-test.txt

Run the app locally

Running the retrieval service

  1. Change into the service directory:

    cd retrieval_service

  2. Open a local connection to your database by starting the Cloud SQL Auth Proxy or a SSH tunnel to your AlloyDB instance.

  3. You should already have a config.yml created with your database config. Continue to use host: 127.0.0.1 and port: 5432, unless you instruct the proxy to listen or the SSH tunnel to forward to a different address.

  4. To run the app using uvicorn, execute the following:

    python run_app.py

Running the frontend

  1. Change into the demo directory:

    cd langchain_tools_demo

  2. To use a live retrieval service on Cloud Run:

    1. Set Google user credentials:

      gcloud auth login

    2. Set BASE_URL environment variable:

      export BASE_URL=$(gcloud run services describe retrieval-service --format 'value(status.url)')

    3. Allow your account to invoke the Cloud Run service by granting the role Cloud Run invoker

  3. [Optional] Turn on debugging by setting the DEBUG environment variable:

    export DEBUG=True

  4. To run the app using uvicorn, execute the following:

    python main.py

    Note: for hot reloading of the app use: uvicorn main:app --host 0.0.0.0 --reload

  5. View app at http://localhost:8081/

Testing

Run tests locally

  1. Change into the retrieval_service directory

  2. Open a local connection to your database by starting the Cloud SQL Auth Proxy or a SSH tunnel to your AlloyDB instance.

  3. Set environment variables:

    export DB_USER=""
export DB_PASS=""
export DB_NAME=""

  4. Run pytest to automatically run all tests:

    pytest

CI Platform Setup

Cloud Build is used to run tests against Google Cloud resources in test project: extension-demo-testing. Each test has a corresponding Cloud Build trigger, see all triggers.

Trigger Setup

Create a Cloud Build trigger via the UI or gcloud with the following specs:

  • Event: Pull request
  • Region:
    • us-central1 - for AlloyDB to connect to private pool in VPC
    • global - for default worker pools
  • Source:
    • Generation: 1st gen
    • Repo: GoogleCloudPlatform/genai-databases-retrieval-app (GitHub App)
    • Base branch: ^main$
  • Comment control: Required except for owners and collaborators
  • Filters: add directory filter
  • Config: Cloud Build configuration file
    • Location: Repository (add path to file)
  • Substitution variables:
    • Add _DATABASE_HOST for AlloyDB IP address
  • Service account: set for demo service to enable ID token creation to use to authenticated services

Project Setup

  1. Follow instructions to setup the test project:
  2. Setup Cloud Build triggers (above)
Setup for retrieval service
  1. Create a Cloud Build private pool
  2. Enable Secret Manager API
  3. Create secret, alloy_db_pass, with your AlloyDB password
  4. Create secret, alloy_db_user, with your AlloyDB user
  5. Allow Cloud Build to access secret
  6. Add role Vertex AI User (roles/aiplatform.user) to Cloud Build Service account. Needed to run database init script.
Setup for demo service tests
  1. Add roles Cloud Run Admin, Service Account User, Log Writer, and Artifact Registry Admin to the demo service's Cloud Build trigger service account.

Run tests with Cloud Build

  • Run Demo Service integration test:

    gcloud builds submit --config langchain_tools_demo/int.tests.cloudbuild.yaml

  • Run retrieval service unit tests:

    gcloud builds submit --config retrieval_service/alloydb.tests.cloudbuild.yaml \
    --substitutions _DATABASE_HOST=$DB_HOST,_DATABASE_NAME=$DB_NAME,_DATABASE_USER=$DB_USER

    Where $DB_HOST,$DB_NAME,$DB_USER are environment variables with your database values.

    Note: Make sure to setup secrets describe in Setup for retrieval service

Trigger

To run Cloud Build tests on GitHub from external contributors, ie RenovateBot, comment: /gcbrun.