Make dashboard multi-user + some admin app cleanup #213
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
amiraliemami
force-pushed
the
multitenancy/dashboard
branch
from
6562852
to
625520b
Compare
amiraliemami
force-pushed
the
multitenancy/dashboard
branch
from
625520b
to
ce1c5b4
Compare
amiraliemami
changed the base branch from
multitenancy/feature
to
multitenancy/tests
amiraliemami
changed the title
suzinyou
approved these changes
May 17, 2024
amiraliemami
force-pushed
the
multitenancy/dashboard
branch
2 times, most recently
from
e531fa8
to
1575208
Compare
amiraliemami
added a commit
that referenced
this pull request
May 28, 2024
* add postgres vars to template.env for clarity of use * remove deprecated "version" tag from local docker-compose scripts * change fullaccess and readonly users to user1 and user2 with fullaccess permissions * clarify `NEXT_PUBLIC_BACKEND_URL` variable in api.ts * Add user_id column to ContentDB and update content endpoints. NOTE: user_id hardcoded into QA and WhatsApp endpoints - to be inferred from bearer token later. * Admin app: Show username in profile dropdown * Disable "readonly" tests. Will add back in next PR. * Fix mismatch between get_llm_answer() and its decorators' expectation of a "question" parameter * Undo dropping of the `content_idx` index. Fix source issue later. * Remove duplicated unused AccessLevel declaration in contents schema * Add user table to db * Update backend to use new users table * Change content table user_id column to foreign key * Update env var names * Fix "question" parameter in get_semantic_matches() to match its decorators' expectation * Hardcode QA endpoints to use "user1"s data. Will make dynamic in next PR. * Disable WhatsApp pytests - WhatsApp endpoint is broken due to no access to auth information * Make initial user-loading script standalone and use it in dev setup and update Dockerfile deployment * Make usernames environment variables too * Add user addition in DB to pytests * update default usernames to user1 and user2 * create_content() description fix * Use `.one()` instead of `.scalar_one()` and add MultipleResultsFound exception * Change "user-queries" table to "query". Prevents confusion with AAQ users. * Table "user-query-responses" -> "query-response" * Table "user-query-response-errors" -> "query-response-error" * Table "response_feedback" -> "query-response-feedback" * Table "content_feedback" -> "content-feedback" * Standardise migration file messages * Stardise migration file names (no functionality change) * Fix feedback tables index col names * Remove "User" prefix from Query, QueryResponse, QueryResponseError table models * Remove "User" prefix from content schemas * Add "retrieval_key" column to user table and update QA endpoints to be user-aware * Update typebot flow with "RETRIEVAL_KEY" wording * Update pytest * Add user_id column to query table and store which user's account each query is associated with * Update feedback endpoints to be user-aware * Catch rogue outdated "UserQueryBase" * Implementing hashing of keys and update DB and tests * update migration filename * add key updation endpoint * Make keys 32 bytes and remove unnecessary character count check * Remove unnecessary spaces in Edit page * BUGFIX: Background on playground is always white even after scroll * Add tags to WhatsApp and Auth endpoint routers * Add temporary password column and move auth env vars to main config.py * [temp] disable new key route to avoid circular import * Temp: Add non-generator async session maker * MAJOR: Auth now checks DB for user info and returns user_db if found. Fullaccess and readonly removed. * Remove mentions of access levels from docs * correctly close db sessions in auth * Remove "readonly" tests and test constructions * organise imports * Make "/key" rotator endpoint functional again (avoiding circular imports 😭) * change KeyResponse parameter * Give tests their own user id/name/password/retrieval key * Remove readonly as an accepted option for access_level * Move number of bytes to top of file * Remove leftover comments * Move username-password-key env vars to loading script instead of configs.py * Don't use catch-all "except Exception" * Use `get_sqlalchemy_async_engine()` with `with` directly in auth code to ensure sessions are closed correctly * Make username and key columns unique and remove MultipleResults errors * Remove leftover .aclose() statements * Make UD multi-user and update tests * add user_id to contentdb pretty print * Cleanup UD parameters and add keywords to function calls * fix migration version after rebase * add line length for ruff * Rename authentication functions * Combine migrations (#197) * Combine pre-multiuser migrations that could be combined * Combine multiuser migrations * standardise wording of migration * add date template for migration filenames * Add dates to migration filenames * Fix "NEXT_PUBLIC_BACKEND_URL" in api.ts (UD merge issue) * Fix merge issues * add names to foreign key migrations * fix auth on page * Google login (#205) * first go at google auth * fixed env vars * updated tests * removed print statement * updated script to add users * added stubs to requirements * deployment updates * Update admin_app/src/app/login/page.tsx Co-authored-by: Amir Emami <41763233+amiraliemami@users.noreply.github.com> * fixes from PR * fixes from PR * skipping type checking for google auth --------- Co-authored-by: Amir Emami <41763233+amiraliemami@users.noreply.github.com> * HOTFIX: Bring unsalted hashing back for retrieval keys * HOTFIX: Mention Google login client ID env var in dev docs * HOTFIX: env var name in api.ts * Key generation UI (#211) * Correct usage of "startIcon" in New buttons * Add API key generation to front-end * Move modals to components file and other improvements * Add link to integrations page in Playground API key popup * Remove mentions of "current key" for now * Make copy button visually responsive too * Improve spacing on new key modal * Clean up modal button margins * fix format * Add user pytests (#206) * Add key management endpoint pytest * TEMP: Add test for user functions (not working due to async issue) * fix scope of asession in test_urgency_detect (future-proofing) * Fix user tests with async session * Add where to set/find login credentials to docs * Temporary fix for rotation endpoint test * Fix scope mismatch (some UD tests break) * Add multi-user tests for content CRUD * Add multi-user tests for UD CRUD + add fetch check first in delete endpoint * Add user2 token denial test to retrieval and UD detect * Put multi-user tests under a different class * fixed async tests in session scope issue * Make test user_ids integers to match new schema * Fix handling of user_ids in pytest * Separate content CRUD multi-user tests and use fixture * Separate UD CRUD multi-user tests and use fixture --------- Co-authored-by: Sid Ravinutala <sid.ravi1@gmail.com> * Make dashboard multi-user + some admin app cleanup (#213) * Add key management endpoint pytest * TEMP: Add test for user functions (not working due to async issue) * fix scope of asession in test_urgency_detect (future-proofing) * Fix user tests with async session * Add where to set/find login credentials to docs * Temporary fix for rotation endpoint test * Fix scope mismatch (some UD tests break) * Add multi-user tests for content CRUD * Add multi-user tests for UD CRUD + add fetch check first in delete endpoint * Add user2 token denial test to retrieval and UD detect * Put multi-user tests under a different class * fixed async tests in session scope issue * Make test user_ids integers to match new schema * Fix handling of user_ids in pytest * Separate content CRUD multi-user tests and use fixture * Separate UD CRUD multi-user tests and use fixture * Playground: Add type of search to user message display * Make dashboard functions filter on user_id * Playground: change "JSON" to "<json>" in error message response. * UD page: Make background white for now * move key_management tests to run last :) * Remove incorrect retrieval tests * improve query type formatting and use save button for ud rule --------- Co-authored-by: Sid Ravinutala <sid.ravi1@gmail.com> Co-authored-by: Suzin You <7042047+suzinyou@users.noreply.github.com> * Add pytests to check user1-user2 overlap in retrieval and urgency detect * Update AWS terraform and secret loading (#217) * Change all instances of "retrieval key" to "api key" * Add new user and google client ID env vars to terraform and bootstrap * fix spacing error * [AAQ-482] Add custom embeddings as optional components (#201) * add embeddings * Update docker-compose * update embeddings app * Add make command * Add default embedding * Add docstring * add as an optional component * Fix UD migration issue * add embeddings endpoint to llm proxy * Update optional_components/embeddings/requirements.txt Co-authored-by: Suzin You <7042047+suzinyou@users.noreply.github.com> * update docker-compose * Remove completion endpoint * fix linting * Fix PR comments * change size of vector --------- Co-authored-by: Suzin You <7042047+suzinyou@users.noreply.github.com> * add workflow dispatch force redeployment to admin app cicd * fix command * add manual force creation of task def to all other cicd scripts --------- Co-authored-by: Suzin You <7042047+suzinyou@users.noreply.github.com> Co-authored-by: lickem22 <44327443+lickem22@users.noreply.github.com> --------- Co-authored-by: Sid Ravinutala <sid.ravi1@gmail.com> Co-authored-by: Suzin You <7042047+suzinyou@users.noreply.github.com> Co-authored-by: lickem22 <44327443+lickem22@users.noreply.github.com>
lickem22
added a commit
that referenced
this pull request
May 28, 2024
* add postgres vars to template.env for clarity of use * remove deprecated "version" tag from local docker-compose scripts * change fullaccess and readonly users to user1 and user2 with fullaccess permissions * clarify `NEXT_PUBLIC_BACKEND_URL` variable in api.ts * Add user_id column to ContentDB and update content endpoints. NOTE: user_id hardcoded into QA and WhatsApp endpoints - to be inferred from bearer token later. * Admin app: Show username in profile dropdown * Disable "readonly" tests. Will add back in next PR. * Fix mismatch between get_llm_answer() and its decorators' expectation of a "question" parameter * Undo dropping of the `content_idx` index. Fix source issue later. * Remove duplicated unused AccessLevel declaration in contents schema * Add user table to db * Update backend to use new users table * Change content table user_id column to foreign key * Update env var names * Fix "question" parameter in get_semantic_matches() to match its decorators' expectation * Hardcode QA endpoints to use "user1"s data. Will make dynamic in next PR. * Disable WhatsApp pytests - WhatsApp endpoint is broken due to no access to auth information * Make initial user-loading script standalone and use it in dev setup and update Dockerfile deployment * Make usernames environment variables too * Add user addition in DB to pytests * update default usernames to user1 and user2 * create_content() description fix * Use `.one()` instead of `.scalar_one()` and add MultipleResultsFound exception * Change "user-queries" table to "query". Prevents confusion with AAQ users. * Table "user-query-responses" -> "query-response" * Table "user-query-response-errors" -> "query-response-error" * Table "response_feedback" -> "query-response-feedback" * Table "content_feedback" -> "content-feedback" * Standardise migration file messages * Stardise migration file names (no functionality change) * Fix feedback tables index col names * Remove "User" prefix from Query, QueryResponse, QueryResponseError table models * Remove "User" prefix from content schemas * Add "retrieval_key" column to user table and update QA endpoints to be user-aware * Update typebot flow with "RETRIEVAL_KEY" wording * Update pytest * Add user_id column to query table and store which user's account each query is associated with * Update feedback endpoints to be user-aware * Catch rogue outdated "UserQueryBase" * Implementing hashing of keys and update DB and tests * update migration filename * add key updation endpoint * Make keys 32 bytes and remove unnecessary character count check * Remove unnecessary spaces in Edit page * BUGFIX: Background on playground is always white even after scroll * Add tags to WhatsApp and Auth endpoint routers * Add temporary password column and move auth env vars to main config.py * [temp] disable new key route to avoid circular import * Temp: Add non-generator async session maker * MAJOR: Auth now checks DB for user info and returns user_db if found. Fullaccess and readonly removed. * Remove mentions of access levels from docs * correctly close db sessions in auth * Remove "readonly" tests and test constructions * organise imports * Make "/key" rotator endpoint functional again (avoiding circular imports 😭) * change KeyResponse parameter * Give tests their own user id/name/password/retrieval key * Remove readonly as an accepted option for access_level * Move number of bytes to top of file * Remove leftover comments * Move username-password-key env vars to loading script instead of configs.py * Don't use catch-all "except Exception" * Use `get_sqlalchemy_async_engine()` with `with` directly in auth code to ensure sessions are closed correctly * Make username and key columns unique and remove MultipleResults errors * Remove leftover .aclose() statements * Make UD multi-user and update tests * add user_id to contentdb pretty print * Cleanup UD parameters and add keywords to function calls * fix migration version after rebase * add line length for ruff * Rename authentication functions * Combine migrations (#197) * Combine pre-multiuser migrations that could be combined * Combine multiuser migrations * standardise wording of migration * add date template for migration filenames * Add dates to migration filenames * Fix "NEXT_PUBLIC_BACKEND_URL" in api.ts (UD merge issue) * Fix merge issues * add names to foreign key migrations * fix auth on page * Google login (#205) * first go at google auth * fixed env vars * updated tests * removed print statement * updated script to add users * added stubs to requirements * deployment updates * Update admin_app/src/app/login/page.tsx Co-authored-by: Amir Emami <41763233+amiraliemami@users.noreply.github.com> * fixes from PR * fixes from PR * skipping type checking for google auth --------- Co-authored-by: Amir Emami <41763233+amiraliemami@users.noreply.github.com> * HOTFIX: Bring unsalted hashing back for retrieval keys * HOTFIX: Mention Google login client ID env var in dev docs * HOTFIX: env var name in api.ts * Key generation UI (#211) * Correct usage of "startIcon" in New buttons * Add API key generation to front-end * Move modals to components file and other improvements * Add link to integrations page in Playground API key popup * Remove mentions of "current key" for now * Make copy button visually responsive too * Improve spacing on new key modal * Clean up modal button margins * fix format * Add user pytests (#206) * Add key management endpoint pytest * TEMP: Add test for user functions (not working due to async issue) * fix scope of asession in test_urgency_detect (future-proofing) * Fix user tests with async session * Add where to set/find login credentials to docs * Temporary fix for rotation endpoint test * Fix scope mismatch (some UD tests break) * Add multi-user tests for content CRUD * Add multi-user tests for UD CRUD + add fetch check first in delete endpoint * Add user2 token denial test to retrieval and UD detect * Put multi-user tests under a different class * fixed async tests in session scope issue * Make test user_ids integers to match new schema * Fix handling of user_ids in pytest * Separate content CRUD multi-user tests and use fixture * Separate UD CRUD multi-user tests and use fixture --------- Co-authored-by: Sid Ravinutala <sid.ravi1@gmail.com> * Make dashboard multi-user + some admin app cleanup (#213) * Add key management endpoint pytest * TEMP: Add test for user functions (not working due to async issue) * fix scope of asession in test_urgency_detect (future-proofing) * Fix user tests with async session * Add where to set/find login credentials to docs * Temporary fix for rotation endpoint test * Fix scope mismatch (some UD tests break) * Add multi-user tests for content CRUD * Add multi-user tests for UD CRUD + add fetch check first in delete endpoint * Add user2 token denial test to retrieval and UD detect * Put multi-user tests under a different class * fixed async tests in session scope issue * Make test user_ids integers to match new schema * Fix handling of user_ids in pytest * Separate content CRUD multi-user tests and use fixture * Separate UD CRUD multi-user tests and use fixture * Playground: Add type of search to user message display * Make dashboard functions filter on user_id * Playground: change "JSON" to "<json>" in error message response. * UD page: Make background white for now * move key_management tests to run last :) * Remove incorrect retrieval tests * improve query type formatting and use save button for ud rule --------- Co-authored-by: Sid Ravinutala <sid.ravi1@gmail.com> Co-authored-by: Suzin You <7042047+suzinyou@users.noreply.github.com> * Add pytests to check user1-user2 overlap in retrieval and urgency detect * Update AWS terraform and secret loading (#217) * Change all instances of "retrieval key" to "api key" * Add new user and google client ID env vars to terraform and bootstrap * fix spacing error * [AAQ-482] Add custom embeddings as optional components (#201) * add embeddings * Update docker-compose * update embeddings app * Add make command * Add default embedding * Add docstring * add as an optional component * Fix UD migration issue * add embeddings endpoint to llm proxy * Update optional_components/embeddings/requirements.txt Co-authored-by: Suzin You <7042047+suzinyou@users.noreply.github.com> * update docker-compose * Remove completion endpoint * fix linting * Fix PR comments * change size of vector --------- Co-authored-by: Suzin You <7042047+suzinyou@users.noreply.github.com> * add workflow dispatch force redeployment to admin app cicd * fix command * add manual force creation of task def to all other cicd scripts --------- Co-authored-by: Suzin You <7042047+suzinyou@users.noreply.github.com> Co-authored-by: lickem22 <44327443+lickem22@users.noreply.github.com> --------- Co-authored-by: Sid Ravinutala <sid.ravi1@gmail.com> Co-authored-by: Suzin You <7042047+suzinyou@users.noreply.github.com> Co-authored-by: lickem22 <44327443+lickem22@users.noreply.github.com>
lickem22
added a commit
that referenced
this pull request
May 30, 2024
* add postgres vars to template.env for clarity of use * remove deprecated "version" tag from local docker-compose scripts * change fullaccess and readonly users to user1 and user2 with fullaccess permissions * clarify `NEXT_PUBLIC_BACKEND_URL` variable in api.ts * Add user_id column to ContentDB and update content endpoints. NOTE: user_id hardcoded into QA and WhatsApp endpoints - to be inferred from bearer token later. * Admin app: Show username in profile dropdown * Disable "readonly" tests. Will add back in next PR. * Fix mismatch between get_llm_answer() and its decorators' expectation of a "question" parameter * Undo dropping of the `content_idx` index. Fix source issue later. * Remove duplicated unused AccessLevel declaration in contents schema * Add user table to db * Update backend to use new users table * Change content table user_id column to foreign key * Update env var names * Fix "question" parameter in get_semantic_matches() to match its decorators' expectation * Hardcode QA endpoints to use "user1"s data. Will make dynamic in next PR. * Disable WhatsApp pytests - WhatsApp endpoint is broken due to no access to auth information * Make initial user-loading script standalone and use it in dev setup and update Dockerfile deployment * Make usernames environment variables too * Add user addition in DB to pytests * update default usernames to user1 and user2 * create_content() description fix * Use `.one()` instead of `.scalar_one()` and add MultipleResultsFound exception * Change "user-queries" table to "query". Prevents confusion with AAQ users. * Table "user-query-responses" -> "query-response" * Table "user-query-response-errors" -> "query-response-error" * Table "response_feedback" -> "query-response-feedback" * Table "content_feedback" -> "content-feedback" * Standardise migration file messages * Stardise migration file names (no functionality change) * Fix feedback tables index col names * Remove "User" prefix from Query, QueryResponse, QueryResponseError table models * Remove "User" prefix from content schemas * Add "retrieval_key" column to user table and update QA endpoints to be user-aware * Update typebot flow with "RETRIEVAL_KEY" wording * Update pytest * Add user_id column to query table and store which user's account each query is associated with * Update feedback endpoints to be user-aware * Catch rogue outdated "UserQueryBase" * Implementing hashing of keys and update DB and tests * update migration filename * add key updation endpoint * Make keys 32 bytes and remove unnecessary character count check * Remove unnecessary spaces in Edit page * BUGFIX: Background on playground is always white even after scroll * Add tags to WhatsApp and Auth endpoint routers * Add temporary password column and move auth env vars to main config.py * [temp] disable new key route to avoid circular import * Temp: Add non-generator async session maker * MAJOR: Auth now checks DB for user info and returns user_db if found. Fullaccess and readonly removed. * Remove mentions of access levels from docs * correctly close db sessions in auth * Remove "readonly" tests and test constructions * organise imports * Make "/key" rotator endpoint functional again (avoiding circular imports 😭) * change KeyResponse parameter * Give tests their own user id/name/password/retrieval key * Remove readonly as an accepted option for access_level * Move number of bytes to top of file * Remove leftover comments * Move username-password-key env vars to loading script instead of configs.py * Don't use catch-all "except Exception" * Use `get_sqlalchemy_async_engine()` with `with` directly in auth code to ensure sessions are closed correctly * Make username and key columns unique and remove MultipleResults errors * Remove leftover .aclose() statements * Make UD multi-user and update tests * add user_id to contentdb pretty print * Cleanup UD parameters and add keywords to function calls * fix migration version after rebase * add line length for ruff * Rename authentication functions * Combine migrations (#197) * Combine pre-multiuser migrations that could be combined * Combine multiuser migrations * standardise wording of migration * add date template for migration filenames * Add dates to migration filenames * Fix "NEXT_PUBLIC_BACKEND_URL" in api.ts (UD merge issue) * Add content tag models * Fix merge issues * add names to foreign key migrations * fix auth on page * Google login (#205) * first go at google auth * fixed env vars * updated tests * removed print statement * updated script to add users * added stubs to requirements * deployment updates * Update admin_app/src/app/login/page.tsx Co-authored-by: Amir Emami <41763233+amiraliemami@users.noreply.github.com> * fixes from PR * fixes from PR * skipping type checking for google auth --------- Co-authored-by: Amir Emami <41763233+amiraliemami@users.noreply.github.com> * HOTFIX: Bring unsalted hashing back for retrieval keys * HOTFIX: Mention Google login client ID env var in dev docs * HOTFIX: env var name in api.ts * Key generation UI (#211) * Correct usage of "startIcon" in New buttons * Add API key generation to front-end * Move modals to components file and other improvements * Add link to integrations page in Playground API key popup * Remove mentions of "current key" for now * Make copy button visually responsive too * Improve spacing on new key modal * Clean up modal button margins * fix format * Add user pytests (#206) * Add key management endpoint pytest * TEMP: Add test for user functions (not working due to async issue) * fix scope of asession in test_urgency_detect (future-proofing) * Fix user tests with async session * Add where to set/find login credentials to docs * Temporary fix for rotation endpoint test * Fix scope mismatch (some UD tests break) * Add multi-user tests for content CRUD * Add multi-user tests for UD CRUD + add fetch check first in delete endpoint * Add user2 token denial test to retrieval and UD detect * Put multi-user tests under a different class * fixed async tests in session scope issue * Make test user_ids integers to match new schema * Fix handling of user_ids in pytest * Separate content CRUD multi-user tests and use fixture * Separate UD CRUD multi-user tests and use fixture --------- Co-authored-by: Sid Ravinutala <sid.ravi1@gmail.com> * Add tags bar * Make dashboard multi-user + some admin app cleanup (#213) * Add key management endpoint pytest * TEMP: Add test for user functions (not working due to async issue) * fix scope of asession in test_urgency_detect (future-proofing) * Fix user tests with async session * Add where to set/find login credentials to docs * Temporary fix for rotation endpoint test * Fix scope mismatch (some UD tests break) * Add multi-user tests for content CRUD * Add multi-user tests for UD CRUD + add fetch check first in delete endpoint * Add user2 token denial test to retrieval and UD detect * Put multi-user tests under a different class * fixed async tests in session scope issue * Make test user_ids integers to match new schema * Fix handling of user_ids in pytest * Separate content CRUD multi-user tests and use fixture * Separate UD CRUD multi-user tests and use fixture * Playground: Add type of search to user message display * Make dashboard functions filter on user_id * Playground: change "JSON" to "<json>" in error message response. * UD page: Make background white for now * move key_management tests to run last :) * Remove incorrect retrieval tests * improve query type formatting and use save button for ud rule --------- Co-authored-by: Sid Ravinutala <sid.ravi1@gmail.com> Co-authored-by: Suzin You <7042047+suzinyou@users.noreply.github.com> * Fix rebase issues * Fix linting * Fix PR comments * Update setup.md * Add additional tests * Fix linting --------- Co-authored-by: amir_emami <amirali1376@gmail.com> Co-authored-by: Amir Emami <41763233+amiraliemami@users.noreply.github.com> Co-authored-by: Sid Ravinutala <sid.ravi1@gmail.com> Co-authored-by: Suzin You <7042047+suzinyou@users.noreply.github.com>
lickem22
added a commit
that referenced
this pull request
May 30, 2024
* add postgres vars to template.env for clarity of use * remove deprecated "version" tag from local docker-compose scripts * change fullaccess and readonly users to user1 and user2 with fullaccess permissions * clarify `NEXT_PUBLIC_BACKEND_URL` variable in api.ts * Add user_id column to ContentDB and update content endpoints. NOTE: user_id hardcoded into QA and WhatsApp endpoints - to be inferred from bearer token later. * Admin app: Show username in profile dropdown * Disable "readonly" tests. Will add back in next PR. * Fix mismatch between get_llm_answer() and its decorators' expectation of a "question" parameter * Undo dropping of the `content_idx` index. Fix source issue later. * Remove duplicated unused AccessLevel declaration in contents schema * Add user table to db * Update backend to use new users table * Change content table user_id column to foreign key * Update env var names * Fix "question" parameter in get_semantic_matches() to match its decorators' expectation * Hardcode QA endpoints to use "user1"s data. Will make dynamic in next PR. * Disable WhatsApp pytests - WhatsApp endpoint is broken due to no access to auth information * Make initial user-loading script standalone and use it in dev setup and update Dockerfile deployment * Make usernames environment variables too * Add user addition in DB to pytests * update default usernames to user1 and user2 * create_content() description fix * Use `.one()` instead of `.scalar_one()` and add MultipleResultsFound exception * Change "user-queries" table to "query". Prevents confusion with AAQ users. * Table "user-query-responses" -> "query-response" * Table "user-query-response-errors" -> "query-response-error" * Table "response_feedback" -> "query-response-feedback" * Table "content_feedback" -> "content-feedback" * Standardise migration file messages * Stardise migration file names (no functionality change) * Fix feedback tables index col names * Remove "User" prefix from Query, QueryResponse, QueryResponseError table models * Remove "User" prefix from content schemas * Add "retrieval_key" column to user table and update QA endpoints to be user-aware * Update typebot flow with "RETRIEVAL_KEY" wording * Update pytest * Add user_id column to query table and store which user's account each query is associated with * Update feedback endpoints to be user-aware * Catch rogue outdated "UserQueryBase" * Implementing hashing of keys and update DB and tests * update migration filename * add key updation endpoint * Make keys 32 bytes and remove unnecessary character count check * Remove unnecessary spaces in Edit page * BUGFIX: Background on playground is always white even after scroll * Add tags to WhatsApp and Auth endpoint routers * Add temporary password column and move auth env vars to main config.py * [temp] disable new key route to avoid circular import * Temp: Add non-generator async session maker * MAJOR: Auth now checks DB for user info and returns user_db if found. Fullaccess and readonly removed. * Remove mentions of access levels from docs * correctly close db sessions in auth * Remove "readonly" tests and test constructions * organise imports * Make "/key" rotator endpoint functional again (avoiding circular imports 😭) * change KeyResponse parameter * Give tests their own user id/name/password/retrieval key * Remove readonly as an accepted option for access_level * Move number of bytes to top of file * Remove leftover comments * Move username-password-key env vars to loading script instead of configs.py * Don't use catch-all "except Exception" * Use `get_sqlalchemy_async_engine()` with `with` directly in auth code to ensure sessions are closed correctly * Make username and key columns unique and remove MultipleResults errors * Remove leftover .aclose() statements * Make UD multi-user and update tests * add user_id to contentdb pretty print * Cleanup UD parameters and add keywords to function calls * fix migration version after rebase * add line length for ruff * Rename authentication functions * Combine migrations (#197) * Combine pre-multiuser migrations that could be combined * Combine multiuser migrations * standardise wording of migration * add date template for migration filenames * Add dates to migration filenames * Fix "NEXT_PUBLIC_BACKEND_URL" in api.ts (UD merge issue) * Fix merge issues * add names to foreign key migrations * fix auth on page * Google login (#205) * first go at google auth * fixed env vars * updated tests * removed print statement * updated script to add users * added stubs to requirements * deployment updates * Update admin_app/src/app/login/page.tsx Co-authored-by: Amir Emami <41763233+amiraliemami@users.noreply.github.com> * fixes from PR * fixes from PR * skipping type checking for google auth --------- Co-authored-by: Amir Emami <41763233+amiraliemami@users.noreply.github.com> * HOTFIX: Bring unsalted hashing back for retrieval keys * HOTFIX: Mention Google login client ID env var in dev docs * HOTFIX: env var name in api.ts * Key generation UI (#211) * Correct usage of "startIcon" in New buttons * Add API key generation to front-end * Move modals to components file and other improvements * Add link to integrations page in Playground API key popup * Remove mentions of "current key" for now * Make copy button visually responsive too * Improve spacing on new key modal * Clean up modal button margins * fix format * Add user pytests (#206) * Add key management endpoint pytest * TEMP: Add test for user functions (not working due to async issue) * fix scope of asession in test_urgency_detect (future-proofing) * Fix user tests with async session * Add where to set/find login credentials to docs * Temporary fix for rotation endpoint test * Fix scope mismatch (some UD tests break) * Add multi-user tests for content CRUD * Add multi-user tests for UD CRUD + add fetch check first in delete endpoint * Add user2 token denial test to retrieval and UD detect * Put multi-user tests under a different class * fixed async tests in session scope issue * Make test user_ids integers to match new schema * Fix handling of user_ids in pytest * Separate content CRUD multi-user tests and use fixture * Separate UD CRUD multi-user tests and use fixture --------- Co-authored-by: Sid Ravinutala <sid.ravi1@gmail.com> * Make dashboard multi-user + some admin app cleanup (#213) * Add key management endpoint pytest * TEMP: Add test for user functions (not working due to async issue) * fix scope of asession in test_urgency_detect (future-proofing) * Fix user tests with async session * Add where to set/find login credentials to docs * Temporary fix for rotation endpoint test * Fix scope mismatch (some UD tests break) * Add multi-user tests for content CRUD * Add multi-user tests for UD CRUD + add fetch check first in delete endpoint * Add user2 token denial test to retrieval and UD detect * Put multi-user tests under a different class * fixed async tests in session scope issue * Make test user_ids integers to match new schema * Fix handling of user_ids in pytest * Separate content CRUD multi-user tests and use fixture * Separate UD CRUD multi-user tests and use fixture * Playground: Add type of search to user message display * Make dashboard functions filter on user_id * Playground: change "JSON" to "<json>" in error message response. * UD page: Make background white for now * move key_management tests to run last :) * Remove incorrect retrieval tests * improve query type formatting and use save button for ud rule --------- Co-authored-by: Sid Ravinutala <sid.ravi1@gmail.com> Co-authored-by: Suzin You <7042047+suzinyou@users.noreply.github.com> * Add pytests to check user1-user2 overlap in retrieval and urgency detect * Update AWS terraform and secret loading (#217) * Change all instances of "retrieval key" to "api key" * Add new user and google client ID env vars to terraform and bootstrap * fix spacing error * [AAQ-482] Add custom embeddings as optional components (#201) * add embeddings * Update docker-compose * update embeddings app * Add make command * Add default embedding * Add docstring * add as an optional component * Fix UD migration issue * add embeddings endpoint to llm proxy * Update optional_components/embeddings/requirements.txt Co-authored-by: Suzin You <7042047+suzinyou@users.noreply.github.com> * update docker-compose * Remove completion endpoint * fix linting * Fix PR comments * change size of vector --------- Co-authored-by: Suzin You <7042047+suzinyou@users.noreply.github.com> * add workflow dispatch force redeployment to admin app cicd * fix command * add manual force creation of task def to all other cicd scripts --------- Co-authored-by: Suzin You <7042047+suzinyou@users.noreply.github.com> Co-authored-by: lickem22 <44327443+lickem22@users.noreply.github.com> --------- Co-authored-by: Sid Ravinutala <sid.ravi1@gmail.com> Co-authored-by: Suzin You <7042047+suzinyou@users.noreply.github.com> Co-authored-by: lickem22 <44327443+lickem22@users.noreply.github.com>
lickem22
added a commit
that referenced
this pull request
Jun 6, 2024
* add postgres vars to template.env for clarity of use * remove deprecated "version" tag from local docker-compose scripts * change fullaccess and readonly users to user1 and user2 with fullaccess permissions * clarify `NEXT_PUBLIC_BACKEND_URL` variable in api.ts * Add user_id column to ContentDB and update content endpoints. NOTE: user_id hardcoded into QA and WhatsApp endpoints - to be inferred from bearer token later. * Admin app: Show username in profile dropdown * Disable "readonly" tests. Will add back in next PR. * Fix mismatch between get_llm_answer() and its decorators' expectation of a "question" parameter * Undo dropping of the `content_idx` index. Fix source issue later. * Remove duplicated unused AccessLevel declaration in contents schema * Add user table to db * Update backend to use new users table * Change content table user_id column to foreign key * Update env var names * Fix "question" parameter in get_semantic_matches() to match its decorators' expectation * Hardcode QA endpoints to use "user1"s data. Will make dynamic in next PR. * Disable WhatsApp pytests - WhatsApp endpoint is broken due to no access to auth information * Make initial user-loading script standalone and use it in dev setup and update Dockerfile deployment * Make usernames environment variables too * Add user addition in DB to pytests * update default usernames to user1 and user2 * create_content() description fix * Use `.one()` instead of `.scalar_one()` and add MultipleResultsFound exception * Change "user-queries" table to "query". Prevents confusion with AAQ users. * Table "user-query-responses" -> "query-response" * Table "user-query-response-errors" -> "query-response-error" * Table "response_feedback" -> "query-response-feedback" * Table "content_feedback" -> "content-feedback" * Standardise migration file messages * Stardise migration file names (no functionality change) * Fix feedback tables index col names * Remove "User" prefix from Query, QueryResponse, QueryResponseError table models * Remove "User" prefix from content schemas * Add "retrieval_key" column to user table and update QA endpoints to be user-aware * Update typebot flow with "RETRIEVAL_KEY" wording * Update pytest * Add user_id column to query table and store which user's account each query is associated with * Update feedback endpoints to be user-aware * Catch rogue outdated "UserQueryBase" * Implementing hashing of keys and update DB and tests * update migration filename * add key updation endpoint * Make keys 32 bytes and remove unnecessary character count check * Remove unnecessary spaces in Edit page * BUGFIX: Background on playground is always white even after scroll * Add tags to WhatsApp and Auth endpoint routers * Add temporary password column and move auth env vars to main config.py * [temp] disable new key route to avoid circular import * Temp: Add non-generator async session maker * MAJOR: Auth now checks DB for user info and returns user_db if found. Fullaccess and readonly removed. * Remove mentions of access levels from docs * correctly close db sessions in auth * Remove "readonly" tests and test constructions * organise imports * Make "/key" rotator endpoint functional again (avoiding circular imports 😭) * change KeyResponse parameter * Give tests their own user id/name/password/retrieval key * Remove readonly as an accepted option for access_level * Move number of bytes to top of file * Remove leftover comments * Move username-password-key env vars to loading script instead of configs.py * Don't use catch-all "except Exception" * Use `get_sqlalchemy_async_engine()` with `with` directly in auth code to ensure sessions are closed correctly * Make username and key columns unique and remove MultipleResults errors * Remove leftover .aclose() statements * Make UD multi-user and update tests * add user_id to contentdb pretty print * Cleanup UD parameters and add keywords to function calls * fix migration version after rebase * add line length for ruff * Rename authentication functions * Combine migrations (#197) * Combine pre-multiuser migrations that could be combined * Combine multiuser migrations * standardise wording of migration * add date template for migration filenames * Add dates to migration filenames * Fix "NEXT_PUBLIC_BACKEND_URL" in api.ts (UD merge issue) * Add content tag models * Fix merge issues * add names to foreign key migrations * fix auth on page * Google login (#205) * first go at google auth * fixed env vars * updated tests * removed print statement * updated script to add users * added stubs to requirements * deployment updates * Update admin_app/src/app/login/page.tsx Co-authored-by: Amir Emami <41763233+amiraliemami@users.noreply.github.com> * fixes from PR * fixes from PR * skipping type checking for google auth --------- Co-authored-by: Amir Emami <41763233+amiraliemami@users.noreply.github.com> * HOTFIX: Bring unsalted hashing back for retrieval keys * HOTFIX: Mention Google login client ID env var in dev docs * HOTFIX: env var name in api.ts * Key generation UI (#211) * Correct usage of "startIcon" in New buttons * Add API key generation to front-end * Move modals to components file and other improvements * Add link to integrations page in Playground API key popup * Remove mentions of "current key" for now * Make copy button visually responsive too * Improve spacing on new key modal * Clean up modal button margins * fix format * Add user pytests (#206) * Add key management endpoint pytest * TEMP: Add test for user functions (not working due to async issue) * fix scope of asession in test_urgency_detect (future-proofing) * Fix user tests with async session * Add where to set/find login credentials to docs * Temporary fix for rotation endpoint test * Fix scope mismatch (some UD tests break) * Add multi-user tests for content CRUD * Add multi-user tests for UD CRUD + add fetch check first in delete endpoint * Add user2 token denial test to retrieval and UD detect * Put multi-user tests under a different class * fixed async tests in session scope issue * Make test user_ids integers to match new schema * Fix handling of user_ids in pytest * Separate content CRUD multi-user tests and use fixture * Separate UD CRUD multi-user tests and use fixture --------- Co-authored-by: Sid Ravinutala <sid.ravi1@gmail.com> * Add tags bar * Make dashboard multi-user + some admin app cleanup (#213) * Add key management endpoint pytest * TEMP: Add test for user functions (not working due to async issue) * fix scope of asession in test_urgency_detect (future-proofing) * Fix user tests with async session * Add where to set/find login credentials to docs * Temporary fix for rotation endpoint test * Fix scope mismatch (some UD tests break) * Add multi-user tests for content CRUD * Add multi-user tests for UD CRUD + add fetch check first in delete endpoint * Add user2 token denial test to retrieval and UD detect * Put multi-user tests under a different class * fixed async tests in session scope issue * Make test user_ids integers to match new schema * Fix handling of user_ids in pytest * Separate content CRUD multi-user tests and use fixture * Separate UD CRUD multi-user tests and use fixture * Playground: Add type of search to user message display * Make dashboard functions filter on user_id * Playground: change "JSON" to "<json>" in error message response. * UD page: Make background white for now * move key_management tests to run last :) * Remove incorrect retrieval tests * improve query type formatting and use save button for ud rule --------- Co-authored-by: Sid Ravinutala <sid.ravi1@gmail.com> Co-authored-by: Suzin You <7042047+suzinyou@users.noreply.github.com> * Fix rebase issues * Add tags filter to landing * Fix position of tags in landing * merge main * Fix tags models * Update setup.md * Update setup.md * Add doc * Update admin_app/src/app/content/edit/page.tsx Co-authored-by: Amir Emami <41763233+amiraliemami@users.noreply.github.com> * Fix tag add warning * Fix tags bar issues * Update add_content_to_db script * Update page.tsx * Add tags * remove changes from script --------- Co-authored-by: amir_emami <amirali1376@gmail.com> Co-authored-by: Amir Emami <41763233+amiraliemami@users.noreply.github.com> Co-authored-by: Sid Ravinutala <sid.ravi1@gmail.com> Co-authored-by: Suzin You <7042047+suzinyou@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reviewer: @lickem22 or @suzinyou
Estimate: 20mins
Ticket
Fixes: 538
Description
Goal
Changes
Future Tasks
How has this been tested?
To do before merge
Checklist
Fill with
xfor completed. Delete any lines that are not relevant