OpenAI Key per Org #82

nishika26 · 2025-03-24T04:12:13Z

Summary

Target issue is #55

#149 will be implemented later on, after this pr is merged. The reason I have kept that implementation for later is because I did not want this pr to become very extensive.

This pull request introduces several key features and updates to the ai-platform repository. It primarily focuses on managing organization credentials and API keys, specifically OpenAI keys, on a per-organization basis. The changes include:

Database Migration: A new migration script is added to create a 'creds' table in the database. This table is designed to store organization credentials, including fields for organization ID, active status, validity, and JSON credential data.
API Enhancements: A new credentials API router is introduced, providing CRUD operations for managing organization credentials and API keys. Most endpoints are secured with superuser access control.
CRUD Operations: New CRUD operations are implemented for handling organization credentials, including functionalities to set, get, and remove credentials, with specific handling for OpenAI API keys.
Model Definitions: Credential models are defined using SQLModel, supporting JSON storage and organization relationships. The models include base, create, update, and public variants.
Project API Update: A return statement is added to the delete_project endpoint to ensure consistency with other API endpoints.
CI Workflow Update: An empty line is added to the continuous integration workflow file.

These changes are merged from the feature/creds branch into the main branch, as part of the effort to manage OpenAI keys per organization.

This pull request introduces a new feature to the ai-platform repository, specifically targeting the management of OpenAI keys per organization. The changes are made on the feature/creds branch and are intended to be merged into the main branch. Key updates include:

Continuous Integration Workflow: A new line has been added to the .github/workflows/continuous_integration.yml file to enhance the CI process.
Database Migration: A new Alembic migration script (e08733a76537_create_credentials_table.py) is added to create a creds table. This table includes columns for organization_id, is_active, valid, id, and credential fields, along with necessary foreign key constraints.
API Routes: New routes are introduced in backend/app/api/routes/credentials.py to manage credentials. These routes support CRUD operations for API keys and organization credentials, with appropriate access control and error handling. Additionally, a return statement is added to the delete_project endpoint in backend/app/api/routes/project.py to ensure consistent API response formats.
CRUD Operations: The backend/app/crud/credentials.py file now includes CRUD operations for managing organization credentials, focusing on setting, getting, and removing credentials, particularly for API key management.
Model System: A new credentials model system is defined in backend/app/models/credentials.py using SQLModel. It includes base, create, update, and public credential classes, with organization relationships and JSON storage capabilities. The backend/app/models/organization.py file is updated to change the relationship from api_keys to creds, maintaining a bidirectional relationship with the Creds model.

These changes collectively enhance the platform's ability to manage OpenAI keys on a per-organization basis, improving both functionality and data integrity.

This pull request introduces a new feature to the ai-platform repository, specifically targeting the management of OpenAI keys per organization. The changes include:

Database Migration: A new Alembic migration script is added to create a credentials table. This table includes columns for organization_id, is_active, valid, id, and credential fields, with appropriate foreign key constraints.
API Enhancements: A new API route for credentials management is introduced, providing CRUD operations for organization credentials. This includes handling API key management and implementing superuser access controls.
CRUD Operations: New CRUD functionalities are added for managing credentials, allowing setting, getting, and removing credentials for organizations, with specific handling for OpenAI API keys.
Model Updates: Credential models are introduced using SQLModel, featuring organization relationships and schemas for base, create, update, and public credential management.
Organization Model Update: The existing APIKey relationship in the Organization model is replaced with a Creds relationship, indicating a transition in the authentication mechanism from API keys to a more comprehensive credentials system.
Project API Update: A return statement is added to the delete_project endpoint to ensure a consistent API response format.
CI Workflow: An empty line is added to the CI workflow file, which does not introduce any functional changes.

These updates collectively enhance the platform's capability to manage credentials at an organizational level, with a focus on OpenAI API key management.

This pull request introduces a new feature to the ProjectTech4DevAI/ai-platform repository, specifically targeting the management of OpenAI API keys on a per-organization basis. The changes are made in the feature/creds branch and are intended to be merged into the main branch.

Key updates include:

Database Migration: A new Alembic migration script is added to create a credentials table. This table includes columns for organization_id, is_active, valid, id, and credential, with appropriate foreign key constraints.
API Enhancements: A new API route for credentials management is implemented, providing CRUD operations to create, read, update, and delete credentials associated with organizations.
CRUD Operations: The pull request introduces CRUD functionalities in the credentials.py file, allowing for the management of credentials, specifically handling OpenAI API keys.
Model Updates:
- A new credentials model system is defined using SQLModel, which includes base, create, update, and public variants of credential objects, establishing relationships with organizations.
- The organization model is updated to replace the existing API keys relationship with a new Creds relationship, reflecting changes in the authentication system.
Project API Consistency: A return statement is added to the delete_project endpoint in the project API for improved consistency, although there are noted areas for improvement in error handling and documentation.
CI Workflow: An empty line is added to the CI workflow file, which does not introduce any functional changes.

These changes collectively aim to enhance the platform's capability to manage OpenAI API keys at the organizational level, improving both functionality and consistency across the system.

This pull request introduces a new feature to the ai-platform repository, specifically targeting the management of OpenAI keys per organization. The changes are made on the feature/creds branch and are intended to be merged into the main branch.

Key updates include:

Database Migration: A new Alembic migration script is added to create a creds table. This table is designed to store organization credentials, including fields for organization_id, is_active status, validity, and JSON credential data.
API Enhancements: A new credentials management API is introduced, providing CRUD operations for organization credentials and API keys. This includes endpoints for creating, reading, updating, and deleting credentials.
CRUD Operations: The backend now includes CRUD operations for managing organization credentials, with functions to set, get, remove, and fetch API keys.
Model Updates:
- New credential model classes are added using SQLModel, which facilitate database interactions. These include base, create, update, and public models with JSON credential storage.
- The Organization model's relationship with APIKey is updated to relate to Creds, maintaining the existing structure but changing the associated entity.
Project API Update: A return statement is added to the delete_project endpoint to ensure consistency with other endpoints.
CI Workflow: An empty line is added to the CI workflow file, which does not have any functional impact.

These changes collectively enhance the platform's ability to manage organization-specific credentials, particularly for OpenAI API keys.

Co-authored-by: sourabhlodha <sourabhlodha@Administrators-MacBook-Pro.local>

rename project and stack --------- Co-authored-by: sourabhlodha <sourabhlodha@Administrators-MacBook-Pro.local>

* use latest docker image * update envsample

* token expiry time can be customize * default to one day

* trial * pushing all * models file * renaming * Rename Project.py to project.py * Rename oganization.py to organization.py * Update README.md (#44) * changes (#45) Co-authored-by: sourabhlodha <sourabhlodha@Administrators-MacBook-Pro.local> * Readme update (#47) rename project and stack --------- Co-authored-by: sourabhlodha <sourabhlodha@Administrators-MacBook-Pro.local> * fix create_user endpoint (#62) * standard api response and http exception handling (#67) * standardization and edits * small edits * small edits * small edits * fixed project post * trial * pushing all * models file * renaming * Rename Project.py to project.py * Rename oganization.py to organization.py * standardization and edits * small edits * small edits * small edits * fixed project post * remove these files since they were somehow pushed into this branch * re-push the docker file * re-push utils file * re-push the file * fixing test cases --------- Co-authored-by: Sourabh Lodha <sourabh_lodha@ymail.com> Co-authored-by: sourabhlodha <sourabhlodha@Administrators-MacBook-Pro.local> Co-authored-by: Aviraj Gour <100823015+avirajsingh7@users.noreply.github.com> Co-authored-by: Ishankoradia <ikoradia@umich.edu>

* intial commit user project mapping and authorization * fix alembic migration * Use standard API response * add pagination * add index and use base model

* fixing testcases and migrations * changes migration file name * remove old migration --------- Co-authored-by: Akhilesh Negi <akhileshnegi.an3@gmail.com>

* Intial setup api key * added Api key auth flow * support both api key and oauth --------- Co-authored-by: Sourabh Lodha <sourabh_lodha@ymail.com>

Back merge Production to staging code

* fix authentication part * Modify test cases to compatible with new auth

* issue CI * first stab at continuous integration * fixing testcases and migrations * syncing with master * moving to python version 3.11.7 * making copy of env * updating env * added migrations * added uv sync * updating working directory * added step to activate env * updating working directory * updating working directory for codecov upload * updating script to upload to codecov * remove working directory * added working directory for % check * clenaup * cleanup * activating env * update the issue template * update readme and env file

* getting threads up and running * added testcases and citation * removing ssl verify * using standardized APIResponse * getting rid of redundant files * refactor code after testing * refactor testcases * setting up init.py * fixing review comments * cleanup * cleanup * removed validate thread as it can be handled by default * fixing few code review suggestions * removed validation testcases for assistant ID

codecov · 2025-03-27T11:21:56Z

Codecov Report

Attention: Patch coverage is 91.69231% with 27 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
backend/app/api/routes/credentials.py	83.05%	10 Missing ⚠️
backend/app/crud/credentials.py	83.63%	9 Missing ⚠️
backend/app/tests/crud/test_creds.py	93.93%	4 Missing ⚠️
backend/app/tests/api/routes/test_creds.py	97.34%	3 Missing ⚠️
backend/app/models/organization.py	80.00%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

…tial_table.py

AkhileshNegi · 2025-04-14T04:18:38Z

backend/app/alembic/versions/fa868aa8debd_add_credential_table.py

+
+
+def upgrade():
+    # ### commands auto generated by Alembic - please adjust! ###


do we need this?

AkhileshNegi · 2025-04-14T04:18:51Z

backend/app/alembic/versions/fa868aa8debd_add_credential_table.py

+
+
+def downgrade():
+    # ### commands auto generated by Alembic - please adjust! ###


check other comments too

AkhileshNegi · 2025-04-14T04:21:32Z

backend/app/api/routes/credentials.py

+    dependencies=[Depends(get_current_active_superuser)],
+    response_model=APIResponse[CredsPublic],
+)
+def create_new_credential(*, session: SessionDep, creds_in: CredsCreate):


As more than one credentials will be added then we should have another column like name or something it separate multiple credentials like OpenAI, Gemini. Might as well add description Column as well

AkhileshNegi · 2025-04-14T04:27:15Z

backend/app/alembic/versions/fa868aa8debd_add_credential_table.py

+        sa.ForeignKeyConstraint(
+            ["organization_id"], ["organization.id"], ondelete="CASCADE"
+        ),
+        sa.PrimaryKeyConstraint("id"),


also add inserted_at and updated_at column

* Update README.md (ProjectTech4DevAI#44) * changes (ProjectTech4DevAI#45) * Readme update (ProjectTech4DevAI#47) rename project and stack --------- Co-authored-by: sourabhlodha <sourabhlodha@Administrators-MacBook-Pro.local> * fix create_user endpoint (ProjectTech4DevAI#62) * standard api response and http exception handling (ProjectTech4DevAI#67) * Upgrade PostgreSQL to 16 & Fix CORS Configuration (ProjectTech4DevAI#57) * use latest docker image * update envsample * Add Customizable Token Expiry Time in Login API (ProjectTech4DevAI#70) * token expiry time can be customize * default to one day * Organization/project : Crud, Endpoint and Test Cases (ProjectTech4DevAI#63) * trial * pushing all * models file * renaming * Rename Project.py to project.py * Rename oganization.py to organization.py * Update README.md (ProjectTech4DevAI#44) * changes (ProjectTech4DevAI#45) Co-authored-by: sourabhlodha <sourabhlodha@Administrators-MacBook-Pro.local> * Readme update (ProjectTech4DevAI#47) rename project and stack --------- Co-authored-by: sourabhlodha <sourabhlodha@Administrators-MacBook-Pro.local> * fix create_user endpoint (ProjectTech4DevAI#62) * standard api response and http exception handling (ProjectTech4DevAI#67) * standardization and edits * small edits * small edits * small edits * fixed project post * trial * pushing all * models file * renaming * Rename Project.py to project.py * Rename oganization.py to organization.py * standardization and edits * small edits * small edits * small edits * fixed project post * remove these files since they were somehow pushed into this branch * re-push the docker file * re-push utils file * re-push the file * fixing test cases --------- Co-authored-by: Sourabh Lodha <sourabh_lodha@ymail.com> Co-authored-by: sourabhlodha <sourabhlodha@Administrators-MacBook-Pro.local> Co-authored-by: Aviraj Gour <100823015+avirajsingh7@users.noreply.github.com> Co-authored-by: Ishankoradia <ikoradia@umich.edu> * Add Project User Management (ProjectTech4DevAI#65) * intial commit user project mapping and authorization * fix alembic migration * Use standard API response * add pagination * add index and use base model * Alembic: migration fixes for organization (ProjectTech4DevAI#77) * fixing testcases and migrations * changes migration file name * remove old migration --------- Co-authored-by: Akhilesh Negi <akhileshnegi.an3@gmail.com> * Added Support of API Key Authentication (ProjectTech4DevAI#76) * Intial setup api key * added Api key auth flow * support both api key and oauth --------- Co-authored-by: Sourabh Lodha <sourabh_lodha@ymail.com> * Main to stage code sync (ProjectTech4DevAI#80) Back merge Production to staging code * added migration for api table (ProjectTech4DevAI#81) * creds table * Refactor Authentication Logic and Testing Enhancements (ProjectTech4DevAI#89) * fix authentication part * Modify test cases to compatible with new auth * Github: CI (ProjectTech4DevAI#74) * issue CI * first stab at continuous integration * fixing testcases and migrations * syncing with master * moving to python version 3.11.7 * making copy of env * updating env * added migrations * added uv sync * updating working directory * added step to activate env * updating working directory * updating working directory for codecov upload * updating script to upload to codecov * remove working directory * added working directory for % check * clenaup * cleanup * activating env * update the issue template * update readme and env file * adding badges (ProjectTech4DevAI#91) * OpenAI: Threads (ProjectTech4DevAI#40) * getting threads up and running * added testcases and citation * removing ssl verify * using standardized APIResponse * getting rid of redundant files * refactor code after testing * refactor testcases * setting up init.py * fixing review comments * cleanup * cleanup * removed validate thread as it can be handled by default * fixing few code review suggestions * removed validation testcases for assistant ID * threads testcases fix (ProjectTech4DevAI#93) * project router changes * endpoint,crud and migration file * models file * minor fix * fixes * fixes * test cases and fixes * alembic file * type checking * cleaner exception * fixing alembic revision heads * using crendentials * init module * running pre commit * running pre commit * final changes * migration file * Rename fa868aa8debd_add_credetial_table.py to fa868aa8debd_add_credential_table.py * test case change * test cases * test cases * test cases * removing duplicate lines * datetime columns addition * migration file --------- Co-authored-by: Sourabh Lodha <sourabh_lodha@ymail.com> Co-authored-by: Aviraj Gour <100823015+avirajsingh7@users.noreply.github.com> Co-authored-by: Akhilesh Negi <akhileshnegi.an3@gmail.com>

sourabhlodha and others added 15 commits March 14, 2025 10:56

Update README.md (#44)

4b18f4d

changes (#45)

ff651a8

Co-authored-by: sourabhlodha <sourabhlodha@Administrators-MacBook-Pro.local>

Readme update (#47)

56c30e6

rename project and stack --------- Co-authored-by: sourabhlodha <sourabhlodha@Administrators-MacBook-Pro.local>

Merge branch 'main' into staging

ce86e7b

fix create_user endpoint (#62)

5942790

standard api response and http exception handling (#67)

bf76406

Upgrade PostgreSQL to 16 & Fix CORS Configuration (#57)

641bc89

* use latest docker image * update envsample

Add Customizable Token Expiry Time in Login API (#70)

765c2ac

* token expiry time can be customize * default to one day

Add Project User Management (#65)

a7a2a9c

* intial commit user project mapping and authorization * fix alembic migration * Use standard API response * add pagination * add index and use base model

Alembic: migration fixes for organization (#77)

17daeb9

* fixing testcases and migrations * changes migration file name * remove old migration --------- Co-authored-by: Akhilesh Negi <akhileshnegi.an3@gmail.com>

Added Support of API Key Authentication (#76)

961b6a7

* Intial setup api key * added Api key auth flow * support both api key and oauth --------- Co-authored-by: Sourabh Lodha <sourabh_lodha@ymail.com>

Main to stage code sync (#80)

8f121b2

Back merge Production to staging code

added migration for api table (#81)

9996a5e

creds table

e67d94e

nishika26 self-assigned this Mar 24, 2025

avirajsingh7 and others added 6 commits March 24, 2025 17:23

Refactor Authentication Logic and Testing Enhancements (#89)

90c9b68

* fix authentication part * Modify test cases to compatible with new auth

adding badges (#91)

261efb7

threads testcases fix (#93)

52197e4

Merge branch 'staging' into feature/creds

22d80e8

nishika26 changed the base branch from staging to main March 31, 2025 14:54

nishika26 and others added 5 commits March 31, 2025 20:27

Merge branch 'main' into feature/creds

9f923fc

project router changes

1211201

endpoint,crud and migration file

8cfc35f

models file

fbca8a1

minor fix

f728f15

nishika26 requested a review from jerome-white March 31, 2025 15:10

nishika26 and others added 15 commits April 8, 2025 19:55

Merge branch 'main' into feature/creds

8e35440

fixing alembic revision heads

c4af5bb

using crendentials

14867aa

init module

3e5fb33

running pre commit

179ed0e

running pre commit

3467ce4

Merge branch 'main' into feature/creds

f221dde

final changes

01ec58c

migration file

85d5620

Rename fa868aa8debd_add_credetial_table.py to fa868aa8debd_add_creden…

356aeed

…tial_table.py

test case change

43fd3ec

test cases

a7c7f0c

test cases

0c6a4c4

test cases

c85890d

removing duplicate lines

a19e41f

AkhileshNegi reviewed Apr 14, 2025

View reviewed changes

AkhileshNegi requested changes Apr 14, 2025

View reviewed changes

sourabhlodha approved these changes Apr 14, 2025

View reviewed changes

nishika26 added 2 commits April 15, 2025 00:59

datetime columns addition

c93ea38

migration file

b364944

nishika26 mentioned this pull request Apr 15, 2025

Cleaner exception handling #126

Closed

nishika26 added 2 commits April 15, 2025 12:20

Merge branch 'main' into feature/creds

c7056ee

Merge branch 'main' into feature/creds

4b7c01b

sourabhlodha approved these changes Apr 16, 2025

View reviewed changes

AkhileshNegi approved these changes Apr 17, 2025

View reviewed changes

AkhileshNegi merged commit 12bf9b6 into main Apr 17, 2025
1 of 2 checks passed

AkhileshNegi deleted the feature/creds branch April 17, 2025 13:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

OpenAI Key per Org #82

OpenAI Key per Org #82

Uh oh!

nishika26 commented Mar 24, 2025 •

edited

Loading

Uh oh!

codecov bot commented Mar 27, 2025 •

edited

Loading

Uh oh!

AkhileshNegi Apr 14, 2025

Uh oh!

AkhileshNegi Apr 14, 2025

Uh oh!

AkhileshNegi Apr 14, 2025 •

edited

Loading

Uh oh!

AkhileshNegi Apr 14, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants



		def upgrade():
		# ### commands auto generated by Alembic - please adjust! ###



		def downgrade():
		# ### commands auto generated by Alembic - please adjust! ###

OpenAI Key per Org #82

OpenAI Key per Org #82

Uh oh!

Conversation

nishika26 commented Mar 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

codecov bot commented Mar 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

AkhileshNegi Apr 14, 2025

Choose a reason for hiding this comment

Uh oh!

AkhileshNegi Apr 14, 2025

Choose a reason for hiding this comment

Uh oh!

AkhileshNegi Apr 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AkhileshNegi Apr 14, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

nishika26 commented Mar 24, 2025 •

edited

Loading

codecov bot commented Mar 27, 2025 •

edited

Loading

AkhileshNegi Apr 14, 2025 •

edited

Loading