Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request]: Backup and Restore #6

Open
tazarov opened this issue Jul 29, 2023 · 0 comments
Open

[Feature Request]: Backup and Restore #6

tazarov opened this issue Jul 29, 2023 · 0 comments
Assignees
Labels
enhancement New feature or request

Comments

@tazarov
Copy link
Contributor

tazarov commented Jul 29, 2023

Describe the problem

Embeddings are relatively cheap but not free, and data is precious so we need a way to keep our data safe from loss in a cloud-native setting.

Describe the proposed solution

It should be possible to automatically back up and restore data.

Alternatives considered

No response

Importance

I cannot run the chart without it

Additional Information

This would take us one step closer using ChromaDB in production

@tazarov tazarov added the enhancement New feature or request label Jul 29, 2023
@tazarov tazarov self-assigned this Jul 29, 2023
tazarov added a commit that referenced this issue Aug 2, 2023
tazarov added a commit that referenced this issue Aug 2, 2023
- Adding sudo to the image for fsfreeze use

Refs: #6
tazarov added a commit that referenced this issue Aug 2, 2023
- Making chroma user sudoer

Refs: #6
tazarov added a commit that referenced this issue Aug 11, 2023
* feat: Added integration workflow

Refs: #1

* fix: Added kubectl wait to wait for deployment to complete

Refs: #1

* fix: Some issue with service. Experimenting with listing service.

Refs: #1

* fix: Fixed the service issue and added matrix testing of kube versions

Refs: #1

* chore: Updated test matrix to match chart kubeVersion range >=1.24 <= 1.27

Refs: #1

* fix: Fixed chart issue with boolean flags

- Added further variance to the test matrix by including allowReset and isPersistent flags (no specific tests are implemented but those are coming soon)

Refs: #1

* fix: Added docker image hnswlib rebuild + sqlite3 for debugging

- Added a small test to ensure chroma is up and running.
- Docker image rebuilt.

* docs: Added small clarification how to build/push the docker image.

* chore: Added bug and feature request issue templates and docker image in ghcr

Refs: #2

* feat: Backup and restore feature

Refs: #6

* feat: Backup and restore feature

- Adding sudo to the image for fsfreeze use

Refs: #6

* feat: Backup and restore feature

- Making chroma user sudoer

Refs: #6

* feat: Adding tests for v0.4.5

* feat: Adding Chroma 0.4.5 support to chart

* feat: Updating WFs

* feat: Lowering chart requirements to k8s API 1.23.0
tazarov added a commit that referenced this issue Aug 15, 2023
* feat: Added integration workflow

Refs: #1

* fix: Added kubectl wait to wait for deployment to complete

Refs: #1

* fix: Some issue with service. Experimenting with listing service.

Refs: #1

* fix: Fixed the service issue and added matrix testing of kube versions

Refs: #1

* chore: Updated test matrix to match chart kubeVersion range >=1.24 <= 1.27

Refs: #1

* fix: Fixed chart issue with boolean flags

- Added further variance to the test matrix by including allowReset and isPersistent flags (no specific tests are implemented but those are coming soon)

Refs: #1

* fix: Added docker image hnswlib rebuild + sqlite3 for debugging

- Added a small test to ensure chroma is up and running.
- Docker image rebuilt.

* docs: Added small clarification how to build/push the docker image.

* chore: Added bug and feature request issue templates and docker image in ghcr

Refs: #2

* feat: Backup and restore feature

Refs: #6

* feat: Backup and restore feature

- Adding sudo to the image for fsfreeze use

Refs: #6

* feat: Backup and restore feature

- Making chroma user sudoer

Refs: #6

* feat: Adding tests for v0.4.5

* feat: Adding Chroma 0.4.5 support to chart

* feat: Updating WFs

* feat: Lowering chart requirements to k8s API 1.23.0

* feat: Adding multi-platform build

* fix: Moved platforms out of the matrix and directly into platform parameter of build-push action

* fix: Fixed a typo in build images workflow.

* chore: Removing hnswlib reinstall to check multi-arch build

* feat: Adding support for Chroma 0.4.6

* chore: Updating integration tests

* feat: Updating chart version and default version to 0.4.6
tazarov added a commit that referenced this issue Aug 17, 2023
* feat: Added integration workflow

Refs: #1

* fix: Added kubectl wait to wait for deployment to complete

Refs: #1

* fix: Some issue with service. Experimenting with listing service.

Refs: #1

* fix: Fixed the service issue and added matrix testing of kube versions

Refs: #1

* chore: Updated test matrix to match chart kubeVersion range >=1.24 <= 1.27

Refs: #1

* fix: Fixed chart issue with boolean flags

- Added further variance to the test matrix by including allowReset and isPersistent flags (no specific tests are implemented but those are coming soon)

Refs: #1

* fix: Added docker image hnswlib rebuild + sqlite3 for debugging

- Added a small test to ensure chroma is up and running.
- Docker image rebuilt.

* docs: Added small clarification how to build/push the docker image.

* chore: Added bug and feature request issue templates and docker image in ghcr

Refs: #2

* feat: Backup and restore feature

Refs: #6

* feat: Backup and restore feature

- Adding sudo to the image for fsfreeze use

Refs: #6

* feat: Backup and restore feature

- Making chroma user sudoer

Refs: #6

* feat: Adding tests for v0.4.5

* feat: Adding Chroma 0.4.5 support to chart

* feat: Updating WFs

* feat: Lowering chart requirements to k8s API 1.23.0

* feat: Adding multi-platform build

* fix: Moved platforms out of the matrix and directly into platform parameter of build-push action

* fix: Fixed a typo in build images workflow.

* chore: Removing hnswlib reinstall to check multi-arch build

* feat: Adding support for Chroma 0.4.6

* chore: Updating integration tests

* feat: Updating chart version and default version to 0.4.6

* fix: chroma-hnswlib reinstall is needed as it does uniformly work across platforms.

* chore: Chart version bump for hnsw lib fix
tazarov added a commit that referenced this issue Aug 17, 2023
* feat: Added integration workflow

Refs: #1

* fix: Added kubectl wait to wait for deployment to complete

Refs: #1

* fix: Some issue with service. Experimenting with listing service.

Refs: #1

* fix: Fixed the service issue and added matrix testing of kube versions

Refs: #1

* chore: Updated test matrix to match chart kubeVersion range >=1.24 <= 1.27

Refs: #1

* fix: Fixed chart issue with boolean flags

- Added further variance to the test matrix by including allowReset and isPersistent flags (no specific tests are implemented but those are coming soon)

Refs: #1

* fix: Added docker image hnswlib rebuild + sqlite3 for debugging

- Added a small test to ensure chroma is up and running.
- Docker image rebuilt.

* docs: Added small clarification how to build/push the docker image.

* chore: Added bug and feature request issue templates and docker image in ghcr

Refs: #2

* feat: Backup and restore feature

Refs: #6

* feat: Backup and restore feature

- Adding sudo to the image for fsfreeze use

Refs: #6

* feat: Backup and restore feature

- Making chroma user sudoer

Refs: #6

* feat: Adding tests for v0.4.5

* feat: Adding Chroma 0.4.5 support to chart

* feat: Updating WFs

* feat: Lowering chart requirements to k8s API 1.23.0

* feat: Adding multi-platform build

* fix: Moved platforms out of the matrix and directly into platform parameter of build-push action

* fix: Fixed a typo in build images workflow.

* chore: Removing hnswlib reinstall to check multi-arch build

* feat: Adding support for Chroma 0.4.6

* chore: Updating integration tests

* feat: Updating chart version and default version to 0.4.6

* fix: chroma-hnswlib reinstall is needed as it does uniformly work across platforms.

* chore: Chart version bump for hnsw lib fix

* chore: Chart version bump for hnsw lib fix
tazarov added a commit that referenced this issue Aug 27, 2023
* feat: Added integration workflow

Refs: #1

* fix: Added kubectl wait to wait for deployment to complete

Refs: #1

* fix: Some issue with service. Experimenting with listing service.

Refs: #1

* fix: Fixed the service issue and added matrix testing of kube versions

Refs: #1

* chore: Updated test matrix to match chart kubeVersion range >=1.24 <= 1.27

Refs: #1

* fix: Fixed chart issue with boolean flags

- Added further variance to the test matrix by including allowReset and isPersistent flags (no specific tests are implemented but those are coming soon)

Refs: #1

* fix: Added docker image hnswlib rebuild + sqlite3 for debugging

- Added a small test to ensure chroma is up and running.
- Docker image rebuilt.

* docs: Added small clarification how to build/push the docker image.

* chore: Added bug and feature request issue templates and docker image in ghcr

Refs: #2

* feat: Backup and restore feature

Refs: #6

* feat: Backup and restore feature

- Adding sudo to the image for fsfreeze use

Refs: #6

* feat: Backup and restore feature

- Making chroma user sudoer

Refs: #6

* feat: Adding tests for v0.4.5

* feat: Adding Chroma 0.4.5 support to chart

* feat: Updating WFs

* feat: Lowering chart requirements to k8s API 1.23.0

* feat: Adding multi-platform build

* fix: Moved platforms out of the matrix and directly into platform parameter of build-push action

* fix: Fixed a typo in build images workflow.

* chore: Removing hnswlib reinstall to check multi-arch build

* feat: Adding support for Chroma 0.4.6

* chore: Updating integration tests

* feat: Updating chart version and default version to 0.4.6

* fix: chroma-hnswlib reinstall is needed as it does uniformly work across platforms.

* chore: Chart version bump for hnsw lib fix

* chore: Chart version bump for hnsw lib fix

* feat: Support for k8s API 1.28 and Chroma 0.4.7

Refs: #14

* feat: Support for k8s API 1.28 and Chroma 0.4.7

- Bumping upper bound of chart support.
- Removed isPeristent/allowReset from test matrix to reduce generated jobs.

Refs: #14

* fix: Made the integration test install local copy of the chart instead of pullingit.

* fix: Updating chart version to allow RC

Refs: #14

* chore: Version bumps

- Default Chroma version set to 0.4.7
- Chart version bump

Refs: #14
tazarov added a commit that referenced this issue Sep 10, 2023
* feat: Added integration workflow

Refs: #1

* fix: Added kubectl wait to wait for deployment to complete

Refs: #1

* fix: Some issue with service. Experimenting with listing service.

Refs: #1

* fix: Fixed the service issue and added matrix testing of kube versions

Refs: #1

* chore: Updated test matrix to match chart kubeVersion range >=1.24 <= 1.27

Refs: #1

* fix: Fixed chart issue with boolean flags

- Added further variance to the test matrix by including allowReset and isPersistent flags (no specific tests are implemented but those are coming soon)

Refs: #1

* fix: Added docker image hnswlib rebuild + sqlite3 for debugging

- Added a small test to ensure chroma is up and running.
- Docker image rebuilt.

* docs: Added small clarification how to build/push the docker image.

* chore: Added bug and feature request issue templates and docker image in ghcr

Refs: #2

* feat: Backup and restore feature

Refs: #6

* feat: Backup and restore feature

- Adding sudo to the image for fsfreeze use

Refs: #6

* feat: Backup and restore feature

- Making chroma user sudoer

Refs: #6

* feat: Adding tests for v0.4.5

* feat: Adding Chroma 0.4.5 support to chart

* feat: Updating WFs

* feat: Lowering chart requirements to k8s API 1.23.0

* feat: Adding multi-platform build

* fix: Moved platforms out of the matrix and directly into platform parameter of build-push action

* fix: Fixed a typo in build images workflow.

* chore: Removing hnswlib reinstall to check multi-arch build

* feat: Adding support for Chroma 0.4.6

* chore: Updating integration tests

* feat: Updating chart version and default version to 0.4.6

* fix: chroma-hnswlib reinstall is needed as it does uniformly work across platforms.

* chore: Chart version bump for hnsw lib fix

* chore: Chart version bump for hnsw lib fix

* feat: Support for k8s API 1.28 and Chroma 0.4.7

Refs: #14

* feat: Support for k8s API 1.28 and Chroma 0.4.7

- Bumping upper bound of chart support.
- Removed isPeristent/allowReset from test matrix to reduce generated jobs.

Refs: #14

* fix: Made the integration test install local copy of the chart instead of pullingit.

* fix: Updating chart version to allow RC

Refs: #14

* chore: Version bumps

- Default Chroma version set to 0.4.7
- Chart version bump

Refs: #14

* feat: Chroma auth support

- Token and basic auth support
- Bumping chart version
- Added chroma 0.4.8 and 0.4.9 support
- Minor tweaks to Dockerfile

Refs: #13

* feat: Chroma auth support

- Fixed image builds

Refs: #13

* feat: Chroma auth support

- bump wait time

Refs: #13

* feat: Chroma auth support

- Adding debug

Refs: #13

* feat: Chroma auth support

- Adding debug

Refs: #13

* feat: Chroma auth support

- Adding debug

Refs: #13

* feat: Chroma auth support

- Merging dev image build with int test

Refs: #13

* feat: Chroma auth support

- Fixed a typo in dev image build

Refs: #13

* feat: Chroma auth support

- Fixed wrong chart value in integration test.

Refs: #13

* feat: Chroma auth support

- Increasing wait time for pod

Refs: #13

* feat: Chroma auth support

- Chasing 0.4.7 bug

Refs: #13

* feat: Chroma auth support

- Chasing 0.4.7 bug

Refs: #13

* feat: Chroma auth support

- Chasing 0.4.7 bug

Refs: #13

* feat: Chroma auth support

- Conditional support of auth: basic (0.4.7), token (0.4.8)

Refs: #13

* feat: Chroma auth support

- Added full test matrix

Refs: #13

* feat: Chroma auth support

- Int testing is now trigger for changes in Chart, tests, Docker image

Refs: #13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant