Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Semantic Memory Integration into Copilot Chat #152

Merged
merged 192 commits into from
Sep 21, 2023
Merged

Conversation

crickman
Copy link
Contributor

@crickman crickman commented Aug 11, 2023

BREAKING CHANGE

Motivation and Context

Integration of chat-copilot with https://github.com/microsoft/semantic-memory. With this change, chat-copilot no longer utilizes the https://github.com/microsoft/semantic-kernel connector

Description

  • New "pipeline" service
  • Memory retrieval updated
  • DocumentImportController removed
  • DocumentMemorySkill removed
  • SematicChatMeorySkill removed
  • Core OCR moved to memory-service
  • Orphaned options removed
  • Support for Postgres and Chroma storage removed (for now)
  • Workflows and deployment scripts updated

Examples:

Screenshot 2023-09-18 090927
image
image
image
image
image

Contribution Checklist

TaoChenOSU and others added 10 commits August 3, 2023 13:41
### Motivation and Context

<!-- Thank you for your contribution to the copilot-chat repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [Contribution
Guidelines](https://github.com/microsoft/copilot-chat/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/copilot-chat/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
### Motivation and Context

<!-- Thank you for your contribution to the copilot-chat repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

### Description

<!-- Describe your changes, the overall approach, the underlying design.
These notes will help understanding how your code works. Thanks! -->

1. Rename project from SemanticMemoryPipelineService to
CopilotChatMemoryPipeline.
2. SemanticMemory.Service to CopilotChat.MemoryPipeline.
3. Remove nuget.config.

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [Contribution
Guidelines](https://github.com/microsoft/copilot-chat/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/copilot-chat/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, and I have added new tests where possible
- [ ] I didn't break anyone 😄
@github-actions github-actions bot added webapp Pull requests that update Typescript code webapi Pull requests that update .net code PR: ready for review github actions Pull requests that update GitHub Actions code labels Aug 11, 2023
@crickman crickman added enhancement New feature or request and removed PR: ready for review github actions Pull requests that update GitHub Actions code labels Aug 11, 2023
@crickman crickman temporarily deployed to feature-semantic-memory August 11, 2023 05:58 — with GitHub Actions Inactive
@github-actions github-actions bot added PR: ready for review github actions Pull requests that update GitHub Actions code labels Aug 11, 2023
@crickman crickman added preview Not even close to completed and removed PR: ready for review github actions Pull requests that update GitHub Actions code labels Aug 11, 2023
@crickman crickman temporarily deployed to feature-semantic-memory August 11, 2023 06:00 — with GitHub Actions Inactive
@TaoChenOSU TaoChenOSU temporarily deployed to feature-semantic-memory September 19, 2023 23:53 — with GitHub Actions Inactive
@TaoChenOSU TaoChenOSU temporarily deployed to feature-semantic-memory September 19, 2023 23:56 — with GitHub Actions Inactive
@TaoChenOSU TaoChenOSU temporarily deployed to feature-semantic-memory September 19, 2023 23:56 — with GitHub Actions Inactive
@TaoChenOSU TaoChenOSU temporarily deployed to feature-semantic-memory September 19, 2023 23:56 — with GitHub Actions Inactive
crickman and others added 2 commits September 19, 2023 17:42
### Motivation and Context

Break-out memory migration work for ease of review

### Description

One-time migration of all chat memory (working and long-term).
Removes all memory sources from storage


![image](https://github.com/microsoft/chat-copilot/assets/66376200/b24aa63e-4fb7-49f3-b243-22843624aa5e)

Needs to be able to create two (x2) indexes.

To reset migration, just remove `chatmemory` and `global-documents`

Pushed targeted fixes in
#365

### Contribution Checklist

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [Contribution
Guidelines](https://github.com/microsoft/chat-copilot/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/chat-copilot/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄
@crickman crickman temporarily deployed to feature-semantic-memory September 20, 2023 00:53 — with GitHub Actions Inactive
alliscode
alliscode previously approved these changes Sep 20, 2023
@microsoft microsoft deleted a comment from teresaqhoang Sep 20, 2023
@TaoChenOSU TaoChenOSU temporarily deployed to feature-semantic-memory September 20, 2023 15:57 — with GitHub Actions Inactive
@crickman crickman temporarily deployed to feature-semantic-memory September 20, 2023 16:06 — with GitHub Actions Inactive
@crickman crickman temporarily deployed to feature-semantic-memory September 20, 2023 17:14 — with GitHub Actions Inactive
@crickman crickman temporarily deployed to feature-semantic-memory September 20, 2023 17:14 — with GitHub Actions Inactive
@crickman crickman temporarily deployed to feature-semantic-memory September 20, 2023 17:14 — with GitHub Actions Inactive
@crickman crickman added this pull request to the merge queue Sep 21, 2023
Merged via the queue into main with commit 2244ab2 Sep 21, 2023
12 checks passed
@crickman crickman deleted the feature-semantic-memory branch September 21, 2023 14:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
deployment Issues related to deploying Chat-Copilot documentation Improvements or additions to documentation enhancement New feature or request github actions Pull requests that update GitHub Actions code PR: breaking change Pull requests that introduce breaking changes preview Not even close to completed webapi Pull requests that update .net code webapp Pull requests that update Typescript code
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

Chat Copilot: Add support for Semantic Memory Handle TODOs for upload/download bot
6 participants