Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Copilot chat: Avoid duplication in document memories #773

Conversation

TaoChenOSU
Copy link
Contributor

Motivation and Context

Importing documents in Copilot chat will not check for duplication. This will potentially create issues where the same documents are uploaded multiple times.

Description

  1. Enable the document import controller to check for duplication before saving a new memory to the document memory.
  2. Allow developers to configure the threshold of duplication (similarity score) in the settings.
  3. Move a couple hard-coded similarity score settings to settings.

Contribution Checklist

@TaoChenOSU TaoChenOSU added PR: ready for review All feedback addressed, ready for reviews samples labels May 2, 2023
@TaoChenOSU TaoChenOSU changed the title Avoid duplication in document memories Copilot chat: Avoid duplication in document memories May 2, 2023
@adrianwyatt adrianwyatt self-assigned this May 3, 2023
@adrianwyatt adrianwyatt added PR: feedback to address Waiting for PR owner to address comments/questions and removed PR: ready for review All feedback addressed, ready for reviews labels May 3, 2023
@TaoChenOSU TaoChenOSU force-pushed the users/taochen/copilot_chat_deduplicate_document_memory branch from bc56294 to 5e80d1e Compare May 5, 2023 17:52
@TaoChenOSU TaoChenOSU added PR: ready for review All feedback addressed, ready for reviews and removed PR: feedback to address Waiting for PR owner to address comments/questions labels May 5, 2023
@TaoChenOSU TaoChenOSU requested a review from glahaye May 8, 2023 00:26
adrianwyatt
adrianwyatt previously approved these changes May 8, 2023
@adrianwyatt adrianwyatt added PR: ready to merge PR has been approved by all reviewers, and is ready to merge. and removed PR: ready for review All feedback addressed, ready for reviews labels May 8, 2023
@TaoChenOSU TaoChenOSU force-pushed the users/taochen/copilot_chat_deduplicate_document_memory branch from a5f593a to 3f3a72e Compare May 8, 2023 16:56
@adrianwyatt adrianwyatt merged commit e38d97f into microsoft:main May 8, 2023
10 checks passed
dehoward pushed a commit to lemillermicrosoft/semantic-kernel that referenced this pull request Jun 1, 2023
### Motivation and Context
Importing documents in Copilot chat will not check for duplication. This
will potentially create issues where the same documents are uploaded
multiple times.

### Description
1. Enable the document import controller to check for duplication before
saving a new memory to the document memory.
2. Allow developers to configure the threshold of duplication
(similarity score) in the settings.
3. Move a couple hard-coded similarity score settings to settings.
golden-aries pushed a commit to golden-aries/semantic-kernel that referenced this pull request Oct 10, 2023
### Motivation and Context
Importing documents in Copilot chat will not check for duplication. This
will potentially create issues where the same documents are uploaded
multiple times.

### Description
1. Enable the document import controller to check for duplication before
saving a new memory to the document memory.
2. Allow developers to configure the threshold of duplication
(similarity score) in the settings.
3. Move a couple hard-coded similarity score settings to settings.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
PR: ready to merge PR has been approved by all reviewers, and is ready to merge.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants