Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

.Net ImageToText Abstraction + HuggingFace Connector ImageToText support. #5150

Merged
merged 52 commits into from
Feb 27, 2024
Merged

.Net ImageToText Abstraction + HuggingFace Connector ImageToText support. #5150

merged 52 commits into from
Feb 27, 2024

Conversation

RogerBarreto
Copy link
Member

Motivation and Context

Add support for a new modality Image -> Text including a Hugging Face service for it.

Resolves #4947

@RogerBarreto
Preparing 1.1 release
3dd55f4
@RogerBarreto
Merge branch 'main' of https://github.com/microsoft/semantic-kernel
8de6748
@RogerBarreto
Merge branch 'main' of https://github.com/microsoft/semantic-kernel
5bbd7fc
@RogerBarreto
Merge branch 'main' of https://github.com/microsoft/semantic-kernel
f8a5bcb
@RogerBarreto
Merge branch 'main' of https://github.com/microsoft/semantic-kernel
80ea9b0
@RogerBarreto
Merge branch 'main' of https://github.com/microsoft/semantic-kernel
e53fbf7
@RogerBarreto
Merge branch 'main' of https://github.com/microsoft/semantic-kernel
4cfba8f
@RogerBarreto
Merge branch 'main' of https://github.com/microsoft/semantic-kernel
8075acf
@RogerBarreto
Adding hugging face httpclient
d5c4e6a
@RogerBarreto
Decimal to Double
ff1fc3c
@RogerBarreto
Streaming object model
6b83649
@RogerBarreto
Merge branch 'main' of https://github.com/microsoft/semantic-kernel i…
b22bfbc 
…nto features/hugging-face-httpclient
@RogerBarreto
Correct naming
9bc339d
@RogerBarreto
Warning fixes
d5fee20
@RogerBarreto
Merge branch 'main' into features/hugging-face-httpclient
502c97a
@RogerBarreto
Merge branch 'main' of https://github.com/microsoft/semantic-kernel i…
ff3fe06 
…nto features/hugging-face-httpclient
@RogerBarreto
Clone overriding
9a28f68
@RogerBarreto
Adding client and missing components to huggingface connectors
1c378ea
@RogerBarreto
Text GEneration Service updated, old removed
da75116
@RogerBarreto
Unit Tests migrated and passing into new HuggingFace UT project
40ab033
@RogerBarreto
Namespace reorganization
4f752f5
@RogerBarreto
Fixing unit tests
e395128
@RogerBarreto
Repetitionpenalty should be nullable
5f37528
@RogerBarreto
Removing null check from PromptExecutionSettings
88fe209
@RogerBarreto
Merge branch 'main' into features/hugging-face-httpclient
32153bc
@RogerBarreto
typo fix
1f749a2
@RogerBarreto
Fix Streaming Bug
9fbb4e8
@RogerBarreto
Removed test project
1bf9e6b
@RogerBarreto
Embeddings Tests and Example update
681c63b
@RogerBarreto
Moving HttpMessageHandlerStub to InternalUtilities for tests
0b498af
RogerBarreto and others added 12 commits February 22, 2024 16:45
@RogerBarreto
Adding ImageToText Abstraction and implementation with HuggingFace
380e1d8
@RogerBarreto
Revert "Moving HttpMessageHandlerStub to InternalUtilities for tests"
7f09d8d 
This reverts commit 0b498af.
@RogerBarreto
Reverting the Stub back
4a06ac5
@RogerBarreto @Krzysztof318
Update dotnet/src/Connectors/Connectors.HuggingFace/Core/HuggingFaceC…
774e43b 
…lient.cs

Co-authored-by: Krzysztof Kasprowicz <60486987+Krzysztof318@users.noreply.github.com>
@RogerBarreto @Krzysztof318
Update dotnet/src/Connectors/Connectors.HuggingFace/Core/HuggingFaceC…
b19f19f 
…lient.cs

Co-authored-by: Krzysztof Kasprowicz <60486987+Krzysztof318@users.noreply.github.com>
@RogerBarreto
Merge branch 'main' into features/hugging-face-httpclient
72e51bf
@RogerBarreto
Fix Warnings
e0f774f
@RogerBarreto
Update to smaller sample
5c8aaa6
@RogerBarreto
Merge branch 'main' into features/hugging-face-httpclient
2daee65
@RogerBarreto
Merge from httpclient changes
9bc84ae
@RogerBarreto
Merge Latest Hugging Face Client Changes
7b9468f
@RogerBarreto
Adding UTs, fixing namings
250daed
@RogerBarreto RogerBarreto requested a review from a team as a code owner February 26, 2024 12:34
@RogerBarreto RogerBarreto self-assigned this Feb 26, 2024
@markwallace-microsoft markwallace-microsoft added .NET Issue or Pull requests regarding .NET code kernel Issues or pull requests impacting the core kernel kernel.core labels Feb 26, 2024
dmytrostruk
dmytrostruk reviewed Feb 26, 2024
View reviewed changes
dotnet/src/Connectors/Connectors.HuggingFace/HuggingFaceKernelBuilderExtensions.cs Outdated Show resolved Hide resolved
...t/src/Connectors/Connectors.HuggingFace.UnitTests/ImageToText/HuggingFaceImageToTextTests.cs Show resolved Hide resolved
dotnet/src/Connectors/Connectors.HuggingFace/HuggingFaceKernelBuilderExtensions.cs Outdated Show resolved Hide resolved
dotnet/src/Connectors/Connectors.HuggingFace/ImageToText/HuggingFaceImageToTextService.cs Outdated Show resolved Hide resolved
dotnet/src/Connectors/Connectors.HuggingFace/ImageToText/HuggingFaceImageToTextService.cs Outdated Show resolved Hide resolved
dotnet/samples/KernelSyntaxExamples/Example85_ImageToText.cs Outdated Show resolved Hide resolved
...t/src/Connectors/Connectors.HuggingFace.UnitTests/ImageToText/HuggingFaceImageToTextTests.cs Outdated Show resolved Hide resolved
dotnet/src/Connectors/Connectors.HuggingFace/HuggingFaceKernelBuilderExtensions.cs Outdated Show resolved Hide resolved
dotnet/src/Connectors/Connectors.HuggingFace/HuggingFaceKernelBuilderExtensions.cs Outdated Show resolved Hide resolved
dotnet/src/Connectors/Connectors.HuggingFace/ImageToText/HuggingFaceImageToTextService.cs Outdated Show resolved Hide resolved
@Krzysztof318
Copy link
Contributor

Krzysztof318 commented Feb 26, 2024 •
edited

@RogerBarreto
Do we really need special ImageToText service, since chatcompletionservice already supports this?
This is probably a mistake in the issue #4947 but it mentions TextToImage not ImageToText.

@RogerBarreto @dmytrostruk
Apply Dmytro suggestions from code review
2c21797 
Co-authored-by: Dmytro Struk <13853051+dmytrostruk@users.noreply.github.com>
@RogerBarreto
Copy link
Member Author

RogerBarreto commented Feb 26, 2024 •
edited

@Krzysztof318 that was an error in the description I updated the issue.
The intended support was to Blip for Image to Text models.

@RogerBarreto RogerBarreto added this pull request to the merge queue Feb 27, 2024
Merged via the queue into microsoft:main with commit 0e10dd5 Feb 27, 2024
18 checks passed
@RogerBarreto RogerBarreto deleted the features/hugging-face-imagetotext branch February 27, 2024 10:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dmytrostruk dmytrostruk dmytrostruk approved these changes

@SergeyMenshykh SergeyMenshykh SergeyMenshykh approved these changes

Assignees

@RogerBarreto RogerBarreto

Labels
kernel.core kernel Issues or pull requests impacting the core kernel .NET Issue or Pull requests regarding .NET code
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

.Net: Update Hugging Face connector
5 participants
@RogerBarreto @Krzysztof318 @dmytrostruk @SergeyMenshykh @markwallace-microsoft