Bot cannot access images added by clipboard

[herrhamilton]

Hi there,
I am working on a bot which receives messages from Teams via the Botframework and sends them to another application. I'm using .NET Core.

When sending a normal file, I can access it with the Attachment.Content["downloadUrl"] of the activity which I receive from the Botframework.

But when I insert images from the clipboard, it is stored in another way, with the Activity I get two attachments with different urls (one for layout and one for the image resource, I suppose?), like the image below, but when trying to GET the image, I receive an 401 Unauthorized.

Is there a way to access these images pasted from the clipboard?
Or are these only visible/accessible from inside Microsoft Teams?
This way the Teams user thinks everything is alright, but the Bot/other application has no access to them.

Also, where do these clipboard-images be safed afterwards, if not in the SharePoint like "properly uploaded" files? Does the user ever has access to them again?

This applies to Stickers, too. Gifs can be sent without problem, because it's a giphy url.

I am using the OnMessageActivityAsync overload described in the Microsoft Docs

public class BotActivityHandler : Microsoft.Bot.Builder.Teams.TeamsActivityHandler
    {
        protected override async Task OnMessageActivityAsync(ITurnContext<IMessageActivity> turnContext, CancellationToken cancellationToken)
        {
            // it's either this url or the one parsed from the html, but both receive a 401 with a GET request like below or with Postman
            var imageUrl = turnContext.Activity.Attachments.Where(x => !x.ContentUrl.IsNullOrEmpty()).ToList().First().ContentUrl;

            var response = await myHttpClient.GetAsync(imageUrl);
            var img = await response.Content.ReadAsStreamAsync();
        }

The turnContext includes an Activity with two Attachments: One has Content like
<div><div>\n<div><span><img src=\"https://eu-api.asm.skype.com/v1/objects/[...]/views/imgo\" width=\"247\" id=\"[...]\" itemscope=\"\" itemtype=\"http://schema.skype.com/AMSImage\"\"></span>\n\n</div>\n\n\n</div>\n</div>

And the other one a ContentUrl like

https://smba.trafficmanager.net/emea/v3/attachments/[...]/views/original

[ghost]

Hi joshamil! Thank you for bringing this issue to our attention. We will investigate and if we require further information we will reach out. Best regards, Teams Platform

[Subhasish-msft]

@joshamil ,

MS teams framework treat attachment object and clipboard image differently.

• When user paste an image in MS teams chat, it does not treat it as an usual attachment. Rather it receives as an image type and stores it in MS Teams internal storage and sends the Image URL in Activity.Attachment[0].
• you need to download the image content from that URL and upload to your blob storage.
• MS Teams attachment URLs are secured by a JWT Token, so we need to pass the token from our bot to access it.

Here is a sample c# code to get the image form MS Teams URL and save it to solution file. You can add this logic in ActivityHandler method.

if (turnContext.Activity.Attachments != null && turnContext.Activity.Attachments.Count > 1
            && turnContext.Activity.Attachments[0].ContentType=="image/*")
        {
            var attachment = turnContext.Activity.Attachments[0];
            using (HttpClient httpClient = new HttpClient())
            {
                // MS Teams attachment URLs are secured by a JwtToken, so we need to pass the token from our bot.
                var token = await new MicrosoftAppCredentials("appId", "password").GetTokenAsync();
                httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", token);
                var responseMessage = await httpClient.GetAsync(attachment.ContentUrl);
                var contentLenghtBytes = responseMessage.Content.Headers.ContentLength;
                // You could not use this response message to fetch the image for further processing.
                if (responseMessage.StatusCode == System.Net.HttpStatusCode.OK)
                {
                    Stream attachmentStream = await responseMessage.Content.ReadAsStreamAsync();
                    attachmentStream.Position = 0;
                    var image = System.Drawing.Image.FromStream(attachmentStream);
                    image.Save(@"ImageFromUser.png");
                    await turnContext.SendActivityAsync($"Attachment of {attachment.ContentType} type and size of {contentLenghtBytes} bytes received.");
                }
            }
        }

[herrhamilton]

Thank you @Subhasish-msft for your great explanation and the code sample!
I completely forgot about the MicrosoftAppCredentials.

Now it works as intended :)