Chat control should support images

[AlexandreSajus]

Description

The chat control of #1092 is really cool! It would be a good addition to support images in the messages of the chat. This would be especially useful since now LLMs start supporting image vision and image generation. I could create cool demos around this chat control.

Florian:
People are also using chat demos to visualize tables inside their chat. This could also be a nice addition.

Acceptance Criteria

• Ensure new code is unit tested, and check code coverage is at least 90%.
• Create related issue in taipy-doc for documentation and Release Notes.
• Check if a new demo could be provided based on this, or if legacy demos could be benefit from it.
• Ensure any change is well documented.

Code of Conduct

• I have checked the existing issues.
• I am willing to work on this issue (optional)

[ioannastantzou]

Hello there I think I could work on this issue!

[FlorianJacta]

Great, I will assign it to you!

[ioannastantzou]

hello again! can you specify where the chat code exists?

[ioannastantzou]

hello again! can you specify where the chat code exists?

is it taipy\frontend\taipy-gui\src\components\Taipy\chat.tsx ? @FabienLelaquais @FredLL-Avaiga

[github-actions]

This issue has been labelled as "🥶Waiting for contributor" because it has been inactive for more than 14 days. If you would like to continue working on this issue, please add another comment or create a PR that links to this issue. If a PR has already been created which refers to this issue, then you should explicitly mention this issue in the relevant PR. Otherwise, you will be unassigned in 14 days. For more information please refer to the contributing guidelines.

[yaten2302]

Hey @FlorianJacta @AlexandreSajus , could you please assign this issue to me, if no one is working on this?

[FlorianJacta]

@ioannastantzou is normally working on it. Could you confirm @ioannastantzou ?

[yaten2302]

@ioannastantzou is normally working on it. Could you confirm @ioannastantzou ?

It's ok if someone is working on this, but if not I'm in 👍

[github-actions]

This issue has been labelled as "🥶Waiting for contributor" because it has been inactive for more than 14 days. If you would like to continue working on this issue, please add another comment or create a PR that links to this issue. If a PR has already been created which refers to this issue, then you should explicitly mention this issue in the relevant PR. Otherwise, you will be unassigned in 14 days. For more information please refer to the contributing guidelines.

[github-actions]

This issue has been unassigned automatically because it has been marked as "🥶Waiting for contributor" for more than 14 days with no activity.

[yaten2302]

Hey @FlorianJacta @AlexandreSajus , can this issue be assigned to me?

Also, I would like to ask that in here, do we've to add a files icon on the left side of the message box (similar to what is present in chatgpt)?

[FlorianJacta]

What do you mean by a file icon?

Can this be assigned? @FredLL-Avaiga @FabienLelaquais

[FlorianJacta]

@yaten2302, I have assigned you this issue!

@sOnU1002 I have already assigned you to another issue. @mehulmathur16, there are other issues to take!

[yaten2302]

@FlorianJacta , thanks for assigning this issue 👍
I'll create a PR for this by Tuesday or Wednesday max!

[quest-bot]

New Quest!

A new Quest has been launched in @Avaiga’s repo.
Merge a PR that solves this issue to loot the Quest and earn your reward.

---

Some loot has been stashed in this issue to reward the solver!

🗡 Comment @quest-bot embark to check-in for this Quest and start solving the issue. Other solvers will be notified!

⚔️ When you submit a PR, comment @quest-bot loot #1314 to link your PR to this Quest.

Questions? Check out the docs.

[yaten2302]

Apologies for the huge delay on this issue. Actually, I was having a doubt that in this file - frontend/taipy-gui/src/components/Taipy/Chat.tsx, it's showing 2 problems

1.

I haven't changed anything in the codebase, is this issue arising because something is not installed? Like in the 1st one, it's showing that - not able to find module Grid2?

@FredLL-Avaiga @AlexandreSajus

[FredLL-Avaiga]

I think your installed version of Mui is not up-to-date.
My advice would be to remove the node_modules directory and the package-lock.json file and reinstall everything from scratch

[yaten2302]

@FredLL-Avaiga , I re cloned the taipy repo and tried to install the dependencies, but now it's showing this error :/

[FredLL-Avaiga]

You first need to build taipy-gui
And then install taipy-gui in taipy with npm i in taipy
Then you can build taipy

[yaten2302]

Thanks @FredLL-Avaiga , now it's working👍

Also, I wanted to ask about this comment, like for the chat to support, we need an icon to show the user to attach an image. Should I add an icon in the chat itself (as in chatGPT)?

[yaten2302]

Also, I wanted to ask that in this PR #1092, you took a screenshot of the chat element, could you please guide that if I want to test the element locally, how can I view them? (as you've shown in this PR)

[FredLL-Avaiga]

Also, I wanted to ask that in this PR #1092, you took a screenshot of the chat element, could you please guide that if I want to test the element locally, how can I view them? (as you've shown in this PR)

You create a simple python script and run it locally
You can find simple examples in doc/examples/gui

[FredLL-Avaiga]

Thanks @FredLL-Avaiga , now it's working👍

Also, I wanted to ask about this comment, like for the chat to support, we need an icon to show the user to attach an image. Should I add an icon in the chat itself (as in chatGPT)?

I'm not sure to understand your question.
If it is about adding an icon button to allow the user to choose an image then yes,I think it should go in the input field besides the send icon.
But we also want to support drag and drop and copy paste...

[yaten2302]

@FredLL-Avaiga , yes, I was asking about this only. I got it now 👍 I'll add an icon in the txt bar for adding images.
First, I'll create a draft PR for this and then if it looks good, then I'll also add support for drag and drop and copy paste.

[yaten2302]

@FredLL-Avaiga , I've a doubt, that when I'm running this command - py -m pip install taipy-location, it's showing this error:

ERROR: Could not install packages due to an OSError: [WinError 2] The system cannot find the file specified: 'C:\\Python312\\Scripts\\cftp.exe' -> 'C:\\Python312\\Scripts\\cftp.exe.deleteme

I tried searching about it on stackoverflow, found this solution but, still it's not wokring :/ - py -m pip install --user taipy-location.
Actually, my application is still using taipy main lib even if I've changed certain things in the taipy elements(locally).

Any suggestions regarding how can I fix this?

[FredLL-Avaiga]

What is taipy-location ?

[yaten2302]

@FredLL-Avaiga ,the taipy location is: d:/taipy and the taipy-prac(in which the main.py file is present) is also in the d:/

[yaten2302]

@FredLL-Avaiga , I was trying to install my local taipy package in a sample project, but when I'm click on the the location of import taipy.gui.builder as tgb, it's showing this:

But, since, my taipy file is located in d:/taipy, shouldn't it show this location? I tried to uninstall taipy as well and then run thes commands - py -m pip install taipy-location and py -m pip install --user taipy-location, since the 1st one was showing this error:

ERROR: Could not install packages due to an OSError: [WinError 2] The system cannot find the file specified: 'C:\\Python312\\Scripts\\cftp.exe' -> 'C:\\Python312\\Scripts\\cftp.exe.deleteme

[Satoshi-Sh]

If you don't mind, can I start working on this issue?

[yaten2302]

@Satoshi-Sh , thanks for the offer, but could you give me some time, currently working on to fix it :)

[Satoshi-Sh]

Hi @yaten2302 , I think several people can work on the same issue. I will wait for a response from the maintainers.

[FlorianJacta]

Yes, multiple people can work on the same issue and you can also collaborate together on it

[Satoshi-Sh]

@quest-bot embark

[quest-bot]

@Satoshi-Sh has embarked on their Quest. 🗡

• @Satoshi-Sh has been on GitHub since 2020.
• They have merged 138 public PRs in that time.
• Their swords are blessed with Python and HTML magic ✨
• They have contributed to this repo before.

This is not an assignment to the issue. Please check the repo’s contribution guidelines before submitting a PR.

Questions? Check out the docs.

[quest-bot]

🧚 @Satoshi-Sh has submitted PR #2078 and is claiming the loot.

Keep up the pace, or you'll be left in the shadows.

Questions? Check out the docs.

[github-actions]

This issue has been labelled as "🥶Waiting for contributor" because it has been inactive for more than 14 days. If you would like to continue working on this issue, please add another comment or create a PR that links to this issue. If a PR has already been created which refers to this issue, then you should explicitly mention this issue in the relevant PR. Otherwise, you will be unassigned in 14 days. For more information please refer to the contributing guidelines.