Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiagents, Multimodality and LeChat automation #64

Merged
merged 40 commits into from
Mar 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
3106155
test(single_chatheads): Test order and new tests for multimodality
ugorsahin Mar 16, 2024
da054b6
feat(reset_thread): now it is possible to reset chat, disabled model …
ugorsahin Mar 16, 2024
2d9f032
fix(logging): uses it's own logging
ugorsahin Mar 16, 2024
0d69292
feat(multiagent): FIrst commit of Multiagent subpackage
ugorsahin Mar 16, 2024
7245a01
feat(conversation): First usable conversation subpackage
ugorsahin Mar 16, 2024
091372b
feat(image): Implementation of image upload feature
ugorsahin Mar 16, 2024
2101abf
feat(gemini_multimodal): It is possible to upload image to Gemini
ugorsahin Mar 16, 2024
e10aef8
feat(lechat): Mistral Le Chat is automated
ugorsahin Mar 16, 2024
cedbdf1
fix:(test_utils): Fixed configuration of test browser
ugorsahin Mar 16, 2024
f780317
test(cat): Cat image to test multimodality
ugorsahin Mar 16, 2024
259a86d
fix(claude): FIxed a response error
ugorsahin Mar 16, 2024
5316f84
fix(logging_interacting): Logging changed to instance based logger an…
ugorsahin Mar 16, 2024
c3fe239
fix(logging_and_driver_arguments): It is now possible to pass driver …
ugorsahin Mar 16, 2024
d2db702
fix(init): Initialization fixes
ugorsahin Mar 16, 2024
a40bcda
test(general): Configuration files added
ugorsahin Mar 16, 2024
38edfce
fix(object_markers): FIxed old markers and new file format
ugorsahin Mar 16, 2024
ab15e17
feat(is_url_check_filetype): new functions to check filetype and if t…
ugorsahin Mar 16, 2024
b22ab22
doc(README): New version of the readme
ugorsahin Mar 16, 2024
b2cc908
doc: New documentation files, various structural fixes
ugorsahin Mar 18, 2024
77efac0
doc(style): fix style
ugorsahin Mar 18, 2024
0c99ca4
fix(logging): import logging removed
ugorsahin Mar 18, 2024
80de9cb
test(lechat_conv): Instruction correction, model switch test order
ugorsahin Mar 18, 2024
d7be3bb
fix(import): Changed init file
ugorsahin Mar 18, 2024
5ef7b6b
fix: save file format fix, init structure
ugorsahin Mar 18, 2024
ad69645
Version 0.5.0
ugorsahin Mar 18, 2024
e63eaa4
Version 0.5.0
ugorsahin Mar 18, 2024
e72a8a6
Merge branch 'multimodal_lechat_multiagent' of github.com:ugorsahin/T…
ugorsahin Mar 18, 2024
545f2f9
fix: dependency include PyYAML
ugorsahin Mar 18, 2024
ef0d270
Version 0.5.0
ugorsahin Mar 18, 2024
7233fb7
fix: Include mergedeep
ugorsahin Mar 18, 2024
166433d
fix(import): Changed init file
ugorsahin Mar 18, 2024
acb98f8
fix: save file format fix, init structure
ugorsahin Mar 18, 2024
33f319d
Version 0.5.0
ugorsahin Mar 18, 2024
90a5fe5
fix: dependency include PyYAML
ugorsahin Mar 18, 2024
e9fc3c7
fix: Include mergedeep
ugorsahin Mar 18, 2024
f313ab6
fix: dependency imports
ugorsahin Mar 18, 2024
a771654
test: adding chrome parameters to avoid memory crash at kubernetes
ugorsahin Mar 18, 2024
14a57e3
test(huggingchat): disable search web, unstable feature
ugorsahin Mar 18, 2024
0d3b780
test(conversation): improved prompt for candidate
ugorsahin Mar 18, 2024
d993820
test: change Pi to LeChat
ugorsahin Mar 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -32,4 +32,7 @@ MANIFEST
_build

# CircleCI
artifacts
artifacts

# Testing
talkingheads_userprofile
21 changes: 21 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
## 0.5.0 (2024-03-18)

### Feat

- **is_url_check_filetype**: new functions to check filetype and if the string is url
- **lechat**: Mistral Le Chat is automated
- **gemini_multimodal**: It is possible to upload image to Gemini
- **image**: Implementation of image upload feature
- **multiagent**: First commit of Multiagent subpackage including Conversation
- **reset_thread**: now it is possible to reset chat, disabled model switch

### Fix

- **import**: Changed init file
- **logging**: import logging removed
- **object_markers**: Fixed old markers and new file format
- **init**: Initialization fixes
- **logging_and_driver_arguments**: It is now possible to pass driver arguments as dictionary
- **logging_interacting**: Logging changed to instance based logger and fixed a bug in interaction
- **claude**: Fixed a response error
- **logging**: uses it's own logging
54 changes: 30 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,20 +20,23 @@
<img src='https://github.com/ugorsahin/TalkingHeads/actions/workflows/codeql.yml/badge.svg' alt='CodeQL Status' />
</a>
<a href="https://codecov.io/gh/ugorsahin/TalkingHeads" >
<img src="https://codecov.io/gh/ugorsahin/TalkingHeads/graph/badge.svg?token=YPXKNXSZAD"/>
<img src="https://badgen.net/codecov/c/github/ugorsahin/talkingheads"/>
</a>
<a href="https://github.com/psf/black"><img alt="Code style: black" src="https://img.shields.io/badge/code%20style-black-000000.svg"></a>
<a href="https://github.com/astral-sh/ruff"><img alt="Code style: black" src="https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json"></a>
</p>

Welcome to TalkingHeads! 💬

TalkingHeads is a versatile Python library that serves as an interface for seamless communication with ChatGPT, Claude, Copilot, Gemini, HuggingChat, and Pi 🤖
TalkingHeads is a Python library that serves as an interface for seamless communication with ChatGPT, Claude, Copilot, Gemini, LeChat, HuggingChat, and Pi 🤖

By leveraging the power of browser automation, this library enables users to effortlessly interact with online LLM tools, providing a streamlined and automated approach to generate responses. 🚀✨
By leveraging the power of browser automation, this library enables users to effortlessly interact with online chat agents 🚀✨

You can utilize more than one agent by using multiagent module, and improve your workflow with an ensemble of models!

# Prerequisites 📋

Before you begin, please ensure that you have Chrome installed on your system. To successfully pass the Cloudflare robot test, it is necessary to have undetected-chrome. 🌐🔒
- Install Chrome
- Register to the provider you would like to use (or not, and use Pi!)

## Installation

Expand All @@ -54,34 +57,37 @@ from talkingheads import PiClient

chathead = PiClient()

answer = chathead.interact("Hello, how are you today")
response = chathead.interact("Hello, how are you today")

print(answer)
print(response)
```

## Features

Features | Gemini | Claude | Copilot | HuggingChat | ChatGPT | Pi |
|--------|------|--------|---------|-------------|---------|----|
Use without login | ❌ | ❌ | ✅ | ❌ | ❌ | ✅ |
Login* | ➖ | ➖ | ➖ | ✅ | ✅ | ➖ |
Interact | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
New Chat | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Regenerate Response | | ✅ | ❌ | ❌ | ✅ | ❌ |
Set custom interactions | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ |
Search Web | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ |
Plugins | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ |
Switch Model | ❌ | ❌ | ✅** | ✅ | ✅ | ✅ |
Features | Claude | ChatGPT | Copilot | Gemini | LeChat | HuggingChat | Pi |
|-------------------|--------|---------|---------|--------|--------|-------------|----|
Use without login | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ✅ |
Login* | ➖ | ✅ | ➖ | ➖ | ✅ | ✅ | ➖ |
Interact | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
New Chat | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Regenerate Response | ✅ | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ |
Custom Interactions | | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ |
Search Web | ❌ | ❌ | ✅ | ❌ | ❌ | ✅ | ❌ |
Plugins | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ |
Switch Model | ❌ | ✅ | ✅** | ❌ | ✅ | ✅ | ✅ |
Multimodal (Visual) | ✅ | ➖*** | ✅ | ✅ | ❌ | ❌ | ❌ |

✅ (Auto) Save the conversation as csv, h5, html, json, orc, pkl, xlsx, xml

- ✅ : Functionality exists and implemented
- ❌ : Functionality does not exist
- ➖ : Fuctionality exists but not implemented

\* The first time login is manually done for Gemini, Claude and Copilot. Check documentation for first time setup. Please take a look at [FAQ](FAQ.md) to set up.
\* You should use a user profile and login manually to use Gemini, Claude and Copilot.Please take a look at documentation and [FAQ](FAQ.md) to set up a user profile.

\*\* The modes of Copilot (Creative, Balanced and Precise) are accessible with model switch. However, it is unclear if they are different models.

\*\*\* ChatGPT has multimodality, but only for premium users, donate me a premium account if you need that to be implemented!

## Issues & Contribution

Feel free to dive in, share your knowledge, and collaborate. I would be happy to answer any questions or accept your contributions. Let me know in issues if you need anything.
Expand All @@ -94,15 +100,15 @@ Please indicate your issue with a tag enclosed by square brackets: [FEATURE], [B
You can still run your code to connect ChatGPT, follow the below tutorial

```bash
export OPENAI_UNAME=<your@email>
export OPENAI_PWD=><password>
export ChatGPT_UNAME=<your@email>
export ChatGPT_PWD=><password>
```
```python
from talkingheads import ChatGPTClient

chathead = ChatGPTClient()

answer = chathead.interact("Hello, how are you today")
response = chathead.interact("Hello, how are you today")

print(answer)
print(response)
```
3 changes: 3 additions & 0 deletions docs/_static/custom_style.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
.wy-nav-content {
max-width: 900px;
}
13 changes: 11 additions & 2 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,28 +11,37 @@
sys.path.insert(0, os.path.abspath("../src/"))

project = "TalkingHeads"
copyright = "2023, Ugur Sahin"
copyright = "2024, Ugur Sahin"
author = "Ugur Sahin"

# -- General configuration ---------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration

extensions = [
"sphinx_markdown_tables",
"sphinx.ext.napoleon",
"sphinx.ext.autodoc",
"sphinx.ext.autosummary",
"sphinx.ext.doctest",
"sphinx.ext.napoleon",
"myst_parser",
]

templates_path = ["_templates"]
exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]

# Markdown support
source_parsers = {
'.md': 'recommonmark.parser.CommonMarkParser',
}

source_suffix = ['.rst', '.md']

# -- Options for HTML output -------------------------------------------------
# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output

html_theme = "sphinx_rtd_theme"
html_static_path = ["_static"]
html_css_files = ['custom_style.css']

add_module_names = True
autodoc_member_order = "groupwise"
Expand Down
6 changes: 4 additions & 2 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,10 @@ By leveraging the power of browser automation, this library enables users to int
:maxdepth: 2
:caption: Contents:

tutorial/features
tutorial/getting_started.md
tutorial/features.md
tutorial/getting_started
tutorial/multiagent
tutorial/conversation
source/talkingheads

Indices and tables
Expand Down
56 changes: 23 additions & 33 deletions docs/source/talkingheads.model_library.rst
Original file line number Diff line number Diff line change
@@ -1,61 +1,51 @@
talkingheads.model\_library package
talkingheads.model\_library
===================================

Submodules
----------

talkingheads.model\_library.Gemini module
---------------------------------------
talkingheads.model\_library.chatgpt
------------------------------------------

.. automodule:: talkingheads.model_library.Gemini
.. automodule:: talkingheads.model_library.chatgpt
:members:
:undoc-members:
:show-inheritance:

talkingheads.model\_library.chatgpt module
------------------------------------------
talkingheads.model\_library.claude
-----------------------------------------

.. automodule:: talkingheads.model_library.chatgpt
.. automodule:: talkingheads.model_library.claude
:members:
:undoc-members:
:show-inheritance:

talkingheads.model\_library.huggingchat module
----------------------------------------------
talkingheads.model\_library.copilot
------------------------------------------

.. automodule:: talkingheads.model_library.huggingchat
.. automodule:: talkingheads.model_library.copilot
:members:
:undoc-members:
:show-inheritance:

talkingheads.model\_library.pi module
-------------------------------------
talkingheads.model\_library.gemini
-----------------------------------------

.. automodule:: talkingheads.model_library.pi
.. automodule:: talkingheads.model_library.gemini
:members:
:undoc-members:
:show-inheritance:

talkingheads.model\_library.copilot module
-------------------------------------
talkingheads.model\_library.huggingchat
----------------------------------------------

.. automodule:: talkingheads.model_library.copilot
.. automodule:: talkingheads.model_library.huggingchat
:members:
:undoc-members:
:show-inheritance:

talkingheads.model\_library.claude module
-------------------------------------
talkingheads.model\_library.lechat
-----------------------------------------

.. automodule:: talkingheads.model_library.claude
.. automodule:: talkingheads.model_library.lechat
:members:
:undoc-members:
:show-inheritance:

Module contents
---------------
talkingheads.model\_library.pi
-------------------------------------

.. automodule:: talkingheads.model_library
.. automodule:: talkingheads.model_library.pi
:members:
:undoc-members:
:show-inheritance:
:show-inheritance:
7 changes: 7 additions & 0 deletions docs/source/talkingheads.multiagent.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
talkingheads.multiagent
===============================

.. automodule:: talkingheads.multiagent
:members:
:undoc-members:
:show-inheritance:
25 changes: 6 additions & 19 deletions docs/source/talkingheads.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
talkingheads package
====================
talkingheads API documentation
==============================

Subpackages
-----------
Expand All @@ -8,38 +8,25 @@ Subpackages
:maxdepth: 4

talkingheads.model_library
talkingheads.multiagent

Submodules
----------

talkingheads.base\_browser module
talkingheads.base\_browser
---------------------------------

.. automodule:: talkingheads.base_browser
:members:
:undoc-members:
:show-inheritance:

talkingheads.object\_map module
talkingheads.object\_map
-------------------------------

.. automodule:: talkingheads.object_map
:members:
:undoc-members:
:show-inheritance:

talkingheads.utils module
talkingheads.utils
-------------------------

.. automodule:: talkingheads.utils
:members:
:undoc-members:
:show-inheritance:

Module contents
---------------

.. automodule:: talkingheads
:members:
:undoc-members:
:show-inheritance:
52 changes: 52 additions & 0 deletions docs/tutorial/conversation.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
How to use Conversation
=======================

Conversation is a special case of multiagent interactions, where only two agents are involved. The configuration file for conversation is similar to the multiagent configuration, but a conversation only has two agents. The following example demonstrates an example:

.. code-block:: python

conversation_head = Conversation('path/to/config.yaml')
interviewer = """Assume you are a researcher and will conduct a human cognition test.
Instruct the subject with the following instructions, one by one. Only ask for one instruction at a time.
Here are the instructions
1) Without further ado, just write the following: "Book."
2) Calculate 4x10.
3) Name three countries with the largest total area in North America.

Start by greeting the subject."""

candidate = (
"You accepted participating in a research study "
"and you will follow the instructions of the researchers."
"Answer with proper sentences."
)

responses = conversation_head.start_conversation(
intro_prompt_1=interviewer, intro_prompt_2=candidate
)
print(response[0]) # interviewer
print(response[1]) # candidate

Output:

.. code-block::

Greetings! Thank you for participating in our human cognition test. Let's begin with the first instruction:
Please write the word "Book."

Book

.. code-block:: python

responses = conversation_head.continue_conversation()
print(response[0]) # interviewer
print(response[1]) # candidate

Output:
.. code-block::

Great job! Now, onto the second instruction:
Please calculate 4 multiplied by 10.

4 multiplied by 10 equals 40.

Loading
Loading