Add Drag-and-Drop Document Upload, Web Scraping, and Alternative Model Support

[mironkraft]

I would like to propose a few improvements for the Auto-GPT repository to enhance its functionality and usability:

1. Drag-and-drop document upload: Implement a drag-and-drop file input area in the front-end, allowing users to easily upload documents to the web application. The uploaded document's content should be processed and passed to the GPT model for further processing. Consider using libraries like Dropzone.js to facilitate this feature.

2. Web scraping integration: Add the capability to perform web scraping and pass the extracted content to the Auto-GPT model for processing. Create a separate Python module or script to handle web scraping and integrate it with the existing server code. Expose this functionality via an API or user interface. Libraries like Beautiful Soup or Scrapy can be used to implement this feature.

3. Alternative model support: Allow users to utilize the application without an API key by replacing the GPT-3 model with another pre-trained language model that does not require an API key, such as GPT-2 or GPT-Neo from Hugging Face's Transformers library. Moreover, explore the possibility of integrating a different model like VICUNA, which would involve modifying the server code, updating the model loading and text generation functions, and making necessary changes to the front-end.

These improvements would significantly enhance the user experience and functionality of the Auto-GPT web application.

[anonhostpi]

Isn't web scraping already supported with WebDriver? Isn't that half of the reason that this repo exists?

[mironkraft]

As far as I am concerned, no.

[deanshelton913]

could you elaborate? I agree that scraping seems to be covered with the selenium command, no?

[anonhostpi]

Other APIs to add would be OPT

[Aavas13]

this could be the way to get any file or folder Technically speaking, the way it works is when you upload a file, the text is extracted from it and chunked using a chunking algorithm – and these chunks are sent to the OpenAI embeddings API to get a vector embedding (basically a long sequence of numbers) for each chunk. Then these vector embeddings are stored in a VectorDB like pinecone. Then when a question comes in, it is also converted to an embedding vector, and that vector is used to query the vector database, to get the most relevant, close matches within the multi-dimensional vector space – this ends up being the most relevant context chunk(s) .

[mironkraft]

So, from your point of view, how should be this done?

[Aavas13]

i got this idea from reddit post: https://www.reddit.com/r/ChatGPT/comments/138jpj8/i_built_an_open_source_website_that_lets_you/jiym8oe?utm_source=share&utm_medium=android_app&utm_name=androidcss&utm_term=1&utm_content=share_button

[Aavas13]

using chat gpt i got this : import os
import openai
import pinecone
from chunking import chunk_text
from config import OPENAI_API_KEY, PINECONE_API_KEY

pinecone.deinit()
pinecone.init(api_key=PINECONE_API_KEY)
pinecone.create_index(index_name="embedding_space", metric="cosine")
pinecone.create_namespace(namespace="file_embeddings")

def extract_text(file_path: str) -> str:
with open(file_path, "r") as file:
return file.read()

def process_file(file_path: str) -> None:
text = extract_text(file_path)
chunks = chunk_text(text)
embeddings = []

openai.api_key = OPENAI_API_KEY

for chunk in chunks:
    response = openai.Embed.create(document=chunk, model="gpt-4-embeddings")
    embeddings.append(response["embedding"])

file_id = os.path.basename(file_path)
pinecone.upsert(index_name="embedding_space", ids=[file_id], vectors=embeddings)

def process_folder(folder_path: str) -> None:
for root, _, files in os.walk(folder_path):
for file in files:
file_path = os.path.join(root, file)
process_file(file_path)

pinecone.deinit()
Now you can use the process_file and process_folder functions to process any file or folder. The text will be extracted, chunked, and the chunks will be sent to the OpenAI embeddings API. The resulting vector embeddings will be stored in a Pinecone index called "embedding_space".

To process a single file:

python
Copy code
process_file("path/to/your/file.txt")
To process a folder:

python
Copy code
process_folder("path/to/your/folder")
Please note that this is a basic implementation and might need adjustments depending on the type of files you want to process.

[mironkraft]

This should be ordered as a project request to be implemented

Anyways, I doubt that will be a fast dev, might take time, if not, always is possible to redefine on a fork or even other type of interface with other model of language

Not bad at all the idea

[Aavas13]

there is always chat gpt to check for it.

[Aavas13]

here is an code i got:
file_processing.py that combines both chunking and processing files and folders:

python
Copy code
import os
import openai
import pinecone
from dotenv import load_dotenv

load_dotenv()
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")
PINECONE_API_KEY = os.getenv("PINECONE_API_KEY")

def chunk_text(text: str, chunk_size: int = 1024) -> list[str]:
return [text[i:i + chunk_size] for i in range(0, len(text), chunk_size)]

pinecone.deinit()
pinecone.init(api_key=PINECONE_API_KEY)
pinecone.create_index(index_name="embedding_space", metric="cosine")
pinecone.create_namespace(namespace="file_embeddings")

def extract_text(file_path: str) -> str:
with open(file_path, "r") as file:
return file.read()

def process_file(file_path: str) -> None:
text = extract_text(file_path)
chunks = chunk_text(text)
embeddings = []

openai.api_key = OPENAI_API_KEY

for chunk in chunks:
    response = openai.Embed.create(document=chunk, model="gpt-4-embeddings")
    embeddings.append(response["embedding"])

file_id = os.path.basename(file_path)
pinecone.upsert(index_name="embedding_space", ids=[file_id], vectors=embeddings)

def process_folder(folder_path: str) -> None:
for root, _, files in os.walk(folder_path):
for file in files:
file_path = os.path.join(root, file)
process_file(file_path)

pinecone.deinit()
Now you have a single file that handles chunking and processing files and folders

[Charlotte-br560]

Great work on consolidating chunking and file processing!