CLIP - Search Engine - Text Requests to Images

Made as the project of Case Study Module from the Engineer Path of CY TECH.

It is a project based on the connection between text and images using CLIP. We realized the project using Python, Fiftyone, Pinecone and Streamlit. The project is deployable through Docker. We describe the project with the following sections.

• I. Project Demonstration
• II. Repository Structure
• III. Project Description
• IV. Project Deployment
  • Online Deployment
  • Local Deployment

I. Project Demonstration

II. Repository Structure

The structure of this repository is described below. It contains the source code of the project as well as the Dockerfile used to build the Docker Image.

├── app.py  
├── Dockerfile  
├── requirements.txt    
└── src    
    ├── embedding.py  
    ├── fiftyone_datasets_models.py  
    ├── static    
    │   └── images  
    │       └── iconCYTECH.png  
    └── utils.py

III. Project Description

In the recent years, AI has been improving quite a lot. The purpose of this project is to demonstrate the power of existing Computer Vision Model (CLIP from OpenAI) to build a Search Engine. The Search Engine is based on the connection between text and images. Thus, the user can type a text and the Search Engine will return the most relevant images based on the text.

To realize this project, we used the following tools.

• Fiftyone
• Pinecone
• Streamlit
• Docker

IV. Project Deployment

Online Deployment

The interface is hosted through Streamlit. It is available at the following Link.

Local Deployment

The local deployment is based on Docker. Thus, to correctly launch our interface, the Docker software must be installed. The deployment is relatively easy since it consists of the three steps described below.

1° - Clone the Repository

Firstly, you must clone this reposity. To do so, you can use the following command.

git clone https://github.com/dorian-roux/Clip-Search-Engine

2° - Obtain the Pinecone Key Parameters

Secondly, you must create a find your Picone Key parameters. To do so, you need to register in their website. Then you need to find the Pinecone Console and keep both the Pinecone Key Value and Pinecone Key Environement

3° - Build the Docker Image

Thirdly, you need to build the Docker Image based on this repository Dockerfile. To do so, you can use the following command.

docker build -t {TAG} .

Note:   {TAG} is the name you want to give to your Docker Image. It is used to identify it later.

4° - Run the Docker Container

Finally, you need to run the Docker Container based on the Docker Image you just built. To do so, you can use the following command.

docker run -p 8501:8501 {TAG}

Note:   We defined the port 8501 as the port used by the Streamlit Interface. Thus, you can access it by typing localhost:8501 in your browser.