Meaningful Search is a powerful tool designed to find the most similar paragraphs in a large collection of PDF files based on the input text. It uses BERT (Bidirectional Encoder Representations from Transformers), a transformer-based machine learning model to create semantic representations of the input and the text from the PDF files. These representations are then compared using cosine similarity to find the most similar paragraphs.
To use the Meaningful Search, you will need to have the following installed:
- Python 3.7 or newer
- PyPDF2
- PyQt5
- transformers
- torch
- numpy
- sqlite3
- sklearn
You can install these packages using pip:
pip install PyPDF2 PyQt5 transformers torch numpy sqlite3 sklearn
- Clone the repository
git clone https://github.com/your_username_/Meaningful_Search.git
- Navigate to the repository
cd Meaningful_Search
- Use
cacher.pyto cache the semantic representations of the text in your PDF files into a SQLite database. You can do this by calling thecache_pdf_encodingsfunction with the path to the directory containing your PDF files as an argument:
from cacher import cache_pdf_encodings
cache_pdf_encodings('path/to/your/pdf/files/')This will create a SQLite database named pdf_encodings.db containing the semantic representations of the text in your PDF files.
- Run
search-GUI.pyorsearch-CLI.pyto start the Meaningful Search application:
python search-GUI.py- Enter the text you want to search in the input field.
- Specify the number of most similar paragraphs you want to display using the spinner.
- Click the 'Search' button to start the search. The application will display the most similar paragraphs along with their similarity scores, the names of the books they came from, and their page numbers.
- Enter the text you want to search.
- Enter the number of most similar paragraohs you want to display.
- Click the 'Enter' button to start the search. The application will display the most similar paragraphs along with their similarity scores, the names of the books they came from, and their page numbers.
Contributions are what make the open-source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.
- Fork the Project
- Create your Feature Branch (
git checkout -b feature/AmazingFeature) - Commit your Changes (
git commit -m 'Add some AmazingFeature') - Push to the Branch (
git push origin feature/AmazingFeature) - Open a Pull Request
Distributed under the GNU GPL 3.0 License. See LICENSE for more information.
Ali Hakim Taskiran - alihakimxyz@gmail.com
Project Link: https://github.com/alihakimtaskiran/SemanticSearch