Skip to content
BranchesTags

Repository files navigation

SmartDoc OCR

A comprehensive OCR (Optical Character Recognition) system for extracting information from Egyptian national ID cards, passports, and highlighted PDF documents. This project uses advanced computer vision techniques, YOLO object detection, and multiple OCR engines to accurately extract and process document information.

Features

1. Egyptian National ID Card Processing

  • Automatic card detection and cropping using YOLO models
  • Intelligent text extraction with EasyOCR and Google Cloud Vision API
  • Arabic and English text support
  • National ID number extraction and decoding
  • Name extraction (with compound name handling)
  • Address extraction with intelligent filtering
  • Fallback mechanisms for robust processing

2. Passport Data Extraction

  • MRZ (Machine Readable Zone) detection using PassportEye
  • Automatic data parsing including:
    • Name and surname
    • Sex
    • Date of birth
    • Passport number
    • Expiration date
    • Issuing country
  • Country code resolution with comprehensive database

3. PDF Highlight Extraction

  • Automatic detection of highlighted text in PDFs (yellow, green, pink)
  • Date extraction from highlighted regions
  • Number/ID extraction from highlighted content
  • Multiple output formats (JSON and Markdown)
  • Multi-page PDF support

Technology Stack

  • FastAPI - Modern web framework for API endpoints
  • YOLO (Ultralytics) - Object detection for card/document localization
  • EasyOCR - Arabic and English text recognition
  • Google Cloud Vision API - Advanced OCR capabilities
  • Tesseract OCR - PDF text extraction
  • PassportEye - MRZ detection and parsing
  • OpenCV - Image processing and computer vision
  • PyTorch - Deep learning backend

Installation

Prerequisites

  • Python 3.8 or higher
  • Tesseract OCR installed on your system
  • Google Cloud Vision API credentials (for ID scanner)

Steps

  1. Clone the repository

    git clone <repository-url>
cd ocr_egyptian_id

  2. Install dependencies

    pip install -r requirements.txt

  3. Set up Google Cloud Vision API

    • Create a Google Cloud project
    • Enable the Vision API
    • Download service account credentials
    • Save credentials as test-ocr.json in the project root

  4. Download YOLO models

    • Ensure the following model files are in the project root:
      • detect_id_card.pt - Card detection model
      • detect_odjects.pt - Object detection model
      • detect_id.pt - ID-specific detection model

Usage

Running the API Server

Start the FastAPI server:

python new_version.py

The server will run on http://0.0.0.0:8006

API Endpoints

1. Health Check

GET /health

2. Scan Egyptian ID Card (YOLO-based)

POST /scan-id
Content-Type: multipart/form-data

file: <image_file>

Response:

{
  "status": "success",
  "data": {
    "firstname": "محمد علي",
    "lastname": "أحمد حسن",
    "national_id": "29501011234567",
    "decoded_info": {
      "birth_date": "01/01/1995",
      "governorate": "Cairo",
      "gender": "Male"
    }
  }
}

3. Scan Egyptian ID Card (Google Vision-based)

POST /id-scanner
Content-Type: multipart/form-data

file: <image_file>

Response:

{
  "status": "success",
  "data": {
    "name": "محمد علي أحمد حسن",
    "address": "شارع النيل، القاهرة",
    "national_id": "٢٩٥٠١٠١١٢٣٤٥٦٧"
  }
}

4. Extract Passport Data

POST /extract-passport
Content-Type: multipart/form-data

file: <passport_image>

Response:

{
  "status": "success",
  "data": {
    "name": "MOHAMED ALI",
    "surname": "AHMED HASSAN",
    "sex": "M",
    "date_of_birth": "01/01/1995",
    "passport_number": "A12345678",
    "expiration_date": "01/01/2030",
    "issuing_country": "EGYPT"
  }
}

5. Extract PDF Highlights

POST /extract-pdf-highlights?output_format=json
Content-Type: multipart/form-data

file: <pdf_file>

Response:

{
  "status": "success",
  "format": "json",
  "highlighted_texts": ["Text 1", "Text 2"],
  "total_highlights": 2,
  "number_id": "Q5 025701 0085",
  "dates": {
    "from_date": "2024-01-01",
    "to_date": "2024-12-31",
    "all_dates": ["2024-01-01", "2024-12-31"]
  }
}

Command-Line Usage

ID Scanner (Google Vision)

python id_scanner.py <image_path>

Passport Scanner

python passport.py <passport_image_path>

PDF Highlight Extractor

python test_pdf_extraction.py

Project Structure

ocr_egyptian_id/
├── scanner.py                    # YOLO-based ID card scanner
├── id_scanner.py                 # Google Vision-based ID scanner
├── passport.py                   # Passport MRZ extraction
├── pdf_highlight_extractor.py    # PDF highlight detection and extraction
├── new_version.py                # FastAPI server with all endpoints
├── test_pdf_extraction.py        # PDF extraction testing script
├── requirements.txt              # Python dependencies
├── country_codes.json            # Country code database
├── test-ocr.json                 # Google Cloud credentials
├── detect_id_card.pt             # YOLO card detection model
├── detect_odjects.pt             # YOLO object detection model
└── detect_id.pt                  # YOLO ID detection model

Key Components

YOLOCardScanner Class

  • Card detection and cropping
  • Perspective transformation
  • Name extraction with intelligent positioning
  • National ID number detection and decoding
  • OCR error correction for Arabic text

ID_Scanner_Model Class

  • Google Cloud Vision integration
  • Arabic text normalization
  • Header and noise filtering
  • National ID extraction with multiple digit formats
  • Address cleaning and validation

PassportModel Class

  • MRZ detection and parsing
  • Date parsing and formatting
  • Country code resolution
  • Data validation and cleaning

PDFHighlightExtractor Class

  • Color-based highlight detection (yellow, green, pink)
  • Multi-page PDF processing
  • Date and ID extraction
  • Markdown and JSON output formats

Configuration

OCR Languages

  • EasyOCR: Arabic (ar) and English (en)
  • Tesseract: English and Arabic (eng+ara)
  • Google Vision: Arabic language hints

YOLO Models

Models can be customized in the API endpoints or when initializing the scanner:

scanner = YOLOCardScanner(
    card_model_path='detect_id_card.pt',
    objects_model_path='detect_odjects.pt',
    id_model_path='detect_id.pt'
)

Error Handling

The system includes comprehensive error handling:

  • Fallback from YOLO to contour-based detection
  • Multiple OCR attempts with different preprocessing
  • Validation of extracted data
  • Detailed error messages in API responses

Logging

Logging is configured for debugging and monitoring:

  • Application logs: app.log
  • Passport processing logs: passport_debug.log
  • Console output for real-time monitoring

Performance Considerations

  • GPU Support: EasyOCR and PyTorch can utilize GPU if available
  • Image Preprocessing: Automatic image enhancement for better OCR accuracy
  • Caching: Temporary files are cleaned up after processing
  • Async Processing: FastAPI supports async operations for better performance

Limitations

  • Requires clear, well-lit images for best results
  • Arabic text recognition accuracy depends on image quality
  • YOLO models need to be trained on representative datasets
  • Google Cloud Vision API requires internet connectivity and valid credentials

Future Enhancements

  • Support for more document types
  • Batch processing capabilities
  • Real-time video stream processing
  • Enhanced error correction for OCR results
  • Multi-language support expansion
  • Docker containerization
  • Database integration for storing results

About

A comprehensive OCR (Optical Character Recognition) system for extracting information from Egyptian national ID cards, passports, and highlighted PDF documents. This project uses advanced computer vision techniques, YOLO object detection, and multiple OCR engines to accurately extract and process document information.

Resources

Readme
Activity

Stars

1 star

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages