Database Migration Tool

A comprehensive Streamlit application for migrating data from Oracle to MySQL databases with an intuitive web interface.

Features

🔄 Database Migration

• Oracle to MySQL data migration
• Batch processing for large datasets
• Real-time progress tracking
• Error handling with resume functionality

🗺️ Smart Mapping

• Automatic table discovery
• Interactive table mapping
• Column-level mapping with data type conversion
• Custom transformation rules (Y/N → True/False, date formats)

⚙️ Configuration Management

• Easy database connection setup
• Migration settings customization
• Error handling strategies
• Performance optimization options

📊 Monitoring & Reporting

• Real-time migration progress
• Detailed error reporting
• Performance statistics
• Migration validation

Prerequisites

• Python 3.8 or higher
• Oracle Database client libraries
• MySQL Database access
• Required Python packages (see requirements.txt)

Installation

1. Clone or download the project files

2. Install Oracle Instant Client

Windows:

1. Download Oracle Instant Client from Oracle website
2. Extract to a folder (e.g., C:\oracle\instantclient_21_6)
3. Add the folder to your PATH environment variable

Linux/macOS:

# Follow Oracle Instant Client installation guide for your OS

3. Install Python dependencies

pip install -r requirements.txt

4. Install additional Oracle dependencies (if needed)

# If you encounter cx_Oracle installation issues:
pip install cx_Oracle --upgrade

# For Windows, you might need:
pip install cx_Oracle --no-binary cx_Oracle

Configuration

Database Setup

Oracle Database (Source):

• Host: localhost (or your Oracle server)
• Port: 1521 (default)
• Service Name: XE (or your service name)
• Username: psp
• Password: psp

MySQL Database (Target):

• Host: localhost (or your MySQL server)
• Port: 3306 (default)
• Database: db_psp
• Username: root
• Password: (empty or your password)

Usage

1. Start the Application

streamlit run app.py

The application will open in your web browser at http://localhost:8501

2. Migration Workflow

Step 1: Database Configuration

• Configure Oracle connection settings
• Configure MySQL connection settings
• Test both connections to ensure they work

Step 2: Table Selection & Mapping

• View all tables from both databases
• Select Oracle tables to migrate
• Map each Oracle table to corresponding MySQL table

Step 3: Column Mapping

• Map columns between selected tables
• Configure data transformations:
  • Y/N → True/False for boolean fields
  • Date format conversions
  • Custom field mappings

Step 4: Migration Settings

• Set batch size (default: 1000 rows)
• Configure error handling strategy
• Set date format for MySQL
• Enable/disable table truncation

Step 5: Execute Migration

• Start migration process
• Monitor real-time progress
• Handle errors with continue/stop options
• Resume paused migrations
• View detailed results

Advanced Features

Error Handling Strategies

1. Stop on Error: Migration stops at first error
2. Skip and Continue: Skip problematic data and continue
3. Ask User: Prompt user for each error encountered

Data Transformations

• Boolean Conversion: Oracle Y/N → MySQL True/False
• Date Format: Configurable date format conversion
• Null Handling: Proper null value management
• Data Type Mapping: Automatic Oracle to MySQL type mapping

Performance Options

• Batch Processing: Configurable batch sizes
• Connection Pooling: Multiple database connections
• Parallel Processing: Process multiple tables simultaneously
• Progress Monitoring: Real-time progress tracking

Troubleshooting

Common Issues

1. Oracle Connection Failed

   • Verify Oracle Instant Client installation
   • Check database connection parameters
   • Ensure Oracle service is running

2. MySQL Connection Failed

   • Verify MySQL server is running
   • Check database credentials
   • Ensure database exists

3. Import Errors

   • Install missing Python packages: pip install -r requirements.txt
   • For Oracle: Install Oracle Instant Client
   • For MySQL: Install MySQL connector

4. Data Type Errors

   • Review column mappings
   • Check data type compatibility
   • Use transformation rules for incompatible types

5. Performance Issues

   • Reduce batch size for large tables
   • Enable connection pooling
   • Check database server resources

Error Messages

• ORA-00942: Table or view does not exist
• Access denied: Check database permissions
• Data too long: Check column length constraints
• Constraint violation: Check foreign key relationships

File Structure

psp-db-migration-app/
├── app.py                 # Main Streamlit application
├── database_manager.py    # Database connection and operations
├── migration_engine.py    # Migration logic and data processing
├── utils.py              # Utility classes and helpers
├── requirements.txt      # Python dependencies
├── README.md            # This file
└── migration.log        # Migration log file (created during execution)

Support

For issues and questions:

1. Check the troubleshooting section above
2. Review the migration logs for detailed error information
3. Ensure all dependencies are properly installed
4. Verify database connections and permissions

License

This project is provided as-is for database migration purposes.