Example Data Migration Scripts

This repository contains code examples of scripts I've written to migrate thousands of data points between platforms, including customers, orders, product reviews, and file assets. These scripts demonstrate practical solutions for handling real-world data migration challenges, such as API rate limits, error handling, and mapping between different data models. It also includes examples in both Python and Node.js to showcase different approaches and libraries.

---

Folder Overview

check-redirect

• Python script to check URLs for 301/200 redirects and log results.
• Reads a list of URLs, sends requests, and writes redirect info to a CSV.
• Used for validating and mapping old URLs to new ones.

checkRedirect--Node

• Node.js version of the redirect checker.
• Reads URLs from a text file, checks for redirects, and writes results to a JSON file.
• Useful for batch processing and error logging in JS environments.

getPdfs

• Node.js script to organize and move PDF files based on a CSV list.
• Scans directories for files, matches them to a list, and moves found files to a "done" folder.
• Writes a list of missing files to a text file for review.

import_customers

• Python script to import customer data into BigCommerce via API.
• Handles customer attributes, addresses, and error logging.
• Breaks large payloads into chunks for API rate limits.

import_orders

• Python script to import order data into BigCommerce.
• Maps Magento order fields to BigCommerce, including products and addresses.
• Handles product options, error logging, and retries for failed posts.

import_product_reviews

• Python script to import product reviews into BigCommerce.
• Checks for existing reviews, fetches product IDs, and posts review data.
• Handles error logging and API rate limits.

importCustomers--Node

• Node.js script to import customer data into BigCommerce.
• Reads customer data, builds payloads, and sends them in batches to the API.
• Includes helper functions for parsing addresses and error logging.

importOrders--Node

• Node.js script to import order data into BigCommerce.
• Maps Magento order fields to BigCommerce, including products, variants, and addresses.
• Handles customer creation, product option mapping, and error logging.

importProductReviews--Node

• Node.js script to import product reviews into BigCommerce.
• Reads review data, fetches product IDs, and posts reviews to the API.
• Includes error logging and checks for existing reviews.

---