Skip to content

Processing JSON files to work with labelme

Guang-Wei Zhang edited this page Aug 25, 2023 · 1 revision
import os
import json

# Define the paths to the source, destination, and corrected directories
source_dir = "XXXXXXXXXXXXXXXXXXXXX"
dest_dir = "XXXXXXXXXXXXXXXXXXXXX"
corrected_dir = "XXXXXXXXXXXXXXXXXXXXX"


# Create the corrected directory if it doesn't exist
if not os.path.exists(corrected_dir):
    os.makedirs(corrected_dir)

# Process each JSON file in the source directory
for source_filename in os.listdir(source_dir):
    if source_filename.endswith(".json"):
        with open(os.path.join(source_dir, source_filename), 'r') as f:
            source_data = json.load(f)

        # Extract the required values
        imagePath = source_data["imagePath"]
        imageData = source_data["imageData"]
        imageHeight = source_data["imageHeight"]
        imageWidth = source_data["imageWidth"]

        # Create a version of source_filename without "_bin1" for matching
        matched_filename = source_filename.replace("_bin1", "")

        # Add "_bin" suffix to the matched filename
        corrected_filename = matched_filename.replace(".json", "_bin1.json")

        # Check if the matched filename exists in the destination directory
        if matched_filename in os.listdir(dest_dir):
            with open(os.path.join(dest_dir, matched_filename), 'r') as f:
                dest_data = json.load(f)
            
            # Update the fields
            dest_data["imagePath"] = imagePath
            dest_data["imageData"] = imageData
            dest_data["imageHeight"] = imageHeight
            dest_data["imageWidth"] = imageWidth
                
            # Save the updated JSON file in the corrected directory with "_bin" suffix
            with open(os.path.join(corrected_dir, corrected_filename), 'w') as f:
                json.dump(dest_data, f, indent=4)

print("Update completed!")

Wiki Page: JSON File Processing Script

Overview

This script is designed to process JSON files located in a source directory, extract specific data from them, and update corresponding JSON files in a destination directory with the extracted data. The updated JSON files are then saved in a separate directory with a "_bin" suffix added to the filename.

Prerequisites

  • Python 3.x
  • os and json modules (standard libraries in Python)

Directory Structure

  1. Source Directory: Contains the original JSON files.
  2. Destination Directory: Contains JSON files that need to be updated.
  3. Corrected Directory: The directory where the updated JSON files will be saved with the "_bin" suffix.

Script Workflow

  1. Define the paths for the source, destination, and corrected directories.
  2. If the corrected directory doesn't exist, it's created.
  3. For each JSON file in the source directory:
    • Open and load the JSON data.
    • Extract specific fields: imagePath, imageData, imageHeight, and imageWidth.
    • Create a version of the source filename without the "_bin1" substring for matching.
    • Check if a corresponding file exists in the destination directory.
    • If it does, open and load the destination JSON data.
    • Update the destination data with the extracted fields from the source.
    • Save the updated data in the corrected directory with a "_bin" suffix added to the filename.
  4. Print a completion message.

Usage

  1. Ensure the source and destination directories are correctly populated with the required JSON files.
  2. Run the script.
  3. Check the corrected directory for the updated JSON files with the "_bin" suffix.

Notes

  • Ensure that the directory paths are correctly set before running the script.
  • The script specifically looks for JSON files with a "_bin1" substring in their filenames in the source directory.
  • The script will overwrite files in the corrected directory if they already exist with the same name.

Future Enhancements

  • Implement error handling for potential issues, such as file read/write errors.
  • Allow for dynamic directory path inputs, rather than hardcoding them.
  • Extend the script to handle other file formats or additional processing requirements.

Data processing pipeline

preparation of cell bin data

Data Structure of scanpy

Clone this wiki locally