Corpus of Anglo-Saxon Stone Sculpture (CASSS)

Project Member	Contact address	Role	Unit
Zeynep AKI	zeynep.aki@dur.ac.uk	Research Software Engineer (RSE)	Advanced Research Computing
Prof. Sarah SEMPLE	s.j.semple@durham.ac.uk	Professor in the Department of Archaeology	Department of Archaeology
Dr. Tudor SKINNER	alexis.t.skinner@durham.ac.uk	Lecturer in the Department of Archaeology	Department of Archaeology

Prerequisites

• Python installed on your machine. Download and install Python from python.org.

Requirements and VENV

• You can create a Virtual Environment for your code using:

  python -m venv venv
  

• After you create your VENV, you can activate it by using:

  source venv/Scripts/activate #On Windows
  

  source venv/bin/activate #On MacOS & Linux
  

• Once you finish setting up your VENV, you can install the libraries used into your VENV using python's pip.

  pip install -r requirements.txt
  

• If you already have a 'conda' environment you can run this for a quick set up:

  conda create --name CASSS
  conda activate CASSS
  conda install pip
  pip install -r requirements.txt
  

Condensing your dataset

This script processes Excel files, condenses the data, and generates merged data in CSV format.

Usage

1. Open a terminal or command prompt.

2. Navigate to the directory containing the condenser.py script.

   cd path/to/pre-preparation

3. Run the script with optional command-line arguments:

   python condenser.py --excel_path path/to/excel/files/*.xlsx --output_path path/to/output/merged_data.csv --with_images

   • --excel_path: Path to Excel files. Default: dataset/unformatted/spreadsheets/*.xlsx
   • --output_path: Path to the output CSV file. Default: dataset/formatted/merged_data.csv
   • --with_images: Include images in the output (optional).

--with_images Flag

The --with_images flag is an optional command-line argument in the condenser.py script. This flag controls whether the script includes images in the generated output or not. Below are the details of how the script behaves based on the presence or absence of this flag:

When --with_images is Used

• The script includes rows in the output CSV file where the 'image-id' column contains the substring 'images/wee/'.
• A new column named 'cleaned_id' is added, containing the cleaned IDs extracted from the 'image-id' column.

When --with_images is Not Used

• The script includes rows in the output CSV file where the 'image-id' column contains the substring ' images/no-image2.jpg'.

• No 'cleaned_id' column is added in this case, as it is specific to rows with images.

  If you are using Python 3, replace python with python3 in the command.

4. Press Enter to execute the command.

5. Check the specified output path for the generated CSV file.

Examples

• Run the script with default settings:

  python condenser.py

• Run the script with custom paths and include images:

  python condenser.py --excel_path custom/path/to/excel/files/*.xlsx --output_path custom/path/to/output/merged_data.csv --with_images

Preparing your data for training

This script processes Excel files, cleans and standardizes tags, and formats the data into a new CSV file.

Usage

1. Open a terminal or command prompt.

2. Navigate to the directory containing the reformatter.py script, which by default is the same as your condenser.py script. You don't need to re-establish your path if you're already there.

   cd path/to/reformatter

3. Run the script with optional command-line arguments:

   python reformatter.py --excel_path path/to/excel/files/*.xlsx --output_path path/to/output/reformatted_data.csv --training_data

   • --excel_path: Path to Excel files. Default: dataset/unformatted/spreadsheets/*.xlsx
   • --output_path: Path to the output CSV file. Default: dataset/formatted/reformatted_data.csv
   • --training_data: Generate training data (optional).

4. Press Enter to execute the command.

5. Check the specified output path for the reformatted CSV file.

Examples

• Run the script with default settings:

  python reformatter.py

• Run the script with custom paths and generate training data:

  python reformatter.py --excel_path custom/path/to/excel/files/*.xlsx --output_path custom/path/to/output/reformatted_data.csv --training_data

Converting Images

This script converts images from a specified input folder to PNG format and saves them in a designated output folder.

Usage

1. Open a terminal or command prompt.

2. Navigate to the directory containing the image_converter.py script. again, if you've run condenser.py and/or reformatter.py you should already be in the right directory by default.

   cd path/to/image_converter

3. Run the script with optional command-line arguments:

   python image_converter.py --input_folder path/to/input/images --output_folder path/to/output/images

   • --input_folder: Path to the input folder containing images. Default: dataset/unformatted/images
   • --output_folder: Path to the output folder where converted images will be saved. Default: dataset/formatted/images

4. Press Enter to execute the command.

5. Check the specified output path for the converted images.

Examples

• Run the script with default settings:

  python image_converter.py

• Run the script with custom paths:

  python image_converter.py --input_folder custom/path/to/input/images --output_folder custom/path/to/output/images

What to do if/when the code crashes

• To avoid crashes, ensure that the input folder contains images in a supported format (e.g., JPEG, PNG) which in this case was mostly TIF.
• The code may crash if an image is corrupted or is an unsupported file. If this happens, check your terminal to see which image the code crashed on, take note of the image, delete it and temporarily remove the directories that the code has already successfully finished running from the dataset/unformatted/images directory by default or the directory you have your Images_Vol_N.
• For example, If the code has successfully finished converting images in Images_Vol_1, Images_Vol_2 and Images_Vol_3 and crashed while trying to convert 4_010.tif in Images_Vol_4, take note of 4_010.tif, then temporarily remove folders Images_Vol_1, Images_Vol_2 and Images_Vol_3 from the directory you're running the code in, and re-run the code.

Notes

• If no command-line arguments are provided, the script will use default paths.
• Ensure that all your files are in specified format before you run the code.

Going back

• Once you're done preparing your data, you can go back to the main project directory using:

  cd ..