Table of Contents
We are a Canadian team of four people, and we are developing tools to assist in the segmentation and virtual unwrapping of the Herculaneum scrolls. Our contribution to the “Segmentation Tooling 2” competition consists of python code with the following features:
- Integration into a fork of the VolumeAnnotate project (maintained by Moshe Levy here)
- Option to upload a pre-generated vector field for each slice to facilitate papyrus line following
- As part of the normal line-building workflow for each slice, VA now uses intelligent line finding. This follows the vector contours of the line of the papyrus between two points. This is in contrast to the manual generation of several straight-line segments that approximate the natural curvature of the underlying papyrus.
- The feature reduces the number of clicks manual segmenters need to make to generate virtual fragments.
For an in depth explanation of our methodology to accomplish this outcome, check out the slide deck here
The following are some examples of flowlines traced using the features of this tool:
This is useful in situations where the flow of the line is visually well defined, and can be determined by the segmenter at a glance. This capability can allow an experienced segmenter to capture significant line value in just a few clicks.
The algorith handles crowded segments well, and allows for segmentation of crowded areas that are visually readable by a segmenter.
Mushy sections can be challenging to parse visually, but if the operator is confident in their judgement about where a segment starts and ends, the algorithm does a decent job of joining the two points.
Overall this line tracing algorithm works well in a variety of situations commonly faced by segmenters. We understand that more work must be done to increase performance in terms of speed, usability, and automation. This project is ultimately intended to help recover the maximum amount of surface area with the minimum amount of suffering.
Download a local copy of the current version of VA-Sheet-Tracer at https://github.com/teeohem96/VA-Sheet-Tracer
git clone https://github.com/teeohem96/VA-Sheet-Tracer.git
cd VA-Sheet-TracerRequirements can be installed with the following command:
pip install -r requirements.txt- Before running the program, create a folder containing the .tif image slices you want to analyze. These files must follow the same rules as VolumeAnnotate. Images must be single-channel greyscale, and the filenames must be entirely numerical ("0001.tif" and "1.tif" are acceptable, "1_edit.tif" and "test.tif" are not).
- From the -Sheet-Tracer folder, run VolumeAnnotate.py.
python VolumeAnnotate.py- From the App Startup widget, use the “Browse” button to select the folder containing the slice images described in Step 2, and then click the “Launch” button.
- In order to use the intelligent line following tool, a vector field file must exist for the current slice. Vector field files can either be loaded from an existing .npy file, or generated on the fly from the current slice. To generate a vector field file from the current slice, select the "Mark Origin" radio button and click on the image approximately where the centre of the scroll should be (currently the only indication that this step was successful is a console message confirming the new origin). Then, click the "Generate Slice Vector Field" button. Enter a number in the popup dialog box. We have tested values between 3 and 151. For preliminary work, a value of 51 is probably acceptable for most users.
- This number represents the downsampling factor for the image.
- The number should be odd for best performance.
- A lower number will give better line tracing at the cost of longer generation time
- A higher number will generate more quickly at the cost of tracing precision.
Note: Generating vector field files is CPU intensive. The current implementation requires significant time to generate for each slice. As of 2023/08/27, we have benchmarked this code at subsampling 3 (maximum resolution) as taking around 4 hours per slice with the following hardware:
- AMD Ryzen 7 2700x (3.7MHz, x64),
- 32GB RAM
- Windows 10 Pro.
This time cost is impractical for the average user running off the shelf hardware, and we are aware of the limits this imposes on testing our submission. We have uploaded pre-generated vector field files from Scroll 1 for slice indices 06666.tif, 06667.tif, 06668.tif are available for download: (file size 487MB). See the Roadmap section for more information on how we are making vector field generation run faster.
- To load an existing vector field file, click the “Load Slice Vector Field” button, and navigate to the vector field file associated with the current slice. When generated, these files will have the same name as the slice image in the numpy format (e.g. slice 01234.tif will generate vector field file 01234.npy). Select the file and click “Open”. This will load the vector field file into VA-Sheet-Tracer.
- To trace lines on the slice, zoom and pan to an area of interest using the existing VA controls. When you are ready to trace a line, enable the radio button labeled “Outline Fragment”. Then, left click on the image at a point on the scroll where you want to start your line. Aim for an existing line of papyrus. A red dot will appear on the image. Then, click on the part of the line you would like to join to the previous point. VA-Stream will work to join the two points in a way that follows the “flow” of papyrus layers.
Note: Joining points in VA-Sheet-Tracer is CPU intensive. The current implementation is not instantaneous. The other features of VA-Sheet-Tracer will not respond while the line joining process is in progress. As of 2023/08/27, joining any two points on the slice takes around 10-20s. See the Roadmap section for more information on how we are making automated line following faster.
- To continue the line, add more points. VA-Sheet-Tracer features that involve moving points have not been tested for interaction with the line finding tool, and may cause crashes. Use other features of VA-Sheet-Tracer as normal.
Note: The intelligent line finding functionality has known stability issues which may cause VA-Sheet-Tracer to terminate at unpredictable times. We are working to improve the stability of this feature. Please submit issues you encounter via the “Issues” tab on the GitHub page link above, being sure to include any information about the scroll number, slice index, and (x, y) points you deem to be associated with the instability.
To generate vector fields in advance and in bulk, run generate_vector_fields.py, passing the image source folder and vector field file output folder as keywords to generate the required vector field files.
python generate_vector_fields.py C:\path\to\input\slice\images, C:\path\to\output\vector\filesUser Interface Features:
- Intelligent point dragging
- Intelligent point insertion
- Intelligent point deletion
- Automatic Slice-Vector file matching
- Shift+Mouse-move for speculative intelligent line overlay
Backend Features:
- Hardware acceleration
- Replace matplotlib streamplot with more efficient Runge-Kutta 2 integrator
- Automated origin detection (slice spiral “center”)
- Automated point seeding
- Automated generation of vector field data for scalable analysis
- Interlayer context detection to avoid flowline intersection
- Z-axis interpolation using optical flow + displacement mapping
- Improved contextual line following using known segments
- Live confidence metrics
See the open issues for a full list of proposed features (and known issues).
Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.
If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement".
- Fork the Project
- Create your Feature Branch (
git checkout -b feature/AmazingFeature) - Commit your Changes (
git commit -m 'Add some AmazingFeature') - Push to the Branch (
git push origin feature/AmazingFeature) - Open a Pull Request
Distributed under the GPL-v3 License. See LICENSE.txt for more information.
- Tom Wei - tom.wei@mail.utoronto.ca
- Trevor Plint - trevor.plint@mail.utoronto.ca
Project Link: https://github.com/teeohem96/VA-Sheet-Tracer
- This project is built directly on top of code developed by Moshe Levy, without whose efforts this tool would not have been possible.