Skip to content

Home

Jump to bottom
jaybgager edited this page Oct 7, 2026 · 11 revisions

The tracertools package is a collection of Python functions designed to streamline common tasks for connectomics researchers, particularly those related to the proofreading process.

Terminology

The Glossary page contains explanations of common connectomics, programming, and mathematical terms used in this wiki.

Installation

Installing from Github

  1. Open a terminal and navigate to the directory where you want the tracertools package to be stored.
  2. Run the code
git clone https://github.com/jaybgager/tracertools.git

to make a local copy of the package at the location you navigated to in step one. This will create a folder named tracertools that's linked to the GitHub repository in the location you were in when you ran the clone command. 3. In the terminal, navigate one folder down, so that you're in the top-level folder of the tracertools package by running the command

cd tracertools
  1. In the terminal, run the command
pip install -e .

It may take a minute to run. This will tell Python's default installer, pip, to add the folder you're currently in to its list of importable packages. The -e modifier causes this installation to be "editable", meaning that if you make changes to the code stored in the tracertools folder, they'll take effect when you import the tracertools package into a Python script. This is important for keeping the package up-to-date, so you don't have to reinstall it every time there are changes. This also lets you experiment with modifying the tools to meet your own needs. The . at the end just tells pip to install everything at the current directory location.

Now whenever you need to use a tracertools function, you can simply import it like any other package using the

import tracertools

in any Python script. Then, when you want to use a function, you add tracertools. in front of it. For example, if you wanted to use the get_current_seg_ids function on a couple segment IDs in the main BANC datastack, you would use the following code:

import tracertools
fresh_ids = tracertools.get_current_seg_ids(
    datastack="brain_and_nerve_cord",
    seg_ids=[720575941586154052,720575941429544111]
)

You can also use an alias when importing the package; "tt" is a common choice. In that case, the above code would be modified slightly to look like this:

import tracertools as tt
fresh_ids = tt.get_current_seg_ids(
    datastack="brain_and_nerve_cord",
    seg_ids=[720575941586154052,720575941429544111]
)

Installing From the Python Package Index (PyPI)

Warning

This installation guide is a placeholder and doesn't currently work. The tracertools package will be added to the Python Package Index (PyPI) soon.

To install the tracertools package, open a terminal window and ensure that the Python environment you want to install it in is active.

For example, if you're using the base virtual environment (venv) for anaconda you would use the following command:

conda activate base

Then use pip to install the package from the Python Package Index (PyPI), like so:

pip install tracertools

Functions Requiring Credentials

Several clusters of functions in this package integrate with other services that require you to setup some sort of credentials before use. These are detailed below.

Bucket Functions

The functions used for reading from and writing to remote buckets require an existing S3-style bucket compatible with the cloudfiles and cloudvolume packages. They also require the creation of a local security token to allow the tracertools package to interface with them programmatically. For a guide on how to set one up, check out the Bucket Setup page.

CAVE Functions

The functions used for reading from backend CAVE tables require a an accoutn with the CAVE service and a locally-saved security token before they can be used. For a guide on how to set this up, check out the CAVE Setup page.

Google Sheets Functions

The functions used for reading from and writing to Google sheets that begin with the gsheet prefix require a google OAuth token to be set up before they can be used. For a guide on how to do so, check out the Google Sheets Setup page.

Note

The use of OAuth tokens will likely be replaced by Google Service Accounts in the near future.

Function Summaries

Below are brief descriptions of each function in the tracertools package. For detailed guides on what each function can do and how to use them (with examples), click their links.

  • bucket_convert_colons - utility function that converts filenames that include colons to Windows-safe alternative and back
  • bucket_copy_folder - copies a folder on a cloudfiles managed bucket from one location to another
  • bucket_delete_file - deletes a file on a cloudfiles-managed bucket
  • bucket_delete_folder - deletes a folder and its contents on a cloudfiles-managed bucket
  • bucket_download_file - downloads a file from a cloudfiles-managed bucket
  • bucket_download_folder - downloads a folder from a cloudfiles-managed bucket
  • bucket_move_file - moves a file on a cloudfiles-managed bucket from one location to another
  • bucket_rename_file - renames a file on a cloudfiles-managed bucket
  • bucket_upload_file - uploads a file to a specified location on a cloudfiles-managed bucket
  • bucket_upload_folder - uploads a folder and all its contents to a specified location on a cloudfiles-managed bucket
  • calc_3d_distance - calculates the distance between two point coordinates in 3D
  • calc_avg_point_coords - calculates the average coordinates of a list of 3D points
  • calc_bbox_corners_from_center - calculates the 3D coordinates for the corners of a bounding box based on a centerpoint and size
  • calc_line_triangle_intersect - determines the intersection point, if any, of a line and a triangular plane in 3D space
  • calc_seg_mesh_intersect - calculates the intersection points, if any, between the skeletons of a list of segments and the faces of a neuroglancer mesh
  • calc_skeleton_mesh_intersect - calculates the intersection points, if any, of a skeleton and a mesh
  • check_seg_freshness - checks which segment IDs in a list are outdated
  • check_seg_proofread_status - checks which segment IDs in a list are marked proofread in a datastack's official proofreading table
  • convert_coord_res - converts the coordinates of a point in 3D from one resolution to another
  • count_synapses - gets the number of synapses for each segment ID in a list, can return total or split into incoming/outgoing
  • count_user_sv_contribution - counts the number of supervoxels each user contributed (added and removed) to a finished segment
  • get_anno_array_from_state_file - gets a numpy array of point coordinates from a point annotation layer in a neuroglancer JSON state file
  • get_bones - gets a numpy array of the endpoint coordinate pairs for each edge (bone) in a segment's skeleton
  • get_cable_lengths - gets the cable lengths (sum of all the bone lengths) for a each segment in a list
  • get_cave_stacks - gets a list of names for all the datastacks currently hosted through CAVE
  • get_cave_stack_info - gets the metadata for a specific CAVE datatack
  • get_cave_stack_tables - gets a list of names for all the CAVE-hosted tables associated with a specific datastack
  • get_cave_table - attempts to get an entire CAVE-hosted table as a Pandas DataFrame object but may truncate due to size limitations, mostly used to check header names and formatting
  • get_cave_table_info - gets metadata for a specific CAVE-hosted table
  • get_config - gets the tracer-format config dictionary for a specific datastack
  • get_current_seg_ids - gets the best guess for the current IDs descended from a list of stale segment IDs using those with the highest proportion of the original supervoxels
  • get_json_state_from_url - gets a JSON state dictionary from a shortened neuroglancer share link for a CAVE-hosted datastack
  • get_mesh_triangles - gets an array of all the faces of a segment's mesh as trios of point coordinates of their vertices
  • get_original_seg_ids - gets a list of the IDs of all the original segments that have been incorporated into a current segment ID
  • get_roots_from_points - gets a list of segment or supervoxel IDs at a list of point coordinates
  • get_seg_3d_volume - gets the 3D volume of a segment ID in cubic micrometers, uses cached L2 node data
  • get_seg_changelog - gets the changelog for a segment ID as a Pandas DataFrame object
  • get_seg_details - gets the current seg ID, 3D volume in cubic micrometers, cable length, and synapse count for a list of seg IDs as a list of lists that can easily be fed into gsheet functions
  • get_seg_edits - gets a DataFrame of all the changes made to a single segment ID
  • get_seg_from_sv - gets a segment ID using a supervoxel ID and datastack name
  • get_seg_skeletons - gets a list of Osteoid-format skeleton objects for a list of segment IDs
  • get_svs_from_seg - gets all supervoxel IDs belonging to a single segment ID
  • get_supported_configs - get a list of the names of all datastacks currently supported with a tracertools config dictionary
  • gsheet_add_column - writes a Python list to a column in a Google sheet
  • gsheet_add_row - writes a Python list to a row in a Google sheet
  • gsheet_add_seg_details - for a Google sheet with a column of segment IDs, fills in columns of 3D volume, cable length, synapse count, and proofreading status
  • gsheet_get_col_as_list - makes a Python list out of a Google Sheet column
  • gsheet_get_tab_as_df - makes a Pandas DataFrame out of a Google sheet tab
  • host_ng_volume_locally - host a locally-stored neuroglancer volume, useful for testing mesh-related projects without bucket access
  • make_anno_layer - convert a list of annotation coordinates into a neuroglancer layer JSON dictionary for use in building NG states, works with any combo of point, line, bbox, sphere/ellipsoid, and polyline
  • make_bucket_volume_from_obj - creates a single-res legacy mesh volume on a bucket out of a local OBJ file
  • make_color_list - generates a list of color hexadecimal values spaced evenly around a hue wheel for maximum difference with an option for alternating light and dark at high numbers, useful for labeling
  • make_edits_link - creates neuroglancer share url with all edits for a segment ID as line annotations
  • make_local_volume_from_obj - makes a single-res legacy mesh neuroglancer volume locally from an OBJ file
  • make_mesh_from_points - generates a bucket-hosted single-res legacy mesh neuroglancer volume from a share url with at least one point annotation layer and creates a share url with the mesh as output
  • make_ng_link - utility tool that generates a neuroglancer share link by feeding in datastack name, optionally allows passing segment IDs, custom colors, annotation layer dictionaries, custom meshes, and toggling region meshes if they exist
  • make_objs_from_state_file - creates local mesh OBJ files from a local neuroglancer JSON state file with one or more point annotation layers
  • make_point_cloud_from_state_file - creates local point cloud OBJ file from a local neuroglancer JSON state file with a point annotation layer
  • make_volume_mesh_from_state_file - creates a local single-res legacy mesh neuroglancer volume from a local JSON state file with a point annotation layer
  • make_volume_packaging - utility function to create the necessary local file structure for a neuroglancer volume containing single-res legacy meshes
  • triage_segs - check to see if a list of segment IDs passes through known rough areas of a dataset
  • visualize_skeletons - creates a microviewer window to view the skeletons of a list of segment IDs, color coded for thickness

License

The tracertools package is licensed under the GNU General Public License v3.0. See the LICENSE file for more details.

Clone this wiki locally