Skip to content
BranchesTags

Latest commit

 

History

235 Commits

Folders and files

NameName
Last commit message
Last commit date
R
R
 
 
data
data
 
 
inst
inst
 
 
python
python
 
 
.gitignore
.gitignore
 
 
.zenodo.json
.zenodo.json
 
 
LICENSE
LICENSE
 
 
README.md
README.md
 
 

Repository files navigation

Fly Connectome Data Tutorial

Tutorial materials for working with Drosophila connectome datasets at the San Juan Winter School on Connectomics and Brain Simulation (SJCABS). We will work with all the major, dense connectome datasets for the fruit fly.

Instructors: Sven Dorkenwald & Alexander Bates

This tutorial provides foundational skills for loading, analysing, and visualising connectome data that will be used throughout the workshop. You'll learn to work with neuronal morphologies, synaptic connectivity, and network analysis across multiple fly brain and nerve cord datasets. Key contributors to the tools used in, and to prepare, this workshop include Philipp Schlegel and Greg Jefferis.

This tutorial

This tutorial offers:

  1. curated data for connectomic analyses.

  2. concise Python and R code for simple but effective analyses of connectome data.

  3. guides you through some general principles of exploratory connectomics analysis that should be useful.

Quick Links

Fly connectome datasets

Connectome Datasets

We focus primarily on two FlyWire datasets—BANC and FAFB—while also providing access to MANC, Hemibrain, and Male CNS. All datasets have been harmonized to use the unified metadata schema we used in the BANC project, enabling cross-dataset comparisons. Possible metadata entries given here: data/meta_data_entries.

Importantly, You can see renderings of neuronal meshes from the BANC, FAFB, MANC and HemiBrain datasets in neuroglancer here.

BANC connectome

BANC (Brain and Nerve Cord)

Primary dataset for this tutorial

The first synapse-resolution connectome that spanning the brain and ventral nerve cord. Contains ~114,000 neurons with ~108 million synaptic connections. A female fly. Missing the first optic relay, the lamina, and the retina.

Male CNS connectome

Male CNS (Central Nervous System)

Complete male CNS connectome with 166,691 neurons spanning the brain and ventral nerve cord. Includes fruitless and doublesex expression data for studying sex-specific circuits. A male fly. Missing retina.

FAFB connectome

FAFB (Full Adult Fly Brain)

Complete adult female fly brain connectome via the FlyWire project. Contains ~139,000 neurons spanning all brain regions, including detailed annotations of 8,453 cell types. A female fly. Missing ventral nerve cord and retina.

MANC connectome

MANC (Male Adult Nerve Cord)

First complete nerve cord connectome with ~23,000 neurons. A male fly. Missing brain.

Hemibrain connectome

Hemibrain

Dense reconstruction of approximately half the central brain (~25,000 neurons). Includes mushroom body learning circuits and central complex navigation circuits. A female fly. Missing ventral nerve cord and approximately half of the brain and optic lobes.

Guides

These guides were created by the FlyWire project:

Analysis Tools

navis - neuron analysis and visualization

Python

Installation guide: Setting up Python for connectomics

  • navis - Neuron analysis and visualisation (works with all datasets)
  • skeletor - Mesh skeletonisation
  • fafbseg-py - FlyWire/FAFB-specific tools
  • navis-flybrains - Coordinate transforms and template brains
  • ConnectomeInfluenceCalculator - Quantify influence between sensory and effector neurons
  • cocoa - Compare inter/intra-dataset connectivity -CAVEclient - Live connectome dataset annotation and tracking (for flies: FAFB, BANC, FANC) -neuprint-python - Data access to Janelian connectome projects, e.g. HemiBrain, MANC and maleCNS.

natverse - NeuroAnatomy Toolbox for R

R

Installation guide: Installing the natverse

Core packages:

  • natverse - NeuroAnatomy Toolbox ecosystem (works with all datasets)
  • neuprintr - neuPrint client for querying connectome databases
  • nat.flybrains - Coordinate transforms and template brains

Dataset-specific packages:

  • bancr - BANC-specific client
  • fafbseg - FlyWire/FAFB-specific tools
  • hemibrainr - Hemibrain-specific tools
  • malevnc - Male VNC (MANC) specific tools
  • malecns - Male CNS specific tools

Analysis tools:

Data Organisation

All processed data is hosted on Google Cloud Storage: Access Data

To download and work with this data locally, you will need gsutil, in terminal you can install and configure with:

# 1) Install Google Cloud CLI (includes gsutil) – macOS / Linux
#    For Windows, use the official Google Cloud CLI installer instead.
curl -O https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-cli-$(uname -s | tr '[:upper:]' '[:lower:]')-x86_64.tar.gz
tar -xf google-cloud-cli-*.tar.gz
./google-cloud-sdk/install.sh

# 2) Restart your terminal, then verify install
gcloud --version       # should print a version
gsutil version         # should also print a version

# 3) Log in with your Google account and set up config
gcloud init            # follows browser flow; pick or create a project

# (Optional but often helpful: refresh auth explicitly)
gcloud auth login
gcloud auth application-default login

# 4) Test access to the bucket
gsutil ls gs://lee-lab_brain-and-nerve-cord-fly-connectome/compiled_data/

# Outcomes:
#  - If you see object names: access OK.
#  - If you get 403 AccessDenied: you need permissions on the bucket/project.
#  - If you get NotFound: bucket name or visibility is wrong.

You can browse and download files directly from the browser, or use command-line tools:

# List available datasets
gsutil ls gs://lee-lab_brain-and-nerve-cord-fly-connectome/compiled_data/

# Download a specific file
gsutil cp gs://lee-lab_brain-and-nerve-cord-fly-connectome/compiled_data/banc_888/banc_888_meta.feather .

# Download an entire dataset folder
gsutil -m cp -r gs://lee-lab_brain-and-nerve-cord-fly-connectome/compiled_data/banc_888 .

Example Structure (BANC)

compiled_data/banc_888/
├── banc_888_meta.feather                  # Neuron annotations
├── banc_888_edgelist_simple_v3.feather    # Neuron-to-neuron connectivity (latest)
├── banc_888_edgelist_simple_v2.feather    # Older neuron-to-neuron connectivity
├── banc_888_edgelist_split.feather        # Compartment-to-compartment connectivity
├── banc_888_synapses_v2_enriched.parquet  # Individual synapse data with neuropil/region/NT
└── banc_888_metrics.feather               # Cable length, volume, synapse counts per neuron

BANC neuron skeletons and region meshes are stored at the bucket root rather than inside compiled_data/:

gs://lee-lab_brain-and-nerve-cord-fly-connectome/
├── neuron_skeletons/swcs-from-pcg-skel/   # Per-neuron SWC files in BANC space
├── neuron_skeletons.zip                    # Zipped copy of the above
├── neuron_meshes/                          # BANC neuron meshes (Neuroglancer precomputed)
└── region_outlines/                        # BANC region meshes (Neuroglancer precomputed)

The other datasets (FAFB, MANC, Hemibrain, maleCNS) keep their SWC and OBJ assets inside their compiled_data/{dataset}_{version}/ folder. See the per-dataset documentation files for the exact contents.

File Types

  • *_meta.feather - Metadata for each neuron: cell type, brain region, neurotransmitter, developmental lineage (schema details)
  • *_simple_edgelist.feather (or *_edgelist_simple_v3.feather for BANC) - Connectivity matrix showing which neurons connect to which, with connection strengths
  • *_split_edgelist.feather (or *_edgelist_split.feather for BANC) - Compartment-to-compartment connectivity (axon → dendrite, etc.)
  • *_synapses.{feather,parquet} - Coordinates and properties of individual synapses
  • *_swc/ directories - 3D skeleton reconstructions in SWC format (one file per neuron)

See individual dataset documentation files in data/dataset_documentation/ for detailed column descriptions.

Subsetting by Neural System / Region

The previous release shipped pre-computed subset folders (mushroom_body/, antennal_lobe/, central_complex/, optic/, suboesophageal_zone/, front_leg/, abdominal_neuromere/) inside each dataset directory. These subset folders no longer exist — the tutorials now build the equivalent subsets in code, mirroring the original logic from bancpipeline/banc/share/banc-sjcabs.R.

The subset_by_region() helper in R/setup/functions.R and python/utils.py applies the same filters that produced the old folders:

  • Antennal lobe / central complex / mushroom body → regex match against the metadata hierarchy (super_class, cell_class, cell_sub_class, cell_type).
  • Optic lobe / suboesophageal zone / front leg / abdominal neuromere → filter the synapse table by neuropil regex with a ≥100 synapse threshold per neuron.

The biology each subset targets is unchanged:

  • Antennal Lobe: Primary olfactory processing centre receiving input from olfactory receptor neurons and projecting to higher brain regions via projection neurons. Critical for odour discrimination and learning.

  • Central Complex: Navigation circuits for spatial orientation, motor control, and goal-directed behaviour. Contains ring neurons encoding heading direction and columnar neurons for path integration.

  • Mushroom Body: Associative learning and memory circuits. Kenyon cells integrate sensory information and form associations with dopaminergic reinforcement signals.

  • Optic Lobe: Visual processing through lamina (motion detection), medulla (colour and contrast), and lobula (object recognition). Includes both retinotopic local circuits and wide-field integration neurons.

  • Suboesophageal Zone: Lower brain region controlling feeding, grooming, and processing gustatory/tactile information from mouthparts and antennae.

  • Front Leg / Abdominal Neuromere: Motor control circuits coordinating limb movements and postural adjustments through local sensory feedback and descending command signals.

Detailed Data Inventory

BANC (Brain and Nerve Cord)

Browse Files | gs://lee-lab_brain-and-nerve-cord-fly-connectome/compiled_data/banc_888/

  • banc_888_meta.feather (~48 MB) - Metadata (188,153 neurons)
  • banc_888_edgelist_simple_v3.feather (~336 MB) - Neuron-to-neuron connectivity (latest)
  • banc_888_edgelist_simple_v2.feather (~285 MB) - Older neuron-to-neuron connectivity
  • banc_888_edgelist_split.feather (~321 MB) - Compartment-to-compartment connectivity
  • banc_888_synapses_v2_enriched.parquet (~9.6 GB) - Individual synapses (neuropil/region/NT enriched)
  • banc_888_metrics.feather (~7.5 MB) - Per-neuron cable length, volume, synapse counts

Skeletons and region meshes for BANC are at the bucket root:

  • gs://lee-lab_brain-and-nerve-cord-fly-connectome/neuron_skeletons/swcs-from-pcg-skel/ - Per-neuron SWC files (BANC space)
  • gs://lee-lab_brain-and-nerve-cord-fly-connectome/neuron_skeletons.zip - Zipped SWC bundle (~206 MB)
  • gs://lee-lab_brain-and-nerve-cord-fly-connectome/neuron_meshes/ - BANC neuron meshes (Neuroglancer precomputed)
  • gs://lee-lab_brain-and-nerve-cord-fly-connectome/region_outlines/ - BANC region meshes (Neuroglancer precomputed)

FAFB (Full Adult Fly Brain)

Browse Files | gs://lee-lab_brain-and-nerve-cord-fly-connectome/compiled_data/fafb_783/

  • fafb_783_meta.feather (~10 MB) - Metadata
  • fafb_783_simple_edgelist.feather (~289 MB) - Neuron-to-neuron connectivity
  • fafb_783_split_edgelist.feather (~523 MB) - Compartment connectivity
  • fafb_783_synapses.feather (~4.0 GB) / fafb_783_synapses.parquet (~1.7 GB) - Individual synapses
  • fafb_783_cell_dcv_detection.feather (~9.7 GB) - Cellular dense-core-vesicle detections
  • fafb_783_soma_dcv_detection.feather (~3.7 GB) - Somatic DCV detections
  • fafb_dcv_scores_metadata_ya_3_5_26.csv (~15 MB) - DCV detection metadata
  • fafb_fafb_space_swc/ - Skeletons in native FAFB space
  • fafb_banc_space_swc/ - Skeletons in BANC space
  • obj/ - FAFB volume and per-neuropil OBJ meshes

MANC (Male Adult Nerve Cord)

Browse Files | gs://lee-lab_brain-and-nerve-cord-fly-connectome/compiled_data/manc_121/

  • manc_121_meta.feather (~1.4 MB) - Metadata
  • manc_121_simple_edgelist.feather (~83 MB) - Neuron-to-neuron connectivity
  • manc_121_split_edgelist.feather (~321 MB) - Compartment connectivity
  • manc_121_synapses.feather (~3.6 GB) / manc_121_synapses.parquet (~2.4 GB) - Individual synapses
  • manc_manc_space_swc/ - Skeletons in native MANC space
  • manc_banc_space_split_swc/ - Skeletons in BANC space (split by compartment)
  • obj/ - MANC volume and neuropil OBJ meshes

Hemibrain

Browse Files | gs://lee-lab_brain-and-nerve-cord-fly-connectome/compiled_data/hemibrain_121/

  • hemibrain_121_meta.feather (~1.9 MB) - Metadata
  • hemibrain_121_simple_edgelist.feather (~88 MB) - Neuron-to-neuron connectivity
  • hemibrain_121_split_edgelist.feather (~145 MB) - Compartment connectivity
  • hemibrain_121_synapses.feather (~13 KB summary) / hemibrain_121_synapses.parquet (~862 MB)
  • hemibrain_hemibrain_raw_space_swc/ - Skeletons in native Hemibrain space
  • hemibrain_banc_space_swc/ - Skeletons in BANC space
  • obj/, neuropils/ - Hemibrain volume and per-glomerulus/neuropil OBJ meshes

Male CNS

Browse Files | gs://lee-lab_brain-and-nerve-cord-fly-connectome/compiled_data/malecns_09/

  • malecns_09_meta.feather (~9.7 MB) - Metadata
  • malecns_09_simple_edgelist.feather (~3.2 GB) - Neuron-to-neuron connectivity
  • malecns_09_split_edgelist.feather (~4.6 GB) - Compartment connectivity
  • malecns_09_synapses.parquet (~7.9 GB) - Individual synapses
  • malecns_malecns_space_swc/ - Skeletons in native maleCNS space
  • malecns_banc_space_swc/ - Skeletons in BANC space
  • obj/ - maleCNS volume and neuropil OBJ meshes
  • JRC2018U/ - JRC2018-Unisex registration assets

Download Examples

# Small file - metadata (recommended to start)
gsutil cp gs://lee-lab_brain-and-nerve-cord-fly-connectome/compiled_data/banc_888/banc_888_meta.feather .

# Whole dataset folder (big — check sizes first!)
gsutil -m cp -r gs://lee-lab_brain-and-nerve-cord-fly-connectome/compiled_data/banc_888 .

# Large file - check size first
gsutil ls -lh gs://lee-lab_brain-and-nerve-cord-fly-connectome/compiled_data/banc_888/banc_888_synapses_v2_enriched.parquet

# BANC skeletons live at the bucket root (not under compiled_data/)
gsutil -m cp -r gs://lee-lab_brain-and-nerve-cord-fly-connectome/neuron_skeletons/swcs-from-pcg-skel ./banc_swc

Tutorial Path

This tutorial follows a progressive learning path designed to take about 2 hours for the core content, with optional extensions for deeper exploration.

Tutorial 01: Data Access (30 minutes)

R version | Python version

Core Tutorial:

  • Understanding file formats (Feather vs Parquet)
  • Loading metadata and exploring hierarchical classifications
  • Working with Google Cloud Storage and local files
  • Filtering and characterizing neurons by connectivity patterns
  • Example: Mushroom body calyx neurons

Extensions:

  • Your Turn: Apply analysis to different datasets (maleCNS, FAFB)
  • Compare biological vs technical differences between datasets

Tutorial 02: Neuron Morphology (30 minutes)

R version | Python version

Core Tutorial:

  • Loading and visualizing 3D neuron skeletons (.swc files)
  • Reading neuropil meshes for spatial context
  • Co-plotting neurons across datasets
  • NBLAST morphological similarity analysis
  • Hierarchical clustering of neuron morphologies

Extensions:

  • Your Turn: Analyze different neuron populations
  • Extension 1: Template brain transformations (MANC → JRCVNC2018F → BANC)
  • Extension 2: Axon-dendrite splits using flow centrality
    • Compartment labels from graph-theoretic algorithm (Schneider-Mizell et al. 2016)
    • Synapse classification by compartment
    • Available for FAFB, MANC, maleCNS (not yet BANC)

Tutorial 03: Connectivity Analyses (40 minutes)

R version | Python version

Core Tutorial:

  • Loading and querying edgelists (connectivity matrices)
  • Neurotransmitter prediction and signed connectivity
  • Basic network statistics (degree distributions, weight correlations)
  • Connectivity matrices and heatmaps
  • Sensory outputs and effector inputs analysis

Extensions:

  • Your Turn: Analyze different brain regions
  • Connectivity-based clustering (cosine similarity, UMAP)
  • Cluster composition and network visualization
  • Morphological analysis of connectivity clusters

Tutorial 04: Indirect Connectivity and Influence (20 minutes)

R version | Python version

Core Tutorial:

  • Understanding influence scores and random walks through connectomes
  • Calculating sensory → dopaminergic neuron influence
  • Influence heatmaps and UMAP visualization
  • Interpreting multi-hop connectivity patterns

Extensions:

  • Your Turn: Different source/target neuron populations
  • Extension 1: Olfactory channel influence on pC1 neurons (BANC vs maleCNS)
  • Extension 2: Abdominal neuromere sensory-effector influence patterns

Getting Started

If running locally:

  1. Download data from the Google Cloud Storage bucket for the dataset(s) you want to work with
  2. Install analysis tools:
    • Python: pip install navis fafbseg
    • R: 
      install.packages("natmanager")
natmanager::install(pkgs = "core")

# Install Python dependencies for fafbseg
library(fafbseg)
simple_python()

Note: For each python tutorial, there is a paired startup .sh file to install its dependencies in python/runtimes

  1. Open the first tutorial in tutorials/python/ or tutorials/R/

Citation

The Google Bucket contains a curation of connectome data by Alexander Bates. The purpose of the curation was to make it easy to work with all major connectome datasets together, e.g. standardising column names and meta data entires. You are welcome to use this data curation in your own work! Just let Alex know!

More generally, if you use these datasets in your work, please cite the original publications:

BANC: Bates, A.S., Phelps, J.S., Kim, M., Yang, H.H., Matsliah, A., Ajabi, Z., Perlman, E., et al. (2025). Distributed control circuits across a brain-and-cord connectome. bioRxiv, 2025.07.31.667571. https://doi.org/10.1101/2025.07.31.667571

FAFB: Schlegel, P., Yin, Y., Bates, A.S., Dorkenwald, S., Eichler, K., Brooks, P., Han, D.S., et al. (2024). Whole-brain annotation and multi-connectome cell typing of Drosophila. Nature, 634(8032), 139-152. https://doi.org/10.1038/s41586-024-07686-5

FAFB: Dorkenwald, S., Matsliah, A., Sterling, A.R., Schlegel, P., ... Bates, A.S., ... et al. (2024). Neuronal wiring diagram of an adult brain. Nature, 634(8032), 124-138. https://doi.org/10.1038/s41586-024-07686-5

MANC: Takemura, S., et al. (2024). A Connectome of the Male Drosophila Ventral Nerve Cord. eLife. https://doi.org/10.7554/eLife.97769

Hemibrain: Scheffer, L.K., Xu, C., Januszewski, M., Lu, Z., Takemura, S.Y., Bates, A.S., et al. (2020). A connectome and analysis of the adult Drosophila central brain. eLife, 9, e57443. https://doi.org/10.7554/eLife.57443

Male CNS: Berg, S., Beckett, I.R., Costa, M., Schlegel, P., Januszewski, M., Marin, E.C., Bates, A.S., et al. (2025). Sexual dimorphism in the complete connectome of the Drosophila male central nervous system. bioRxiv, 2025.10.09.680999. https://doi.org/10.1101/2025.10.09.680999

Neurotransmitter Predictions: Eckstein, N., Bates, A.S., Champion, A., Du, M., Yin, Y., Schlegel, P., Lu, A.K.Y., et al. (2024). Neurotransmitter classification from electron microscopy images at synaptic sites in Drosophila melanogaster. Cell, 187(10), 2574-2594.e23. https://doi.org/10.1016/j.cell.2024.03.016

License & Contact

Data: Licensed under CC-BY by the respective dataset creators Code: MIT License Questions: Open an issue on this repository or contact the instructors during the workshop

Workshop: SJCABS Winter School Year: 2025

FlyWire visualization

Open access (HHMI)

This software was developed with support from the Howard Hughes Medical Institute (HHMI). Per HHMI's open-access policy, the associated manuscript ("Distributed control circuits across a brain-and-cord connectome") and its Harvard Dataverse data deposit are released under the Creative Commons Attribution 4.0 International License (CC BY 4.0). This source code remains under its existing OSI-approved open-source license — see LICENSE.

About

Tutorial on how to access the major fly connectome datasets

Resources

Readme
MIT license
Activity
Custom properties

Stars

17 stars

Watchers

2 watching

Forks

9 forks
Report repository

Releases

Packages

Contributors

Languages