Skip to content

fix: replace dead streamlines URL with Allen mesoscale connectivity via cloud-volume#438

Open
AdityaGupta716 wants to merge 5 commits intobrainglobe:mainfrom
AdityaGupta716:fix/streamlines-url
Open

fix: replace dead streamlines URL with Allen mesoscale connectivity via cloud-volume#438
AdityaGupta716 wants to merge 5 commits intobrainglobe:mainfrom
AdityaGupta716:fix/streamlines-url

Conversation

@AdityaGupta716
Copy link

What does this PR do?

Fixes the broken streamlines functionality by replacing the defunct
neuroinformatics.nl cache with the official Allen mesoscale connectivity
dataset hosted on Google Cloud Storage.

Why is this needed?

Closes #266

The URL https://neuroinformatics.nl/HBP/allen-connectivity-viewer/json/streamlines_*.json.gz
returns 403 Forbidden. This was a third-party cache that is no longer maintained.

Changes

  • Replaced dead URL with precomputed://gs://allen_neuroglancer_ccf/allen_mesoscale
    accessed via cloud-volume — the official source recommended by AIBS (@fcollman)
  • Rewrote get_streamlines_data() to fetch skeleton data using cloudvolume.CloudVolume
  • Added _skeleton_to_dataframe() to convert the new precomputed skeleton format
    (vertices in nm + edges) back to the {"x", "y", "z"} dict format expected by
    Streamlines._make_mesh()
  • Added DV axis flip (y = 8000 - y) to convert from Allen CCF PIR space to
    brainrender's ASR orientation
  • Added _get_injection_site_um() to fetch real injection coordinates from the Allen
    Brain Atlas API (ProjectionStructureUnionize) instead of using an approximation
  • Added graceful fallback to vertex centroid if Allen API is unreachable
  • Added cloud-volume to pyproject.toml dependencies
  • Preserved the existing local JSON caching behaviour and force_download flag

How was this tested?

  • Verified the GCS bucket is live and accessible
  • Fetched skeleton for experiment 479983421, confirmed 602 streamline components
  • Verified coordinate ranges match Allen CCF atlas mesh ranges for TH after axis correction
  • Confirmed real injection site coordinates match expected values from Allen API
  • Verified output DataFrame format is fully compatible with existing Streamlines actor

Is this a breaking change?

No — the public API (get_streamlines_data, get_streamlines_for_region) is unchanged.
The output DataFrame format is identical to the old format.

Does this require a documentation update?

Yes — users will need cloud-volume installed. This is now listed as a dependency
in pyproject.toml so it will be installed automatically.

@AdityaGupta716
Copy link
Author

Hey @alessandrofelder @adamltyson, please review!
Replaces the defunct neuroinformatics.nl cache with the official Allen mesoscale GCS dataset via cloud-volume (as suggested by @fcollman). Handles coordinate conversion, DV axis flip, and real injection sites from the Allen API. Full visual render testing wasn't possible here due to Python 3.14 C++ dependency issues — would appreciate if someone could run examples/streamlines.py to confirm visual alignment. Closes #266.

@adamltyson
Copy link
Member

Thanks for raising this @AdityaGupta716. This seems like a fairly big change, could you add some tests? Also could you make sure this works locally for you before asking reviewers to check it?

@codecov
Copy link

codecov bot commented Mar 2, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.90%. Comparing base (42c6ef1) to head (5eae276).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #438      +/-   ##
==========================================
+ Coverage   86.60%   87.90%   +1.29%     
==========================================
  Files          27       27              
  Lines        1239     1281      +42     
==========================================
+ Hits         1073     1126      +53     
+ Misses        166      155      -11     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@AdityaGupta716
Copy link
Author

Hi @adamltyson, I've added unit tests with mocked I/O covering the main functions and also verified it works locally — attached a screenshot of the render for experiment , you can see the brain mesh, injection site, and streamlines all coming through correctly. Let me know if there's anything else you'd like me to change!

Screenshot 2026-03-03 at 1 38 36 AM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BUG streamlines URL has changed

2 participants