Visualizer

[khustup]

🚀 🚀 Pull Request

Checklist:

• My code follows the style guidelines of this project and the Contributing document
• I have commented my code, particularly in hard-to-understand areas
• I have kept the coverage-rate up
• I have performed a self-review of my own code and resolved any problems
• I have checked to ensure there aren't any other open Pull Requests for the same change
• I have described and made corresponding changes to the relevant documentation
• New and existing unit tests pass locally with my changes

Changes

Added visualization of the datasets in the Jupyter notebook.

from hub.visualizer import visualize
ds = hub.load(...)
visualize(ds)

It opens the grid of the visualizer with the following controls:

• Ability to observe tensors adjust their opacity, make them hidden
• Observe version control state, view message/author/time of the specific commit, checkout to specific commit
• Jump to specific sample by index
• Change to dark mode

[codecov]

Codecov Report

Merging #1567 (ec5bfa8) into main (3af48e3) will decrease coverage by 0.60%.
The diff coverage is 85.20%.

@@            Coverage Diff             @@
##             main    #1567      +/-   ##
==========================================
- Coverage   92.50%   91.89%   -0.61%     
==========================================
  Files         201      204       +3     
  Lines       18617    18538      -79     
==========================================
- Hits        17221    17035     -186     
- Misses       1396     1503     +107     

Flag	Coverage Δ
unittests	91.89% <85.20%> (-0.61%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
hub/core/dataset/hub_cloud_dataset.py	90.62% <ø> (-0.15%)	⬇️
setup.py	0.00% <ø> (ø)
hub/core/dataset/dataset.py	92.60% <33.33%> (-0.43%)	⬇️
hub/core/storage/s3.py	68.43% <57.14%> (-1.50%)	⬇️
hub/visualizer/visualizer.py	88.29% <88.29%> (ø)
hub/core/storage/lru_cache.py	95.97% <90.90%> (-0.30%)	⬇️
hub/core/storage/gcs.py	77.29% <100.00%> (+0.92%)	⬆️
hub/core/storage/tests/test_storage_provider.py	83.63% <100.00%> (-0.30%)	⬇️
hub/visualizer/__init__.py	100.00% <100.00%> (ø)
hub/visualizer/tests/test_visualizer.py	100.00% <100.00%> (ø)
... and 68 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3af48e3...ec5bfa8. Read the comment docs.

[davidbuniat]

Should we add flask as a dependency (not sure though why tests pass, maybe it is shipped with another dependency)?

[AbhinavTuli]

curious why we went with visualize(ds) as a function instead of a method of the Dataset object.

[khustup]

The reason is to avoid handling additional dependencies at the dataset core level.

[AbhinavTuli]

that can still be done, the dependencies can be imported when the user tries to use the functionality, if they aren't installed, we can throw an exception, urging the user to pip install hub[visualizer]

[AbhinavTuli]

you need to add these to common.txt too

[farizrahman4u]

I dont think we need the IPython dependency.

[AbhinavTuli]

we can perhaps allow users to pass height and width as optional arguments

[tatevikh]

@khustup please make sure that all the tests pass and coverage is up before merging. Thx!