Add `jpeg_quality` parameter to `log_image` #2418

emilk · 2023-06-13T17:31:23Z

What

You can now do log_image("image", image, jpeg_quality=75) to compress your logged images so they take up less space on disk and in RAM.

I also updated the UI so you can actually tell whether or not a tensor was compressed.

This makes a many of our example data lot smaller. For instance, tracking_hf_opencv goes from 753 MB to 101MB.

Checklist

I have read and agree to Contributor Guide and the Code of Conduct

PR Build Summary: https://build.rerun.io/pr/2418

Docs preview: https://rerun.io/preview/194b4be/docs
Examples preview: https://rerun.io/preview/194b4be/examples

abey79 · 2023-06-13T18:09:09Z

OpenCV isn't the nicest dependency to have. For example, sometime one must use opencv-python-headless when it interferes with some UI toolkits, etc. Could we consider alternative package, e.g. Pillow?

emilk · 2023-06-13T18:13:03Z

Good idea - pillow we can probably depend on directly too; it is a pretty light dependency. Let's see if it supports jpeg compression…

EDIT: it did, and it works great!

nikolausWest · 2023-06-13T20:22:24Z

Pillow is supposed to be pretty slow. Did you test speed at all? simplejpeg looks like it might be a better fit.

rerun_py/rerun_sdk/rerun/log/image.py

abey79 · 2023-06-13T20:29:41Z

Pillow is supposed to be pretty slow. Did you test speed at all? simplejpeg looks like it might be a better fit.

Unfortunately, they dont back their bold (and indeed tempting) claims with proper binaries for Apple silicon. I tried to install the package, it installed from source, and failed :(

Edit: it looks like next release will fix this: https://gitlab.com/jfolz/simplejpeg/-/issues/7

rerun_py/rerun_sdk/rerun/log/image.py

emilk · 2023-06-14T06:44:35Z

Yes, this costs the user a bit of CPU at log-time. We could alleviate this by throwing the jpeg encoding in a background thread and wait for that thread on shutdown, but I don't want to do such a risky change right now. I've warned the user about the CPU cost in the docstring.

abey79 · 2023-06-14T07:25:27Z

.github/workflows/reusable_build_and_test_wheels.yml needs fixing. I share @jleibs's perplexity around that part btw (line 242):

      - name: Install wheel dependencies
        if: needs.set-config.outputs.RUN_TESTS == 'true'
        # First we install the dependencies manually so we can use `--no-index` when installing the wheel.
        # This needs to be a separate step for some reason or the following step fails
        # TODO(jleibs): pull these deps from pyproject.toml
        # TODO(jleibs): understand why deps can't be installed in the same step as the wheel
        shell: bash
        run: |
          pip install deprecated numpy>=1.23 pyarrow==10.0.1 pytest==7.1.2

      - name: Install built wheel
        if: needs.set-config.outputs.RUN_TESTS == 'true'
        # Now install the wheel using a specific version and --no-index to guarantee we get the version from
        # the pre-dist folder. Note we don't use --force-reinstall here because --no-index means it wouldn't
        # find the dependencies to reinstall them.
        shell: bash
        run: |
          pip uninstall rerun-sdk
          pip install rerun-sdk==${{ steps.expected_version.outputs.EXPECTED_VERSION }} --no-index --find-links dist

Couldn't we just do that?

pip uninstall rerun-idk
pip install local/path/to/dist/rerun_xxx.whl

nikolausWest · 2023-06-14T08:15:03Z

rerun_py/rerun_sdk/rerun/log/image.py

@@ -90,6 +102,23 @@ def log_image(
    if interpretable_as_image and num_non_empty_dims != len(shape):
        image = np.squeeze(image)

+    if jpeg_quality is not None:
+        # TODO(emilk): encode JPEG in background thread instead


it might be nice to do this in rust instead in the future to make it more cross-platform and to introduce less python dependencies

nikolausWest · 2023-06-14T13:38:25Z

examples/python/colmap/main.py

            img = cv2.imread(str(image_file))
            img = cv2.resize(img, resize)
-            jpeg_quality = [int(cv2.IMWRITE_JPEG_QUALITY), 75]
-            _, encimg = cv2.imencode(".jpg", img, jpeg_quality)
-            rr.log_image_file("camera/image", img_bytes=encimg)
+            rr.log_image("camera/image", img, jpeg_quality=75)


This is a BGR vs RGB bug

### What Bug introduced in #2418 It's back to being red: ![image](https://github.com/rerun-io/rerun/assets/1148717/9ce4ab6b-a782-4b0d-9f75-2c905bf3313e) ### Checklist * [x] I have read and agree to [Contributor Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md) * [x] I've included a screenshot or gif (if applicable)  PR Build Summary: https://build.rerun.io/pr/2430  Docs preview: https://rerun.io/preview/df06d4c/docs Examples preview: https://rerun.io/preview/df06d4c/examples

### What Updates the titles, and adds tags to all "python examples with real data" (a separate PR is needed for the "fake data" examples). Renames: | before | after | | --- | --- | | arkitscenes | arkit_scenes | | colmap | structure_from_motion | | deep_sdf | signed_distance_fields | | dicom | dicom_mri | | mp_pose | human_pose_tracking | | nyud | rgbd | | opencv_canny | live_camera_edge_detection | | ros | ros_node | | segment_anything | segment_anything_model | | stable_diffusion | depth_guided_stable_diffusion | | tracking_hf_opencv | detect_and_track_objects | In order to minimise merge issues, this is best updated and merged AFTER: - #2419 - #2420 - #2418 - #2424 - #2426 - #2360 TODO - [x] Update names script names - [x] Update any references to the scripts - [ ] ~~Mirror updates for Rust examples~~ (none of the renamed examples have rust counterparts) - [x] Convert tags to list - [x] Quick typo/grammar pass with pycharm's checker ### Checklist * [x] I have read and agree to [Contributor Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md) * [ ] I've included a screenshot or gif (if applicable)  PR Build Summary: https://build.rerun.io/pr/2416  Docs preview: https://rerun.io/preview/3fb07ee/docs Examples preview: https://rerun.io/preview/3fb07ee/examples  --------- Co-authored-by: Antoine Beyeler <antoine@rerun.io>

Closes #2175 You can now do `log_image("image", image, jpeg_quality=75)` to compress your logged images so they take up less space on disk and in RAM. I also updated the UI so you can actually tell whether or not a tensor was compressed. This makes a many of our example data lot smaller. For instance, `tracking_hf_opencv` goes from 753 MB to 101MB. * [x] I have read and agree to [Contributor Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md)  PR Build Summary: https://build.rerun.io/pr/2418  Docs preview: https://rerun.io/preview/194b4be/docs Examples preview: https://rerun.io/preview/194b4be/examples

### What Bug introduced in #2418 It's back to being red: ![image](https://github.com/rerun-io/rerun/assets/1148717/9ce4ab6b-a782-4b0d-9f75-2c905bf3313e) ### Checklist * [x] I have read and agree to [Contributor Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md) * [x] I've included a screenshot or gif (if applicable)  PR Build Summary: https://build.rerun.io/pr/2430  Docs preview: https://rerun.io/preview/df06d4c/docs Examples preview: https://rerun.io/preview/df06d4c/examples

Updates the titles, and adds tags to all "python examples with real data" (a separate PR is needed for the "fake data" examples). Renames: | before | after | | --- | --- | | arkitscenes | arkit_scenes | | colmap | structure_from_motion | | deep_sdf | signed_distance_fields | | dicom | dicom_mri | | mp_pose | human_pose_tracking | | nyud | rgbd | | opencv_canny | live_camera_edge_detection | | ros | ros_node | | segment_anything | segment_anything_model | | stable_diffusion | depth_guided_stable_diffusion | | tracking_hf_opencv | detect_and_track_objects | In order to minimise merge issues, this is best updated and merged AFTER: - #2419 - #2420 - #2418 - #2424 - #2426 - #2360 TODO - [x] Update names script names - [x] Update any references to the scripts - [ ] ~~Mirror updates for Rust examples~~ (none of the renamed examples have rust counterparts) - [x] Convert tags to list - [x] Quick typo/grammar pass with pycharm's checker * [x] I have read and agree to [Contributor Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md) * [ ] I've included a screenshot or gif (if applicable)  PR Build Summary: https://build.rerun.io/pr/2416  Docs preview: https://rerun.io/preview/3fb07ee/docs Examples preview: https://rerun.io/preview/3fb07ee/examples  --------- Co-authored-by: Antoine Beyeler <antoine@rerun.io>

emilk added 2 commits June 13, 2023 19:23

Show if an image is JPEG-compressed in the UI

ff743c2

Add jpeg_quality parameter to log_image, requiring cv2

ae378ac

emilk added 🐍 Python API Python logging API 📉 performance Optimization, memory use, etc labels Jun 13, 2023

emilk added 5 commits June 13, 2023 19:53

Fix BGR/RGB shenanigans

a95e8e8

Better docstring

82c332a

Less spam when running arkitscenes

fa81775

Turn on jpeg-encoding for a bunch of our examples

7dd7202

py-format

ea3f6cf

Use pillow/PIL instead of cv2

2b1bc1a

emilk changed the title ~~Add jpeg_quality parameter to log_image, requiring cv2~~ Add jpeg_quality parameter to log_image Jun 13, 2023

emilk added 2 commits June 13, 2023 20:23

Fix warning when running arkitscenes

a1534a1

Fix the pillow version

b5d0f1f

emilk mentioned this pull request Jun 13, 2023

Update web examples #2420

Merged

1 task

emilk added 2 commits June 13, 2023 21:13

Try another pillow version

8aaa3b3

Try another pillow version

5ac2759

nikolausWest reviewed Jun 13, 2023

View reviewed changes

rerun_py/rerun_sdk/rerun/log/image.py Outdated Show resolved Hide resolved

nikolausWest reviewed Jun 13, 2023

View reviewed changes

rerun_py/rerun_sdk/rerun/log/image.py Outdated Show resolved Hide resolved

abey79 mentioned this pull request Jun 13, 2023

Update titles and tags for examples with real data #2416

Merged

7 tasks

emilk added 4 commits June 14, 2023 08:36

remove try-catch block

08782a4

Handle any dtype

681e803

fix pillow version snafu

a0c5194

py-format

9e8cc6e

emilk requested a review from nikolausWest June 14, 2023 06:44

emilk added 3 commits June 14, 2023 09:39

Update .github/workflows/reusable_build_and_test_wheels.yml

151fe3b

Merge branch 'main' into emilk/log_image-jpeg-encode

0410513

Merge branch 'main' into emilk/log_image-jpeg-encode

04dbb44

emilk removed the request for review from nikolausWest June 14, 2023 07:57

nikolausWest reviewed Jun 14, 2023

View reviewed changes

nikolausWest approved these changes Jun 14, 2023

View reviewed changes

Maybe yaml doesn't support commas in strings. who the hell knows

194b4be

emilk merged commit bf3725b into main Jun 14, 2023
20 checks passed

emilk deleted the emilk/log_image-jpeg-encode branch June 14, 2023 09:15

nikolausWest reviewed Jun 14, 2023

View reviewed changes

emilk mentioned this pull request Jun 14, 2023

Fix BGR/RGB mixup in colmap example #2430

Merged

2 tasks

This was referenced Jun 15, 2023

Release 0.6.1 #2443

Closed

Prepare release of 0.7.0 #2446

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `jpeg_quality` parameter to `log_image` #2418

Add `jpeg_quality` parameter to `log_image` #2418

emilk commented Jun 13, 2023 •

edited by github-actions bot

abey79 commented Jun 13, 2023

emilk commented Jun 13, 2023 •

edited

nikolausWest commented Jun 13, 2023

abey79 commented Jun 13, 2023 •

edited

emilk commented Jun 14, 2023

abey79 commented Jun 14, 2023

nikolausWest Jun 14, 2023

emilk Jun 14, 2023

nikolausWest Jun 14, 2023

Add jpeg_quality parameter to log_image #2418

Add jpeg_quality parameter to log_image #2418

Conversation

emilk commented Jun 13, 2023 • edited by github-actions bot

What

Checklist

abey79 commented Jun 13, 2023

emilk commented Jun 13, 2023 • edited

nikolausWest commented Jun 13, 2023

abey79 commented Jun 13, 2023 • edited

emilk commented Jun 14, 2023

abey79 commented Jun 14, 2023

nikolausWest Jun 14, 2023

Choose a reason for hiding this comment

emilk Jun 14, 2023

Choose a reason for hiding this comment

nikolausWest Jun 14, 2023

Choose a reason for hiding this comment

Add `jpeg_quality` parameter to `log_image` #2418

Add `jpeg_quality` parameter to `log_image` #2418

emilk commented Jun 13, 2023 •

edited by github-actions bot

emilk commented Jun 13, 2023 •

edited

abey79 commented Jun 13, 2023 •

edited