Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add modified version of draw rects for non ascii characters #2648

Merged
merged 12 commits into from Jan 5, 2022

Conversation

iory
Copy link
Member

@iory iory commented Dec 22, 2021

What is this?

image

The current draw_rects node cannot draw non-accii chracters such as Japanese.
This PR adds a new node to draw text using the PIL feature to draw non-ascii characters.

The sample program (sample_draw_rects_for_non_ascii_labels.launch) needs to build jsk_recognition_utils in order to install Japanese fonts.

Copy link
Member

@708yamaguchi 708yamaguchi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I do the following command,

roslaunch jsk_perception sample_draw_rects_for_non_ascii_labels.launch

I got error.

Traceback (most recent call last):
  File "/home/leus/ros/melodic/src/jsk-ros-pkg/jsk_recognition/jsk_perception/node_scripts/draw_rects.py", line 10, in <module>
    from jsk_recognition_utils.color import labelcolormap
  File "/home/leus/ros/melodic/devel/lib/python2.7/dist-packages/jsk_recognition_utils/__init__.py", line 34, in <module>
    exec(__fh.read())
  File "<string>", line 8, in <module>
  File "/home/leus/ros/melodic/src/jsk-ros-pkg/jsk_recognition/jsk_recognition_utils/python/jsk_recognition_utils/mask.py", line 5, in <module>
    from skimage.morphology import binary_closing
  File "/usr/lib/python2.7/dist-packages/skimage/__init__.py", line 158, in <module>
    from .util.dtype import *
  File "/usr/lib/python2.7/dist-packages/skimage/util/__init__.py", line 7, in <module>
    from .arraycrop import crop
  File "/usr/lib/python2.7/dist-packages/skimage/util/arraycrop.py", line 8, in <module>
    from numpy.lib.arraypad import _validate_lengths
ImportError: cannot import name _validate_lengths
[draw_rects-5] process has died [pid 19319, exit code 1, cmd /home/leus/ros/melodic/src/jsk-ros-pkg/jsk_recognition/jsk_perception/node_scripts/draw_rects.py ~input:=image_publisher/output ~input/rects:=mask_rcnn_73b2_kitchen/output/rects __name:=draw_rects __log:=/home/leus/.ros/log/9748f09c-631e-11ec-9e
7f-0242c7226360/draw_rects-5.log].
log file: /home/leus/.ros/log/9748f09c-631e-11ec-9e7f-0242c7226360/draw_rects-5*.log
Traceback (most recent call last):
  File "/home/leus/ros/melodic/src/jsk-ros-pkg/jsk_recognition/jsk_perception/node_scripts/draw_rects.py", line 10, in <module>
    from jsk_recognition_utils.color import labelcolormap
  File "/home/leus/ros/melodic/devel/lib/python2.7/dist-packages/jsk_recognition_utils/__init__.py", line 34, in <module>
    exec(__fh.read())
  File "<string>", line 8, in <module>
  File "/home/leus/ros/melodic/src/jsk-ros-pkg/jsk_recognition/jsk_recognition_utils/python/jsk_recognition_utils/mask.py", line 5, in <module>
    from skimage.morphology import binary_closing
  File "/usr/lib/python2.7/dist-packages/skimage/__init__.py", line 158, in <module>
    from .util.dtype import *
  File "/usr/lib/python2.7/dist-packages/skimage/util/__init__.py", line 7, in <module>
    from .arraycrop import crop
  File "/usr/lib/python2.7/dist-packages/skimage/util/arraycrop.py", line 8, in <module>
    from numpy.lib.arraypad import _validate_lengths
ImportError: cannot import name _validate_lengths
[draw_rects_with_classification_result-6] process has died [pid 19332, exit code 1, cmd /home/leus/ros/melodic/src/jsk-ros-pkg/jsk_recognition/jsk_perception/node_scripts/draw_rects.py ~input:=image_publisher/output ~input/rects:=mask_rcnn_73b2_kitchen/output/rects ~input/class:=mask_rcnn_73b2_kitchen/output/class __name:=draw_rects_with_classification_result __log:=/home/leus/.ros/log/9748f09c-631e-11ec-9e7f-0242c7226360/draw_rects_with_classification_result-6.log].
log file: /home/leus/.ros/log/9748f09c-631e-11ec-9e7f-0242c7226360/draw_rects_with_classification_result-6*.log

Maybe related to scikit-image/scikit-image#3906 ?

My environment is

$ pip show scikit-image
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Name: scikit-image
Version: 0.13.1
Summary: Image processing routines for SciPy
Home-page: http://scikit-image.org
Author: Stefan van der Walt
Author-email: stefan@sun.ac.za
License: Modified BSD
Location: /usr/lib/python2.7/dist-packages
Requires: PyWavelets, networkx, pillow, six
Required-by: imgaug, fcn
$ pip show numpy
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Name: numpy
Version: 1.16.6
Summary: NumPy is the fundamental package for array computing with Python.
Home-page: https://www.numpy.org
Author: Travis E. Oliphant et al.
Author-email: None
License: BSD
Location: /home/leus/.local/lib/python2.7/site-packages
Requires:
Required-by: scipy, numpy-quaternion, pandas, matplotlib, imgaug, fcn, cupy-cuda101, chainer

@708yamaguchi
Copy link
Member

I think draw_rects for non ascii characters is great, but can't this feature be merged into draw_rects.cpp?

@tkmtnt7000
Copy link
Member

tkmtnt7000 commented Dec 22, 2021

I got the following error.

$ roslaunch jsk_perception sample_draw_rects_for_non_ascii_labels.launch
... logging to /home/tsukamoto/.ros/log/ba5519ce-6320-11ec-8e97-5405dbb276e2/roslaunch-tsukamoto-p15s-24345.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://tsukamoto-p15s:46131/

SUMMARY
========

CLEAR PARAMETERS
 * /draw_rects/
 * /draw_rects_with_classification_result/

PARAMETERS
 * /draw_rects/label_size: 16
 * /draw_rects/use_classification_result: False
 * /draw_rects_with_classification_result/font_path: /home/tsukamoto/r...
 * /draw_rects_with_classification_result/label_size: 16
 * /draw_rects_with_classification_result/use_classification_result: True
 * /rosdistro: melodic
 * /rosversion: 1.14.12
 * /use_sim_time: True

NODES
  /
    draw_rects (jsk_perception/draw_rects.py)
    draw_rects_with_classification_result (jsk_perception/draw_rects.py)
    image_view00 (image_view/image_view)
    image_view01 (image_view/image_view)
    rosbag_play (rosbag/play)
  /image_publisher/
    republish (image_transport/republish)
  /mask_rcnn_73b2_kitchen/output/
    republish (image_transport/republish)

auto-starting new master
process[master]: started with pid [24362]
ROS_MASTER_URI=http://localhost:11311

setting /run_id to ba5519ce-6320-11ec-8e97-5405dbb276e2
process[rosout-1]: started with pid [24383]
started core service [/rosout]
process[rosbag_play-2]: started with pid [24391]
process[image_publisher/republish-3]: started with pid [24399]
process[mask_rcnn_73b2_kitchen/output/republish-4]: started with pid [24400]
process[draw_rects-5]: started with pid [24432]
process[draw_rects_with_classification_result-6]: started with pid [24465]
process[image_view00-7]: started with pid [24467]
process[image_view01-8]: started with pid [24469]
Traceback (most recent call last):
  File "/home/tsukamoto/ros/jsk_recognition_ws/src/jsk_recognition/jsk_perception/node_scripts/draw_rects.py", line 10, in <module>
    from jsk_recognition_utils.color import labelcolormap
  File "/home/tsukamoto/ros/jsk_recognition_ws/devel/lib/python2.7/dist-packages/jsk_recognition_utils/__init__.py", line 34, in <module>
    exec(__fh.read())
  File "<string>", line 21, in <module>
  File "/usr/local/lib/python2.7/dist-packages/chainercv/__init__.py", line 4, in <module>
    from chainercv import datasets  # NOQA
  File "/usr/local/lib/python2.7/dist-packages/chainercv/datasets/__init__.py", line 1, in <module>
    from chainercv.datasets.ade20k.ade20k_semantic_segmentation_dataset import ADE20KSemanticSegmentationDataset  # NOQA
  File "/usr/local/lib/python2.7/dist-packages/chainercv/datasets/ade20k/ade20k_semantic_segmentation_dataset.py", line 7, in <module>
    from chainercv.datasets.ade20k.ade20k_utils import get_ade20k
  File "/usr/local/lib/python2.7/dist-packages/chainercv/datasets/ade20k/ade20k_utils.py", line 8, in <module>
    from chainercv import utils
  File "/usr/local/lib/python2.7/dist-packages/chainercv/utils/__init__.py", line 2, in <module>
    from chainercv.utils.bbox.non_maximum_suppression import non_maximum_suppression  # NOQA
  File "/usr/local/lib/python2.7/dist-packages/chainercv/utils/bbox/non_maximum_suppression.py", line 6, in <module>
    from chainercv.utils.bbox._nms_gpu_post import _nms_gpu_post
  File "__init__.pxd", line 872, in init chainercv.utils.bbox._nms_gpu_post
ValueError: numpy.ufunc has the wrong size, try recompiling. Expected 192, got 216
Traceback (most recent call last):
  File "/home/tsukamoto/ros/jsk_recognition_ws/src/jsk_recognition/jsk_perception/node_scripts/draw_rects.py", line 10, in <module>
    from jsk_recognition_utils.color import labelcolormap
  File "/home/tsukamoto/ros/jsk_recognition_ws/devel/lib/python2.7/dist-packages/jsk_recognition_utils/__init__.py", line 34, in <module>
    exec(__fh.read())
  File "<string>", line 21, in <module>
  File "/usr/local/lib/python2.7/dist-packages/chainercv/__init__.py", line 4, in <module>
    from chainercv import datasets  # NOQA
  File "/usr/local/lib/python2.7/dist-packages/chainercv/datasets/__init__.py", line 1, in <module>
    from chainercv.datasets.ade20k.ade20k_semantic_segmentation_dataset import ADE20KSemanticSegmentationDataset  # NOQA
  File "/usr/local/lib/python2.7/dist-packages/chainercv/datasets/ade20k/ade20k_semantic_segmentation_dataset.py", line 7, in <module>
    from chainercv.datasets.ade20k.ade20k_utils import get_ade20k
  File "/usr/local/lib/python2.7/dist-packages/chainercv/datasets/ade20k/ade20k_utils.py", line 8, in <module>
    from chainercv import utils
  File "/usr/local/lib/python2.7/dist-packages/chainercv/utils/__init__.py", line 2, in <module>
    from chainercv.utils.bbox.non_maximum_suppression import non_maximum_suppression  # NOQA
  File "/usr/local/lib/python2.7/dist-packages/chainercv/utils/bbox/non_maximum_suppression.py", line 6, in <module>
    from chainercv.utils.bbox._nms_gpu_post import _nms_gpu_post
  File "__init__.pxd", line 872, in init chainercv.utils.bbox._nms_gpu_post
ValueError: numpy.ufunc has the wrong size, try recompiling. Expected 192, got 216
[draw_rects-5] process has died [pid 24432, exit code 1, cmd /home/tsukamoto/ros/jsk_recognition_ws/src/jsk_recognition/jsk_perception/node_scripts/draw_rects.py ~input:=image_publisher/output ~input/rects:=mask_rcnn_73b2_kitchen/output/rects __name:=draw_rects __log:=/home/tsukamoto/.ros/log/ba5519ce-6320-11ec-8e97-5405dbb276e2/draw_rects-5.log].
log file: /home/tsukamoto/.ros/log/ba5519ce-6320-11ec-8e97-5405dbb276e2/draw_rects-5*.log
[draw_rects_with_classification_result-6] process has died [pid 24465, exit code 1, cmd /home/tsukamoto/ros/jsk_recognition_ws/src/jsk_recognition/jsk_perception/node_scripts/draw_rects.py ~input:=image_publisher/output ~input/rects:=mask_rcnn_73b2_kitchen/output/rects ~input/class:=mask_rcnn_73b2_kitchen/output/class __name:=draw_rects_with_classification_result __log:=/home/tsukamoto/.ros/log/ba5519ce-6320-11ec-8e97-5405dbb276e2/draw_rects_with_classification_result-6.log].
log file: /home/tsukamoto/.ros/log/ba5519ce-6320-11ec-8e97-5405dbb276e2/draw_rects_with_classification_result-6*.log

My environment is

$ pip show numpy
Name: numpy
Version: 1.13.3
Summary: NumPy: array processing for numbers, strings, records, and objects.
Home-page: http://www.numpy.org
Author: NumPy Developers
Author-email: numpy-discussion@python.org
License: BSD
Location: /usr/lib/python2.7/dist-packages
Requires: 

$ pip show scikit-image
Name: scikit-image
Version: 0.14.5
Summary: Image processing routines for SciPy
Home-page: http://scikit-image.org
Author: None
Author-email: None
License: Modified BSD
Location: /usr/local/lib/python2.7/dist-packages
Requires: PyWavelets, scipy, six, cloudpickle, networkx, matplotlib, pillow
$ pip show chainer
Name: chainer
Version: 6.7.0
Summary: A flexible framework of neural networks
Home-page: https://chainer.org/
Author: Seiya Tokui
Author-email: tokui@preferred.jp
License: MIT License
Location: /usr/local/lib/python2.7/dist-packages
Requires: setuptools, typing, typing-extensions, filelock, numpy, protobuf, six
$ pip show chainercv
Name: chainercv
Version: 0.13.1
Summary: 
Home-page: None
Author: None
Author-email: None
License: None
Location: /usr/local/lib/python2.7/dist-packages
Requires: Pillow, chainer

@iory
Copy link
Member Author

iory commented Dec 22, 2021

@708yamaguchi @tkmtnt7000
Thank you for your error reports.
Could you run another program using labelcolormap such as label_image_decomposer.py?

@iory
Copy link
Member Author

iory commented Dec 22, 2021

I think draw_rects for non ascii characters is great, but can't this feature be merged into draw_rects.cpp?

That is the best solution. However, the old opencv doesn't support utf-8 in putText.
https://github.com/opencv/opencv/wiki/2020#2020-07-31

The good thing about this PR is that the put_text_to_image function has been added to jsk_recognition_utils, so it can be easily used in other python programs.
https://github.com/jsk-ros-pkg/jsk_recognition/pull/2648/files#diff-99735ddcaeeb6616718cce3530dd7c595ab24990d3854494e8608ff7d8591a68R7

Replacing the put_text_to_image function with a program that uses putText internally, such as LabelImageDecomposer, is a future direction.
https://github.com/jsk-ros-pkg/jsk_recognition/blob/master/jsk_perception/node_scripts/label_image_decomposer.py#L81

@708yamaguchi
Copy link
Member

Could you run another program using labelcolormap such as label_image_decomposer.py?

Sorry, I understand. This error is not related to this PR.

That is the best solution. However, the old opencv doesn't support utf-8 in putText. opencv/opencv/wiki/2020#2020-07-31

I see. Thank you for your explanation.

@708yamaguchi
Copy link
Member

It seems that install_font_data.py is executed when jsk_recognition_utils is built.
However, sample_draw_rects_for_non_ascii_labels.launch is located in jsk_perception.
I think this is confusing.
Please add document about how to install font data.

@708yamaguchi
Copy link
Member

I succeeded executing sample_draw_rects_for_non_ascii_labels.launch.
Thank you.

However, I got the following image.
I am sorry, but I have no idea about how to debug this.
What should I check?
frame0000

@iory
Copy link
Member Author

iory commented Dec 22, 2021

I think this is confusing.
Please add document about how to install font data.

OK. I added it.
d385773

However, I got the following image.

What is the result of running the following program?

roslaunch jsk_perception sample_draw_rects_for_non_ascii_labels.launch use_japanese_label:=false

@708yamaguchi
Copy link
Member

What is the result of running the following program?
roslaunch jsk_perception sample_draw_rects_for_non_ascii_labels.launch use_japanese_label:=false

Thank you for your help.
It seems good to me.
frame0000

@iory
Copy link
Member Author

iory commented Dec 22, 2021

I fixed the problem. 4f6c797
Please check it.

@708yamaguchi
Copy link
Member

It works!
Thank you.

frame0000

@tkmtnt7000
Copy link
Member

tkmtnt7000 commented Dec 22, 2021

It works, too! Thank you.

I think numpy=>1.16.0 is needed, but the version of numpy that comes with sudo apt install python-numpy is 1.13.3 in ubuntu 18.04. https://packages.ubuntu.com/en/bionic/python-numpy
If numpy<1.16.0, I got the error I mentioned before.

@tkmtnt7000
Copy link
Member

tkmtnt7000 commented Dec 23, 2021

I am sorry, I put jsk_recognition in a clean environment and I confirmed rosdep resolved the numpy version. (numpy==1.16.6 is installed automatically)
It seems to be because I mistakenly uninstalled numpy==1.16.6 manually.

I think numpy=>1.16.0 is needed, but the version of numpy that comes with sudo apt install python-numpy is 1.13.3 in ubuntu 18.04. packages.ubuntu.com/en/bionic/python-numpy
If numpy<1.16.0, I got the error I mentioned before.

@k-okada
Copy link
Member

k-okada commented Dec 23, 2021 via email

@tkmtnt7000
Copy link
Member

tkmtnt7000 commented Dec 23, 2021

cc:@708yamaguchi
At first I thought it was because labelcolormap depends on the version of numpy, but I started to think that it is not a numpy version dependency problem.

labelcolormapの内部で行われていること

  • labelcolormapを呼ぶとchainercvが呼ばれる
  • chainercvの中でnumpyが呼ばれている.

numpy/numpy#12785
このissueで言われていることと同様に
chainercvライブラリのコンパイル時に使っているバージョンより低いバージョンのnumpyを使うと,cythonが期待しているよりも小さなCヘッダとなってしまい,以下の最後の行のようなエラーが出ると考えられます.(すみません,よくわかりませんでした.)

Traceback (most recent call last):
  File "/home/tsukamoto/ros/jsk_recognition_ws/src/jsk_recognition/jsk_perception/node_scripts/draw_rects.py", line 10, in <module>
    from jsk_recognition_utils.color import labelcolormap
  File "/home/tsukamoto/ros/jsk_recognition_ws/devel/lib/python2.7/dist-packages/jsk_recognition_utils/__init__.py", line 34, in <module>
    exec(__fh.read())
  File "<string>", line 21, in <module>
  File "/usr/local/lib/python2.7/dist-packages/chainercv/__init__.py", line 4, in <module>
    from chainercv import datasets  # NOQA
  File "/usr/local/lib/python2.7/dist-packages/chainercv/datasets/__init__.py", line 1, in <module>
    from chainercv.datasets.ade20k.ade20k_semantic_segmentation_dataset import ADE20KSemanticSegmentationDataset  # NOQA
  File "/usr/local/lib/python2.7/dist-packages/chainercv/datasets/ade20k/ade20k_semantic_segmentation_dataset.py", line 7, in <module>
    from chainercv.datasets.ade20k.ade20k_utils import get_ade20k
  File "/usr/local/lib/python2.7/dist-packages/chainercv/datasets/ade20k/ade20k_utils.py", line 8, in <module>
    from chainercv import utils
  File "/usr/local/lib/python2.7/dist-packages/chainercv/utils/__init__.py", line 2, in <module>
    from chainercv.utils.bbox.non_maximum_suppression import non_maximum_suppression  # NOQA
  File "/usr/local/lib/python2.7/dist-packages/chainercv/utils/bbox/non_maximum_suppression.py", line 6, in <module>
    from chainercv.utils.bbox._nms_gpu_post import _nms_gpu_post
  File "__init__.pxd", line 872, in init chainercv.utils.bbox._nms_gpu_post
ValueError: numpy.ufunc has the wrong size, try recompiling. Expected 192, got 216
Traceback (most recent call last):
  File "/home/tsukamoto/ros/jsk_recognition_ws/src/jsk_recognition/jsk_perception/node_scripts/draw_rects.py", line 10, in <module>
    from jsk_recognition_utils.color import labelcolormap
  File "/home/tsukamoto/ros/jsk_recognition_ws/devel/lib/python2.7/dist-packages/jsk_recognition_utils/__init__.py", line 34, in <module>
    exec(__fh.read())
  File "<string>", line 21, in <module>
  File "/usr/local/lib/python2.7/dist-packages/chainercv/__init__.py", line 4, in <module>
    from chainercv import datasets  # NOQA
  File "/usr/local/lib/python2.7/dist-packages/chainercv/datasets/__init__.py", line 1, in <module>
    from chainercv.datasets.ade20k.ade20k_semantic_segmentation_dataset import ADE20KSemanticSegmentationDataset  # NOQA
  File "/usr/local/lib/python2.7/dist-packages/chainercv/datasets/ade20k/ade20k_semantic_segmentation_dataset.py", line 7, in <module>
    from chainercv.datasets.ade20k.ade20k_utils import get_ade20k
  File "/usr/local/lib/python2.7/dist-packages/chainercv/datasets/ade20k/ade20k_utils.py", line 8, in <module>
    from chainercv import utils
  File "/usr/local/lib/python2.7/dist-packages/chainercv/utils/__init__.py", line 2, in <module>
    from chainercv.utils.bbox.non_maximum_suppression import non_maximum_suppression  # NOQA
  File "/usr/local/lib/python2.7/dist-packages/chainercv/utils/bbox/non_maximum_suppression.py", line 6, in <module>
    from chainercv.utils.bbox._nms_gpu_post import _nms_gpu_post
  File "__init__.pxd", line 872, in init chainercv.utils.bbox._nms_gpu_post
ValueError: numpy.ufunc has the wrong size, try recompiling. Expected 192, got 216

@k-okada k-okada merged commit 6ff031d into jsk-ros-pkg:master Jan 5, 2022
@iory iory deleted the draw-rects-for-non-ascii branch January 5, 2022 12:58
k-okada added a commit that referenced this pull request Nov 12, 2023
#2648 requires jsk_data package
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants