The RLE or Polygon format of "segmentation“ for extending to coco dataset #100
Comments
|
I had a similar issue: some of the functions in lib/utils/segms.py expect segmentations to be in “poly” format and break when those are provided in RLE.
It is inconvenient but seems to be in line with the spec for non-crowded regions(iscrowd=0):
The segmentation format depends on whether the instance represents a single object (iscrowd=0 in which case polygons are used) or a collection of objects (iscrowd=1 in which case RLE is used).
[1] http://cocodataset.org/#download, section "4.1. Object Instance Annotations”
The workaround for me was to transform to “poly” format, which is essentially a list of (x,y) vertexes.
…-Lesha.
On 2. Feb 2018, at 06:41, Hu Xingui ***@***.***> wrote:
Hi Detectron,
Recently I tried to add my custom coco data to run Detectron and encountered the following issues.
(1) "segmentation" in coco data like below，
{"segmentation": [[499.71, 397.28,......342.71, 172.31]], "area": 43466.12825, "iscrowd": 0, "image_id": 182155, "bbox": [338.89, 51.69, 205.82, 367.61], "category_id": 1, "id": 1248258},
{"segmentation": {"counts": [66916, 6, 587,..... 1, 114303], "size": [594, 640]}, "area": 6197, "iscrowd": 1, "image_id": 284445, "bbox": [112, 322, 335, 94], "category_id": 1, "id": 9.001002844e+11},
The first format of "segmentation" is polygon and the second is need to encode/decode for RLE format.
Above formats can run on Detectron.
(2) I added a new category , and generated a new RLE format for "segmentation" field via coco api encode()/decode() mask.
I generated data like this.
"segmentation": [{"counts": "mng=1fb02O1O1O001N2O001O1O0O2O1O1O001N2O001O1O0O2O1O001O1O1O010000O01000O010000O01000O01000O01000O01N2N2M2O2N2N1O2N2O001O10O?B000O10O1O001^OQ^O9Pb0EQ^O;Wb0OO01O1O1O001O1N2N`jT3","size": [600,1000]}]
I found the bolded characters is different from the original coco "segmentation" json format although it can run on MatterPort's implementation to Mask-RCNN.
Also, I tried to modify some Detectron's code to meet my requirement, but very difficult to me because lots of code need to change.
Could you give me some suggestions to run my custom data?
Thanks.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.
|
|
@amokeev ,How to convert "RLE" to "poly" format to your workaround? |
|
@amokeev , Are you sure the number in segmentation is (x,y) vertexes? For example "66916" ,a large number! Another, although I set "iscrowd" as "1" for RLE format, yet could not run on Detectron. |
|
I interpret poly as list of polygons, defined by the vertexes, like [[x1,y1,x2,y2…xN,yN],…[x1,y1,x2,y2…xN,yN] ], where the coordinates are of the same scale as the image.
The masks, encoded this way, are shown correctly by CocoAPI [1]
But you may want to get “official” answer.
[1] https://github.com/cocodataset/cocoapi <https://github.com/cocodataset/cocoapi>
… On 2. Feb 2018, at 09:18, Hu Xingui ***@***.***> wrote:
@amokeev <https://github.com/amokeev> , Are you sure the number in segmentation is (x,y) vertexes? For example "66916" ,a large number! Another, although I set "iscrowd" as "1" for RLE format, yet could not run on Detectron.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#100 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFlh63ObnXg-DcaDKwIi3pB4Ppig464Hks5tQsTkgaJpZM4R2tN3>.
|
|
@topcomma labels_info = []
for mask in mask_list:
# opencv 3.2
mask_new, contours, hierarchy = cv2.findContours((mask).astype(np.uint8), cv2.RETR_TREE,
cv2.CHAIN_APPROX_SIMPLE)
# before opencv 3.2
# contours, hierarchy = cv2.findContours((mask).astype(np.uint8), cv2.RETR_TREE,
# cv2.CHAIN_APPROX_SIMPLE)
segmentation = []
for contour in contours:
contour = contour.flatten().tolist()
# segmentation.append(contour)
if len(contour) > 4:
segmentation.append(contour)
if len(segmentation) == 0:
continue
# get area, bbox, category_id and so on
labels_info.append(
{
"segmentation": segmentation, # poly
"area": area, # segmentation area
"iscrowd": 0,
"image_id": index,
"bbox": [x1, y1, bbox_w, bbox_h],
"category_id": category_id,
"id": label_id
},
) |
|
@Sundrops , As your method to convert, can get the result of "poly" list. Thank you very much！ But I still not know why the coordinate in COCO json file is large/small number such as "66916" or "1"? |
|
@topcomma COCO annotations have two types of segmentation annotations
The polygon and uncompressed RLE will be converted to compact RLE format withe the MaskApi. |
|
@Sundrops , |
|
@topcomma , |
|
@lg12170226 ， |
|
@topcomma : I have a raw image and |
|
Just wondering if there is a way to convert compressed RLEs to polys/uncompressed RLEs? |
|
@realwecan after decoding RLE with pycocotools.mask.decode, you can check my implementation to generate polygons with opencv: |
|
@hazirbas: thanks for your code. Why not use Davis 2017 that contains instance segmenyation? Could we use your code to convert Davis 2017 to coco format to use this maskrcnn implementation? |
|
@John1231983 you need to modify the script accordingly for reading split files as well as db_info.yml file. For my own research, I needed it for DAVIS 2016. |
|
Another solution to generating polygons, but using |
|
how would you convert a binary mask or encoded RLE into uncompressed RLE for use in the "iscrowd: 1" "counts" field? |
|
@waspinator Segmentations with your code using skimage are differenct from the code of @Sundrops using cv2. Results using your code with skimage:
Results with @Sundrops code using cv2:
|
|
@Kongsea I haven't tested @Sundrops cv2 implementation, but the basic idea should be the same. They will produce different results since there are an infinite amount of sets of points you can use to describe a shape. But otherwise they should both work. I just didn't have cv2 installed so I wrote something that doesn't require it. |
|
@Kongsea @waspinator I have tested my code. It works. |
|
Thank you @Sundrops @waspinator . |
|
@waspinator Is any way to convert segmentation poly vertex to RLE? My target is made iscrowd=1 |
|
@Sundrops Why did you comment this part in your code? We indeed need to handle such a case, right? |
|
@Yuliang-Zou You should uncomment this part when contours are uesd for Detectron. Beacase the Detectron will treat it as a rectangle when len(contour) ==4. I have updated my previous code. |
|
@Sundrops Thanks. But we still need to handle |
|
@Yuliang-Zou Yes, but the code |
|
@Sundrops I see, thank you! |
|
I wrote a library and article to help with creating COCO style datasets. |
|
@Sundrops segmentation = ann['segmentation'] |
|
@manketon def is_contour_bad(c):
# approximate the contour
peri = cv2.arcLength(c, True)
approx = cv2.approxPolyDP(c, 0.02 * peri, True)
# return True if it is not a rectangle
return not len(approx) == 4
image = cv2.imread('xx.jpg')
contours = ann['segmentation']
mask = np.ones(image.shape[:2], dtype="uint8") * 255
# loop over the contours
for c in contours:
# if the contour is not a rectangle, draw it on the mask
if is_contour_bad(c):
cv2.drawContours(mask, [c], -1, 0, -1)
area = (mask==0).sum() |
|
@Sundrops @waspinator I have a question. If my original object mask has big holes, when it is converted to polygons, how should I correctly convert it back? The decode and merge functions in coco API would treat the hole as parts of the object, so when converted back the holes become masks. How should I do in this case? |
|
@wangg12 As far as I know COCO doesn't have a native way of encoding holes. |
for contour in contours:
contour = contour.flatten().tolist()
segmentation.append(contour)
if len(contour) > 4:
segmentation.append(contour)Hi @Sundrops , sincerely thanks for your codes. I'm a beginner on Detectron. I feel confused why the length of contour (which is a list) should be larger than 4, in other words, what will happen for the model when the length is smaller than 4. One of your reply said that Detectron will treat it as a rectangle. In my perspective, there may be some objects with the shape of rectangle, so I think it's fine. Also, I wonder if you append the contour for twice in one iteration. I think the right code should be. for contour in contours:
contour = contour.flatten().tolist()
if len(contour) > 4:
segmentation.append(contour)It' will be highly appreciated if you can give me some suggestions. Thank you so much! |
|
@BobZhangHT Yes, it should append the contour for once in one iteration. # cocoapi/PythonAPI/pycocotools/coco.py
def annToRLE(self, ann):
t = self.imgs[ann['image_id']]
h, w = t['height'], t['width']
segm = ann['segmentation']
if type(segm) == list:
rles = maskUtils.frPyObjects(segm, h, w)
rle = maskUtils.merge(rles)
......# cocoapi/PythonAPI/pycocotools/_mask.pyx
def frPyObjects(pyobj, h, w):
# encode rle from a list of python objects
if type(pyobj) == np.ndarray:
objs = frBbox(pyobj, h, w)
elif type(pyobj) == list and len(pyobj[0]) == 4:
objs = frBbox(pyobj, h, w)
elif type(pyobj) == list and len(pyobj[0]) > 4:
objs = frPoly(pyobj, h, w)
...... |
|
@Sundrops Thanks for your reply! |
How can I transform RLE to polygons when "iscrowd" is "1" because the matterport/MaskRCNN works only with polygons. @amokeev. I want to use the matterport implementation of maskRCNN with a coco dataset created using pycococreator. |
I still didn't understand what 66916 and 114303 stand for? |
I understood. 66916 is the number of 0 pixels before the first pixel in the mask. 114303 is the number of zeros after the last 0 pixel in the mask |
--> 12 encoded_ground_truth = mask.encode(fortran_ground_truth_binary_mask) |
---> 12 encoded_ground_truth = mask.encode(fortran_ground_truth_binary_mask) |
|
@Sundrops how do I convert mask images into json files for yolo v8 format |
Hi Detectron,
Recently I tried to add my custom coco data to run Detectron and encountered the following issues.
(1) "segmentation" in coco data like below，
{"segmentation": [[499.71, 397.28,......342.71, 172.31]], "area": 43466.12825, "iscrowd": 0, "image_id": 182155, "bbox": [338.89, 51.69, 205.82, 367.61], "category_id": 1, "id": 1248258},
{"segmentation": {"counts": [66916, 6, 587,..... 1, 114303], "size": [594, 640]}, "area": 6197, "iscrowd": 1, "image_id": 284445, "bbox": [112, 322, 335, 94], "category_id": 1, "id": 9.001002844e+11},
The first format of "segmentation" is polygon and the second is need to encode/decode for RLE format.
Above formats can run on Detectron.
(2) I added a new category , and generated a new RLE format for "segmentation" field via coco api encode()/decode() mask.
I generated data like this.
"segmentation": [{"counts": "mng=1fb02O1O1O001N2O001O1O0O2O1O1O001N2O001O1O0O2O1O001O1O1O010000O01000O010000O01000O01000O01000O01N2N2M2O2N2N1O2N2O001O10O?B000O10O1O001^OQ^O9Pb0EQ^O;Wb0OO01O1O1O001O1N2N`jT3","size": [600,1000]}]
I found the bolded characters is different from the original coco "segmentation" json format although it can run on MatterPort's implementation to Mask-RCNN.
Also, I tried to modify some Detectron's code to meet my requirement, but very difficult to me because lots of code need to change.
Could you give me some suggestions to run my custom data?
Thanks.
The text was updated successfully, but these errors were encountered: