convert mask binary image to polygon format

[raninbowlalala]

I want to create a new dataset same as coco format, and now I have converted mask binary image to RLE format by using encode function in mask.py. But I don't know how to convert mask binary image or RLE format to polygon format, someone can help me? Thanks in advance.

[waspinator]

You can try this. I haven't tested it to actually work as a valid COCO format though. Let me know if it works for you.

import json
import numpy as np
from pycocotools import mask
from skimage import measure

ground_truth_binary_mask = np.array([[  0,   0,   0,   0,   0,   0,   0,   0,   0,   0],
                                     [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0],
                                     [  0,   0,   0,   0,   0,   1,   1,   1,   0,   0],
                                     [  0,   0,   0,   0,   0,   1,   1,   1,   0,   0],
                                     [  0,   0,   0,   0,   0,   1,   1,   1,   0,   0],
                                     [  0,   0,   0,   0,   0,   1,   1,   1,   0,   0],
                                     [  1,   0,   0,   0,   0,   0,   0,   0,   0,   0],
                                     [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0],
                                     [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0]], dtype=np.uint8)

fortran_ground_truth_binary_mask = np.asfortranarray(ground_truth_binary_mask)
encoded_ground_truth = mask.encode(fortran_ground_truth_binary_mask)
ground_truth_area = mask.area(encoded_ground_truth)
ground_truth_bounding_box = mask.toBbox(encoded_ground_truth)
contours = measure.find_contours(ground_truth_binary_mask, 0.5)

annotation = {
        "segmentation": [],
        "area": ground_truth_area.tolist(),
        "iscrowd": 0,
        "image_id": 123,
        "bbox": ground_truth_bounding_box.tolist(),
        "category_id": 1,
        "id": 1
    }

for contour in contours:
    contour = np.flip(contour, axis=1)
    segmentation = contour.ravel().tolist()
    annotation["segmentation"].append(segmentation)
    
print(json.dumps(annotation, indent=4))

produces the following output:

{
    "segmentation": [
        [
            7.0,
            5.5,
            6.0,
            5.5,
            5.0,
            5.5,
            4.5,
            5.0,
            4.5,
            4.0,
            4.5,
            3.0,
            4.5,
            2.0,
            5.0,
            1.5,
            6.0,
            1.5,
            7.0,
            1.5,
            7.5,
            2.0,
            7.5,
            3.0,
            7.5,
            4.0,
            7.5,
            5.0,
            7.0,
            5.5
        ],
        [
            0.0,
            5.5,
            0.5,
            6.0,
            0.0,
            6.5
        ]
    ],
    "area": 13,
    "iscrowd": 0,
    "image_id": 123,
    "bbox": [
        0.0,
        2.0,
        8.0,
        5.0
    ],
    "category_id": 1,
    "id": 1
}

[raninbowlalala]

@waspinator I modified your code to read a file path of mask binary images, and it worked for me. Thank you very much!

[JerryLingjieMei]

@waspinator I was wondering why you would use np.flip on a contour? Is it because of the difference between c-like and Fortran-like numpy arrays?

[waspinator]

@JerryLingjieMei I think it was just because [x, y] coordinates were in the wrong order.

I've since worked on packaging up a tool to create coco json files. Take a look at https://github.com/waspinator/pycococreator/

[thecondofitz]

First of all, thank you @waspinator for all the tools that you've created for us to use. You've saved me so much time along the way.
I recently ran into some issues trying to write Mask R-CNN predictions/results to json. More specifically, in the Crowd-AI mapping challenge. The coordinates come out as a long string of letters and characters. here's one for example:

"segmentation": {"size": [512, 512], "counts": "UjU5g2W=6L1O01N3TOdVg2"}, "bbox": [331.0, 321.0, 7.0, 97.0]}, {"image_id": "1Z514B22DA1B2XYZ.jpg", "category_id": 1, "score": 0.9703407883644104, "segmentation": {"size": [512, 512], "counts": "dgU5l1P=U1N1001Nh0cM_Zg2"}, "bbox": [331.0, 207.0, 6.0, 102.0]}, {"image_id": "1Z412D6B6A1B2XYZ.jpg", "category_id": 2, "score": 0.9999915361404419,

that's part of the output of the code in the predictions notebook

Gather all JPG files in the test set as small batches

files = glob.glob(os.path.join(IMAGE_DIR, "*.jpg"))
ALL_FILES=[]
_buffer = []
for _idx, _file in enumerate(files):
if len(_buffer) == config.IMAGES_PER_GPU * config.GPU_COUNT:
ALL_FILES.append(_buffer)
_buffer = []
else:
_buffer.append(_file)

if len(_buffer) > 0:
ALL_FILES.append(_buffer)

Iterate over all the batches and predict

_final_object = []
for files in tqdm.tqdm(ALL_FILES):
images = [skimage.io.imread(x) for x in files]
predictions = model.detect(images, verbose=0)
for _idx, r in enumerate(predictions):
_file = files[_idx]
image_id = os.path.basename(_file)
for _idx, class_id in enumerate(r["class_ids"]):
if class_id == 1:
mask = r["masks"].astype(np.uint8)[:, :, _idx]
bbox = np.around(r["rois"][_idx], 1)
bbox = [float(x) for x in bbox]
_result = {}
_result["image_id"] = image_id
_result["category_id"] = 1
_result["score"] = float(r["scores"][_idx])
_mask = maskUtils.encode(np.asfortranarray(mask))
_mask["counts"] = _mask["counts"].decode("UTF-8")
_result["segmentation"] = _mask
_result["bbox"] = [bbox[1], bbox[0], bbox[3] - bbox[1], bbox[2] - bbox[0]]
_final_object.append(_result)

fp = open("predictions.json", "w")
import json
print("Writing JSON...")
fp.write(json.dumps(_final_object))
fp.close()

Any idea what I'm doing wrong? I'd like to see the mask coordinates in the same format as in your pycococreator.

[atlurip]

Hi Now i am facing the same problem.Please let me know the solution if you have fixed

[ghost]

You can try this. I haven't tested it to actually work as a valid COCO format though. Let me know if it works for you.

import json
import numpy as np
from pycocotools import mask
from skimage import measure

ground_truth_binary_mask = np.array([[  0,   0,   0,   0,   0,   0,   0,   0,   0,   0],
                                     [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0],
                                     [  0,   0,   0,   0,   0,   1,   1,   1,   0,   0],
                                     [  0,   0,   0,   0,   0,   1,   1,   1,   0,   0],
                                     [  0,   0,   0,   0,   0,   1,   1,   1,   0,   0],
                                     [  0,   0,   0,   0,   0,   1,   1,   1,   0,   0],
                                     [  1,   0,   0,   0,   0,   0,   0,   0,   0,   0],
                                     [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0],
                                     [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0]], dtype=np.uint8)

fortran_ground_truth_binary_mask = np.asfortranarray(ground_truth_binary_mask)
encoded_ground_truth = mask.encode(fortran_ground_truth_binary_mask)
ground_truth_area = mask.area(encoded_ground_truth)
ground_truth_bounding_box = mask.toBbox(encoded_ground_truth)
contours = measure.find_contours(ground_truth_binary_mask, 0.5)

annotation = {
        "segmentation": [],
        "area": ground_truth_area.tolist(),
        "iscrowd": 0,
        "image_id": 123,
        "bbox": ground_truth_bounding_box.tolist(),
        "category_id": 1,
        "id": 1
    }

for contour in contours:
    contour = np.flip(contour, axis=1)
    segmentation = contour.ravel().tolist()
    annotation["segmentation"].append(segmentation)
    
print(json.dumps(annotation, indent=4))

produces the following output:

{
    "segmentation": [
        [
            7.0,
            5.5,
            6.0,
            5.5,
            5.0,
            5.5,
            4.5,
            5.0,
            4.5,
            4.0,
            4.5,
            3.0,
            4.5,
            2.0,
            5.0,
            1.5,
            6.0,
            1.5,
            7.0,
            1.5,
            7.5,
            2.0,
            7.5,
            3.0,
            7.5,
            4.0,
            7.5,
            5.0,
            7.0,
            5.5
        ],
        [
            0.0,
            5.5,
            0.5,
            6.0,
            0.0,
            6.5
        ]
    ],
    "area": 13,
    "iscrowd": 0,
    "image_id": 123,
    "bbox": [
        0.0,
        2.0,
        8.0,
        5.0
    ],
    "category_id": 1,
    "id": 1
}

how to make this code work for multiple classes not only binary?

[SevenMpp]

@waspinator I modified your code to read a file path of mask binary images, and it worked for me. Thank you very much!
Could you please share your method with me

[kimsangseob]

You can try this. I haven't tested it to actually work as a valid COCO format though. Let me know if it works for you.

import json
import numpy as np
from pycocotools import mask
from skimage import measure

ground_truth_binary_mask = np.array([[  0,   0,   0,   0,   0,   0,   0,   0,   0,   0],
                                     [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0],
                                     [  0,   0,   0,   0,   0,   1,   1,   1,   0,   0],
                                     [  0,   0,   0,   0,   0,   1,   1,   1,   0,   0],
                                     [  0,   0,   0,   0,   0,   1,   1,   1,   0,   0],
                                     [  0,   0,   0,   0,   0,   1,   1,   1,   0,   0],
                                     [  1,   0,   0,   0,   0,   0,   0,   0,   0,   0],
                                     [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0],
                                     [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0]], dtype=np.uint8)

fortran_ground_truth_binary_mask = np.asfortranarray(ground_truth_binary_mask)
encoded_ground_truth = mask.encode(fortran_ground_truth_binary_mask)
ground_truth_area = mask.area(encoded_ground_truth)
ground_truth_bounding_box = mask.toBbox(encoded_ground_truth)
contours = measure.find_contours(ground_truth_binary_mask, 0.5)

annotation = {
        "segmentation": [],
        "area": ground_truth_area.tolist(),
        "iscrowd": 0,
        "image_id": 123,
        "bbox": ground_truth_bounding_box.tolist(),
        "category_id": 1,
        "id": 1
    }

for contour in contours:
    contour = np.flip(contour, axis=1)
    segmentation = contour.ravel().tolist()
    annotation["segmentation"].append(segmentation)
    
print(json.dumps(annotation, indent=4))

produces the following output:

{
    "segmentation": [
        [
            7.0,
            5.5,
            6.0,
            5.5,
            5.0,
            5.5,
            4.5,
            5.0,
            4.5,
            4.0,
            4.5,
            3.0,
            4.5,
            2.0,
            5.0,
            1.5,
            6.0,
            1.5,
            7.0,
            1.5,
            7.5,
            2.0,
            7.5,
            3.0,
            7.5,
            4.0,
            7.5,
            5.0,
            7.0,
            5.5
        ],
        [
            0.0,
            5.5,
            0.5,
            6.0,
            0.0,
            6.5
        ]
    ],
    "area": 13,
    "iscrowd": 0,
    "image_id": 123,
    "bbox": [
        0.0,
        2.0,
        8.0,
        5.0
    ],
    "category_id": 1,
    "id": 1
}

Where can I put my input data. I mean I want to convert PNG files to json COCO dataset format, which has x,y cordinates. I want use your code but I don't know where can I put my images directory.
Can you help me...?

[IgnacioAmat]

Hi @kimsangseob, for converting an image you will just need to change the value of the variable ground_truth_binary_mask from :
ground_truth_binary_mask = np.array([[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [ 0, 0, 0, 0, 0, 1, 1, 1, 0, 0], [ 0, 0, 0, 0, 0, 1, 1, 1, 0, 0], [ 0, 0, 0, 0, 0, 1, 1, 1, 0, 0], [ 0, 0, 0, 0, 0, 1, 1, 1, 0, 0], [ 1, 0, 0, 0, 0, 0, 0, 0, 0, 0], [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]], dtype=np.uint8)
to :
ground_truth_binary_mask = cv2.imread(image_location, 0)

Hope it helps !

[narimanakh]

@waspinator I modified your code to read a file path of mask binary images, and it worked for me. Thank you very much!
Could you please share your method with me

can u help me how to read binary images from path and convert it to json using the previous code

[narimanakh]

can anyone help me to convert polygon format to image to show it

[narimanakh]

You can try this. I haven't tested it to actually work as a valid COCO format though. Let me know if it works for you.

import json
import numpy as np
from pycocotools import mask
from skimage import measure

ground_truth_binary_mask = np.array([[  0,   0,   0,   0,   0,   0,   0,   0,   0,   0],
                                     [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0],
                                     [  0,   0,   0,   0,   0,   1,   1,   1,   0,   0],
                                     [  0,   0,   0,   0,   0,   1,   1,   1,   0,   0],
                                     [  0,   0,   0,   0,   0,   1,   1,   1,   0,   0],
                                     [  0,   0,   0,   0,   0,   1,   1,   1,   0,   0],
                                     [  1,   0,   0,   0,   0,   0,   0,   0,   0,   0],
                                     [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0],
                                     [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0]], dtype=np.uint8)

fortran_ground_truth_binary_mask = np.asfortranarray(ground_truth_binary_mask)
encoded_ground_truth = mask.encode(fortran_ground_truth_binary_mask)
ground_truth_area = mask.area(encoded_ground_truth)
ground_truth_bounding_box = mask.toBbox(encoded_ground_truth)
contours = measure.find_contours(ground_truth_binary_mask, 0.5)

annotation = {
        "segmentation": [],
        "area": ground_truth_area.tolist(),
        "iscrowd": 0,
        "image_id": 123,
        "bbox": ground_truth_bounding_box.tolist(),
        "category_id": 1,
        "id": 1
    }

for contour in contours:
    contour = np.flip(contour, axis=1)
    segmentation = contour.ravel().tolist()
    annotation["segmentation"].append(segmentation)
    
print(json.dumps(annotation, indent=4))

produces the following output:

{
    "segmentation": [
        [
            7.0,
            5.5,
            6.0,
            5.5,
            5.0,
            5.5,
            4.5,
            5.0,
            4.5,
            4.0,
            4.5,
            3.0,
            4.5,
            2.0,
            5.0,
            1.5,
            6.0,
            1.5,
            7.0,
            1.5,
            7.5,
            2.0,
            7.5,
            3.0,
            7.5,
            4.0,
            7.5,
            5.0,
            7.0,
            5.5
        ],
        [
            0.0,
            5.5,
            0.5,
            6.0,
            0.0,
            6.5
        ]
    ],
    "area": 13,
    "iscrowd": 0,
    "image_id": 123,
    "bbox": [
        0.0,
        2.0,
        8.0,
        5.0
    ],
    "category_id": 1,
    "id": 1
}

Where can I put my input data. I mean I want to convert PNG files to json COCO dataset format, which has x,y cordinates. I want use your code but I don't know where can I put my images directory. Can you help me...?

how can i convert this format to mask image to show it