Cookies help us deliver our services. By using our services, you agree to our use of cookies. Learn more

crowdAI is shutting down - please read our blog post for more information

Mapping Challenge

Building Missing Maps with Machine Learning


Ask for Help: I get 93.3% Precision Accuracy and 94% Recall Rate in Validation Images But I can not edit to the Correct Format is and Failed to Submitting

Posted by jestshen92 about 3 years ago

Dear Every One:


I found the challenge three days ago, and then, I proposed to attend it.

I extract the COCO format to Png files in order to train by our model in Semantic Segmentation.

The results in validation is amazing that I obtained 95% precision accuracy but I cannot edit the format into the correct way which make me get a very low scores. (Fail to use pngtococodemo in coco api)

I upload the results images in google drive https://drive.google.com/file/d/1AO4ySij4AtSTkenQkj8N-4yxzAAJtPmr/view?usp=sharing

I hope that anyone can help me, if the originizar read this, I hope that you can give me a chance to attend round two.

Best regards,

Shenlu Jiang

Posted by janickrohrbach  about 3 years ago |  Quote

Hi Jiang

I’m using a similar approach to yours, so you should be able to run my code. A lot of it is copied from the starter kit which is great. While you have correctly segmented the houses, you need to create a mask for every single house. First you should transform your predictions to a numpy array with the shape (60697, 300, 300, 1), dtype=np.bool_ with houses being ones and everything else zeros. If you have memory issues use keras.utils.io_utils.HDF5Matrix.

You can then give the different houses in every image different labels with the function label() from the scipy package. (preds is your array from above)

from scipy.ndimage import label
pred_labeled = [label(x) for x in preds]

Next you’re going to create the labels per house with the following code. bbox() generates the bounding boxes and single_annotation() will convert everything into the submission format. In order to calculate the score I use another array preds_percentage with float predictions between 0 and 1 instead of booleans. If you don’t have that you can also just set it to a fixed number e.g. 1.0. Now you can iterate through every image and every label within that image and create a list with the labels in the COCO format. If you don’t have an array x_test, you can just replace x_test.shape[0] with 60697.

def bbox(mask):
    rows = np.any(mask, axis=1)
    cols = np.any(mask, axis=0)
    ymin, ymax = np.where(rows)[0][[0, -1]]
    xmin, xmax = np.where(cols)[0][[0, -1]]
    return [float(xmin), float(ymin), float(xmax - xmin), float(ymax-ymin)]

def single_annotation(image_id, single_mask):
    _result = {}
    _result["image_id"] = image_id
    _result["category_id"] = 100 # as 100 is the category_id of Building
    _result["score"] = float(np.mean(preds_percentage[image_id, single_mask])) # a score between 0 and 1
    _mask = cocomask.encode(np.asfortranarray(single_mask.astype("uint8")))
    _mask["counts"] = _mask["counts"].decode("UTF-8")
    _result["segmentation"] = _mask
    _result["bbox"] = bbox(single_mask)
    return _result

pred_ann = []
for i in range(x_test.shape[0]):
    for m in range(pred_labeled[i][1]):
            pred_ann.append(single_annotation(i, pred_labeled[i][0] == m + 1))
    print("Progress {:2.1%}".format((i + 1) / x_test.shape[0]), end="\r")

The last step is saving the json file.

import json
fp = open("predictions.json", "w")

I hope this helped and you manage to upload you predictions!


Posted by jestshen92  about 3 years ago |  Quote

Dear brothers, can you pasta the whole code? I have tried but with some error. Currently I save a whole image into the computer with H X W format and then, transfer it similiar to yours. Does it have a function to spilit the labels?

Posted by jestshen92  about 3 years ago |  Quote

Can you please tell me what is cocomask? cocomask.encode(np.asfortranarray(single_mask.astype(“uint8”)))

Posted by Ursus  about 3 years ago |  Quote

Can you please tell me what is cocomask? cocomask.encode(np.asfortranarray(single_mask.astype(“uint8”))

from pycocotools import mask as cocomask

Posted by neptune.ml  about 3 years ago |  Quote

If you are still having trouble our entire code that generates predictions can be found here


Also remember that it is really important to provide scores for detected objects. It affects your score soooo much.

Good luck!