Loading
Cookies help us deliver our services. By using our services, you agree to our use of cookies. Learn more

crowdAI is shutting down - please read our blog post for more information

Mapping Challenge

Building Missing Maps with Machine Learning


Completed
719
Submissions
1070
Participants
44621
Views

Creating mosaic images from the entire dataset

Posted by neptune.ml over 1 year ago

Hi there,

As we are working on our solution we come to realize that for some images at the edges there apears to be a samll label but no building under it. That results from someone labeling a large image which is later cut into pieces. That means 2 things: 1. it is impossible to predict small buildings on the edges from the image crop itself 2. one could first glue all the crops into k large images and run predictions on those large images

I suspect that top2 contestants are doing just that.

My question to you and the organizers are Will this technique be punished in stage2 by randomly selecting only a small subset of crops from those k large images?

I understand the answer either way. Especially since in production you would actually have those large images and you would want to run inference on them rather than the crops. I just think it would be fair to know that outright.

Posted by spMohanty  over 1 year ago |  Quote

Hi @minerva.ml ,

Yes, labels like those are indeed present in the dataset, and as you guessed are from the nature of how the dataset is created.

Trying to glue the images together to stitch them and then make predictions might work, but I believe there will be substiantial effort that will go into that restitching which will not generalise to round-2.

In round-2 the predictions will have to be made on internal datasets from UNOSAT, where the images will not be released, and participant will instead be expected to package and submit their models. So if any of the top participants are overfitting to this dataset, their models will not generalise to the round-2 dataset.

Hope this answers your questions. Cheers, Mohanty

Posted by neptune.ml  over 1 year ago |  Quote

Thanks Mohanty,

In terms of effort I think there was a bunch of scripts for similar antics on kaggle so I don’t think it’s that difficult to do. Also the fact that the winner of stage1 will be awarded something may incentivise people to use this technique even still. Potentially as soon as the stage2 starts one would drop the part of the solution that did the mosaic glueing and work on the general solution (with the top1 stage 1 prize bonus).

Anyhow just wanted to point that out as I am thinking about those border building edges. We will work on the best solution that doesn’t use additional hacky tricks so that everyone could easily generalize our solution to their usecase.

Cheers, Jakub from minerva.ml

Posted by EOS_Data_Analytics  over 1 year ago |  Quote

@spMohanty the problem could be solved easily - if you adjust metric that does not take small objects into account

Posted by neptune.ml  over 1 year ago |  Quote

@ EOS_Data_Analytics I presume that you mean small objects at the edges? I strongly believe that dealing with small object in the center should definitely be a part of this challenge.

Posted by spMohanty  over 1 year ago |  Quote

@spMohanty the problem could be solved easily - if you adjust metric that does not take small objects into account

@EOS_Data_Analytics : At this point of the challenge, I would hesitate to make changes to the evaluation metric. Apart from that, detecting small objects is indeed a major part of the whole problem, so we will let the evaluation metric stay as it is for now.

The dataset for round-2 would be completely different, so only solutions which generalize well will perform better there.

Cheers, Mohanty