Cookies help us deliver our services. By using our services, you agree to our use of cookies. Learn more

crowdAI is shutting down - please read our blog post for more information

Mapping Challenge

Building Missing Maps with Machine Learning


solution journal [0.821]

Posted by neptune.ml about 3 years ago

Hi there,

We took a closer look at a few things and managed to improve the score significantly:

  • We’ve analysed the competition metric which suggested that we should add scoring (averaged probability of a building area)
  • We took a look at the results and found out that small images are the problem. Quick fix was to add area based weighing of per-building scores.
  • We started experimenting with using VGG and Resnet as the encoder part of the unet network
  • We added erosion-based pre-processing and per-buidling dilation postprocessing
  • We added Conditional Random Fields postprocessing to smoothen up the predictions
  • We added distance-to-edge-weighted loss (still training)

There are still a lot of problems with small objects. To tackle that we are implementing the idea of weighing pixel loss with object size, where smaller buldings get higher weight. Also there is a lot of experimenting left to be done as we implemented a bunch of ideas but due to hardware problems couldn’t see them through. As of now our results are based on early checkpoints from those experiments (obviously we are hoping for significant improvements when they finish training).

Everything that we do is as always available for you to use/change/comment in our open-solution repo https://github.com/minerva-ml/open-solution-mapping-challenge . Newest ideas are usually on the dev branch so make sure to check that out too.

Best, and good luck!


Posted by EOS_Data_Analytics  about 3 years ago |  Quote

@minerva.ml I can’t find where do you add weighting for scores, can you pls help me?

Posted by neptune.ml  about 3 years ago |  Quote

Sure, first you have it defined via Step and Transformer right here: https://github.com/minerva-ml/open-solution-mapping-challenge/blob/master/pipelines.py#L240-L247

But the actual function that calculates it is here:


It takes prediction maskas from unet and labeled (1-k) masks after instance assignment and calculates mean over probability in that region but weighs it with size.