Cookies help us deliver our services. By using our services, you agree to our use of cookies. Learn more

crowdAI is shutting down - please read our blog post for more information

Mapping Challenge

Building Missing Maps with Machine Learning


External datasets

Posted by taraspiotr over 3 years ago

Hi! I have some difficulties understanding the following rule: “Participants are not allowed to use any other datasets other than the ones released in context of this challenge. The use of pre-trained models is nevertheless permitted.” Does it mean that for round 2 we have to use only the dataset from Round 1? Pre-trained model can obviously be trained on other datasets, right? That’s kind of what it means that they are pre-trained. Could you provide a definition of pre-trained model? Because the distinction between using other datasets and pre-trained models seems very blurry to me.



Posted by spMohanty  over 3 years ago |  Quote

@taraspiotr : Thats a good point. Publicly available pretrained models are allowed to be used for this challenge. Any other exterrnal datasets (even if they are publicly allowed) are not allowed in this challenge.

And Round-2 will test the generalisability of your models. Hence the submitted models will be tested against a completely different dataset of refugee camps and tents which has never been publicly released. The overall scores will be definitely be much lower than in round-2, because the dataset for round-2 is completely different than dataset for round-1, but nevertheless the level of difficulty will be consistently same for all the participants; and hence the scores will still be comparable among all the round-2 submissions.

Hope that answers your questions.

Posted by apyskir  over 3 years ago |  Quote

@spMohanty So, according to your post, I can pretrain my model on external dataset, then share the pretrained model with weights publicly, and everything will be OK, right?

Thank you for sharing a little bit of information about Round 2! I have a question regarding this: do you count tents as buildings, or not? That would be very valuable information. What about the rules and instructions about Round 2? Will you post it soon? The LB for Round 2 will be calculated the same way as for Round 1?