Loading
Cookies help us deliver our services. By using our services, you agree to our use of cookies. Learn more

crowdAI is shutting down - please read our blog post for more information

WWW 2018 Challenge: Learning to Recognize Musical Genre

Learning to Recognize Musical Genre from Audio on the Web


Completed
728
Submissions
409
Participants
20189
Views

Specifications for Round 2

Posted by Benjamin_Murauer over 1 year ago

Hello, i wanted to ask if you can provide more detailed specifications about the second round of the challenge, in particular to answer the following questions:

  • What resources will the docker image be supplied with? E.g., can I use a GPU in my model?
  • Is there a time restriction which is enforced when running the docker image?
  • Does the model of round 2 have to be the same as the one used in round 1?

These questions could influence the choice of model that I work on, so it would be very helpful if they can be answered early on. Thank you for your time

1

Posted by Gillesvdw  over 1 year ago |  Quote

Good call! Would be nice to get a quick answer…

Posted by Michaël_Defferrard  over 1 year ago |  Quote

Hello, these are very relevant questions.

I’ve added the list of resources in the rules section. Let me know if you see something missing.

The model can be different. But you won’t be able to modify it after submission! The goal is to prevent overfitting on the test data, as can happen now on round 1.

Thanks for asking and participating!

Posted by Gillesvdw  over 1 year ago |  Quote

Hello Michaël,

That amount of resources definitely suffices!

I also have a question concerning the evaluation metrics. Is it true that the macro F1-score only counts when there are equal logloss scores? Seems rather weird since logloss is quite granular, and chances on ex aequos are small…

Posted by spMohanty  over 1 year ago |  Quote

Hi @Gillesvdw ,

True, the principal evaluation metric is indeed logloss. The high degree of precision in the logloss score also makes it very unlikely that the secondary score (F1-score) will ever be used. The reason we have F1 score as a secondary score is just to have a more easily interpretable score for every submission associated with every submission.

Cheers, Mohanty

Posted by Gillesvdw  over 1 year ago |  Quote

Hi @Gillesvdw ,

True, the principal evaluation metric is indeed logloss. The high degree of precision in the logloss score also makes it very unlikely that the secondary score (F1-score) will ever be used. The reason we have F1 score as a secondary score is just to have a more easily interpretable score for every submission associated with every submission.

Cheers, Mohanty

Thank you!