Loading
Cookies help us deliver our services. By using our services, you agree to our use of cookies. Learn more

crowdAI is shutting down - please read our blog post for more information

IEEE Investment Ranking Challenge

Explore methodology to identify assets with extreme positive or negative returns.


Completed
541
Submissions
207
Participants
12217
Views

Which metric will be used for the final rankings ?

Posted by sshkhr over 1 year ago

Say I have two models :

  • Model 1: Spearman Correlation = 0.041 and NDCG = 0.037
  • Model 2: Spearman Correlation = 0.035 and NDCG = 0.041

Both are doing better on alternate metrics. I wanted to know how the final rankings will be determined. Is their a weighted average of sorts that will be taken ? Because right now the Public Leaderboard is being determined solely by Spearman correlation it seems.

It would be really helpful if organisers could clarify. Thanks

1

Posted by spMohanty  over 1 year ago |  Quote

Hi @sshkhr ,

True, right now its based solely on the Spearman correlation. If two submissions have the exact same spearman correlation, then NDCG should take over, but there is a very low probability that two submissions will have exactly the same spearman correlation. But in a few weeks, we will update the leaderboard to first sort all the submissions by Spearman Correlation (upto 3 significant digits), and when two submissions with the same spearman correlation (upto 3 signification digits) will be ranked based on NDCG.