Cookies help us deliver our services. By using our services, you agree to our use of cookies. Learn more

crowdAI is shutting down - please read our blog post for more information

IEEE Investment Ranking Challenge

Explore methodology to identify assets with extreme positive or negative returns.


Which metric will be used for the final rankings ?

Posted by sshkhr over 3 years ago

Say I have two models :

  • Model 1: Spearman Correlation = 0.041 and NDCG = 0.037
  • Model 2: Spearman Correlation = 0.035 and NDCG = 0.041

Both are doing better on alternate metrics. I wanted to know how the final rankings will be determined. Is their a weighted average of sorts that will be taken ? Because right now the Public Leaderboard is being determined solely by Spearman correlation it seems.

It would be really helpful if organisers could clarify. Thanks


Posted by spMohanty  over 3 years ago |  Quote

Hi @sshkhr ,

True, right now its based solely on the Spearman correlation. If two submissions have the exact same spearman correlation, then NDCG should take over, but there is a very low probability that two submissions will have exactly the same spearman correlation. But in a few weeks, we will update the leaderboard to first sort all the submissions by Spearman Correlation (upto 3 significant digits), and when two submissions with the same spearman correlation (upto 3 signification digits) will be ranked based on NDCG.