Cookies help us deliver our services. By using our services, you agree to our use of cookies. Learn more

crowdAI is shutting down - please read our blog post for more information

IEEE Investment Ranking Challenge

Explore methodology to identify assets with extreme positive or negative returns.


Notice: Error in NDCG calculation in evaluation script

Posted by benharlander over 2 years ago

See comment section for details.

Posted by benharlander  over 2 years ago |  Quote

Hi everyone,

Yesterday, it was brought to our attention that there was a small, but important error in the evaluation script and random forest starter kit. (Thanks @cyga for the tip!) The calculation for NDCG was using a discount factor of $\frac{n}{n-i}$ rather than the correct $\frac{n}{n+i}$. See correct formulation for NDCG on the “Overview” tab.

This error would have affected those that have already submitted solutions through the crowdAI API.

Apologies for this inconvenience. Best of luck on the challenge!

Ben Harlander

Data Science & Operations Research Principal Global Investors