crowdAI is shutting down - please read our blog post for more information
Explore methodology to identify assets with extreme positive or negative returns.
By Principal Financial Group
about 2 years ago
Round 2 is open to all challenge participants. Round 1 focused on prototyping models that maximized statistical measures and Round 2 will enhance this with a deeper dive into your methodology and a new set of holdout data from 2017. To compete, all participants must submit the following items:
Final predictions for all time periods
A brief written solution using the IEEE template for conference proceedings. MS Word and LaTeX are both acceptable. At a minimum, the document should include an introduction, description of your methodology, results, and any other information needed to understand your solution and its merit. Tables, charts, and other visuals are highly encouraged.
All code and files needed to reproduce your results uploaded to Gitlab. (More details on this soon)
The top 6 solutions will be selected based on their statistical performance, calculated in the following manner:
Final score = (A+B+C+D)/4
A = Rank of spearman correlation on holdout data from 2002 – 2016
B = Rank of NDCG score on holdout data from 2002 – 2016
C = Rank of spearman correlation on holdout data from 2017
D = Rank of NDCG score on holdout data from 2017
All ranks will be determined using 3 significant digits. Performance on the data from 2017 will be used as a tiebreaker if needed. Participants will still have access to the crowdAI API to test their submissions on the original dataset, but no feedback on the holdout from 2017 will be provided until the challenge closes.
about 2 years ago |
So according to the above there will be no penalty for overfitting the validation data, in order to boost A and B?
about 2 years ago |
Can you answer a question about overfitting on 2002-2016 data? As far as I can see at the moment the winning strategy in not to create robust model, but to overfit on 2002-2016 data and make neutral submission for 2017 data. That’s apparently will maximize scores, but would have poor generalization ability.