Loading
Cookies help us deliver our services. By using our services, you agree to our use of cookies. Learn more

crowdAI is shutting down - please read our blog post for more information

Train Schedule Optimisation Challenge

Optimizing train schedules


Completed
209
Submissions
444
Participants
29677
Views

URGENT NOTICE BY ADMINS: Bugs in Validator. Please do not submit via the Challenge homepage for the time being

Posted by jordiju about 1 year ago

Hi All

Unfortunately it has been pointed out that the validator still does not catch all errors. Specifically, it can happen that it does not detect a resource occupation conflict in a solution.

A fix is in the works, but can unfortunately not be deployed over the weekend.

After the fix is deployed, we will manually re-check all submissions appearing on the leaderboard to make sure they still comply with all rules.

Until the fix is deployed,

  • please do not upload any submissions via the challenge page
  • you may keep using the validation scripts / notebooks. However, be advised that the validator may miss resource occupation conflicts in certain cases. Currently, a validation feedback with zero errors is no guarantee that the solution indeed complies with all business rules.

Sorry for the inconvenience caused. We will keep you updated.

Posted by jordiju  about 1 year ago |  Quote

Update: Issue resolved.

There was indeed a bug in the validator which should now be fixed. You are free to submit solutions again.

In the coming days, we will re-check the submissions on top of the leaderboard to make sure all submissions comply with the rules.

Posted by ms123  about 1 year ago |  Quote

Can you give an indication, what the problem in the validator was? E.g., was it some “systematic” error in reading the data or so? Because since your update this afternoon, solutions which were valid in the previous validator now are infeasible for me due to resource conflicts, so I probably miss the same thing that was wrong in your previous validator in the moment.

Posted by jordiju  about 1 year ago |  Quote

Can you give an indication, what the problem in the validator was? E.g., was it some “systematic” error in reading the data or so? Because since your update this afternoon, solutions which were valid in the previous validator now are infeasible for me due to resource conflicts, so I probably miss the same thing that was wrong in your previous validator in the moment.

It was a bug in how it was decided whether any two train_run_sections should be compared to each other to see if they cause resource occupation conflicts. Remember, for example, that two train_run_sections from the same service intention are allowed to use a resource without respecting the release_time of the resource. Therefore, the validator does not check a pair of train_run_sections from the same service intention. Basically what happened is that it also did not check certain pairs of train_run_sections from distinct trains. If this pair happened to cause a resource occupation conflict, we did not detect that. So in this sense, it was a systematic problem, although the effect was indeed a little random.

The effect that certain solutions that were previously valid are now rejected is expected. This is why we will also re-evaluate all leading submission on the leaderboard. The detailed validation feedback should tell you precisely which trains on which sections cause the conflict on what resource. I guess your first step would be to take a few examples of the feedback and confirm whether the solution really violates the rules and then try to determine how that could happen in your solver/model.