Loading
Cookies help us deliver our services. By using our services, you agree to our use of cookies. Learn more

crowdAI is shutting down - please read our blog post for more information

Ants Challenge - Part I

Ants as tools to understand society


Completed
26
Submissions
57
Participants
7460
Views

L1 Loss Function

Posted by lesnikow about 3 years ago

Hi, could someone explain what is meant by the < gamma part of the denominator in the L1 loss function? Is the part in parens just the max( MSE, gamma)?

Also what is the intuition for the normalizing denominator factor?

I’m trying to write a quick evaluation script so I can measure the L1 loss on my own validation set, thanks!

Posted by spMohanty  about 3 years ago |  Quote

Hi @lesnikow,

The gamma variable refers to a threshold……if the euclidean distamce between your prediction and the actual point (in the answer file) is less than this threshold gamma, then it is considered as a “correct prediction”. And the said loss function basically gives you the percentage of “correct predictions” (as we just defined correct predictions based on the threshold gamma) from among all ant_id::frame_id pairs in your submission.

Also if you are curious about the Beta variable, then it is simply used to exclude all data points(ant_id::frame_id pairs) that are known to be outside of the frame according to the reference answers file.

Hope this clarifies your doubt.

Posted by lesnikow  about 3 years ago |  Quote

Ok so the L1 is essentially the proportion of responses that are correct modulo the amount gamma. And L2 is the MSE, discarding true labels that aren’t in the test frame for both L1 and L2. That helps a lot, thanks spMohanty!

Posted by spMohanty  about 3 years ago |  Quote

@lesnikow Not sure if Modulo is the correct word (from a discrete mathematics POV), but I guess you get the idea !! :D

Cheers, Mohanty