Classification of Nighttime Images from the International Space Station
By crowdAI
Completed
Submissions
Participants
Views
Overview
The International Space Station (ISS) has taken millions of images of Earth. About 30% of those images were taken at night. These photograph are the highest-resolution night imagery available from orbit. For technical reasons, we don't precisely know what the camera was pointing at when taking the images. However, if we knew exactly what these images were showing (stars, cities, auroras, etc.), we could use this wealth of information to address issues such as night pollution, population movement, energy usage, etc.
Until recently, it has been thought that algorithms cannot distinguish between stars, cities, and other objects. This challenge tries to prove otherwise. Thousands of volunteers have hand-labeled tens of thousands of images as part of a citizen science project. The goal of this challenge is to build on this manually evaluated data set, and develop an image classification algorithm that can correctly identify whether an image shows stars, cities, or other objects.
The Iberian Peninsula at night, showing Spain and Portugal. Madrid is the bright spot just above the center. Credits: NASA
Evaluation
Submissions will be evaluated using a Multi Class Log Loss evaluation function, which are defined as :
Mean F1 score
The F1 score is computed separately for all classes by using:
- p refers to the precision
- r refers to the recall
- tp refers to the number of true positives,
- fp refers to the number of false positives
- fn refers to the number of false negatives
Then finally the mean of all the F1 scores across all the classes is used for come up with the combined mean F1 score.
Mean Log Loss
- N is the total number of examples in the test set
- M is the total number of class labels (7 for this challenge)
- y ij is a boolean value representing if the i-th instance in the test set belongs to the j-th label.
- p ij is the probability according to your submission that the i-th instance may belong to the j-th label.
- Ln is the natural logarithmic function.
All submissions will be evaluated on the test dataset in the docker containers referenced in the Resources section. The code archive will be uncompressed into the /darkskies path, and every code archive is expected to contain a main.sh script which takes path to a folder containing images as its first parameter. So to test your code submission, we will finally execute :
/darkskies/main.sh pathToFolderContainingTestImages
This is expected to output a CSV file containing the name of the file, and the associated probabilities for all the classes at the location :
/darkskies/classification.csv
Rules
- Participants are allowed a maximum of five submissions each 24 hours.
- crowdAI reserves the right to modify challenge rules as required.
Prizes
The author of the most highly ranked submission will be invited to the crowdAI winner's symposium at EPFL in Switzerland on January 30/31, 2017. The educational award is given to the participant with the either the most insightful submission posts, or the best tutorial - the recipient of this award will also be invited to the symposium (the crowdAI team will pick the recipient of this award). Expenses for travel and accommodation are covered by crowdAI.