The importance of Open Source and Open Data
29 April 2018
Did you know that crowdAI is an open source project? Not only the front-end application, which has been built in Ruby on Rails but also the backend which is a combination of GitLab Community Edition and our advanced grading infrastructure, developed mostly in Python.
crowdAI has rapidly grown from it’s infancy, about 2 years ago, out of the Digital Epidemiology Lab into it’s present form where we are running concurrent large-scale challenges with thousands of submissions.
We believe in pushing the envelope at crowdAI, we really do follow the often touted maxim of “move fast and break things”, but then we (usually) fix them equally fast. Our newest challenges will accept live code submissions, build them into Docker containers and even pit submitted agents against each other in out adversarial and reinforcement learning challenges, such as the Adversarial Vision Challenge, which is on of two NIPS 2018 challenges being hosts on crowdAI. Or consider the VizDOOM Adversarial Shootout where only the last agent will remaining standing in a fight to the death in a hosted DOOM world.
We are a small team, with limited funding. How do we do it? Well, lots of hard work of course, but we couldn’t do it without the many layers of Open Source software we build upon. We are building on the previous efforts of countless open source volunteers.
You may have heard that Machine Learning is the new electricity, and the major frameworks have all been either built as collaborative open source efforts, or have been open sourced by tech companies. But the electricity cannot be generated without fuel, and that fuel is data.
crowdAI strives to make open data a reality, the data for all of our previous challenges is kept online, increasingly with the code and configuration developed by our participants to transform and elicit meaning and knowledge from that data. We are moving to make it even easier to not just review a past challenge’s submissions, but to click and see code, models, source data and then fork it all to run on your own.
Want to help? You can! We encourage contributions from the open source communities, and we appreciate them all, no matter how large or small.
Our code is housed in our GitHub repos. Take a look for “Help Wanted” tags, open issues or even just code that needs some work.
Not a Data Science expert? The front end is a web app.
Not a developer? Consider adding comments to issues, defining feature requests or even draft documentation.
Recent contributors include: