Multi-Agent Reinforcement Learning in Minecraft
Please consider citing the following paper, if you find this work useful : Diego Perez-Liebana, Katja Hofmann, Sharada Prasanna Mohanty, Noburu Kuno, Andre Kramer, Sam Devlin, Raluca D. Gaina: “The Multi-Agent Reinforcement Learning in MalmÖ (MARLÖ) Competition”, 2019, Challenges in Machine Learning (NIPS Workshop), 2018 http://arxiv.org/abs/1901.08129
We are accepting submissions for Round 2. Please feel free to make a submission using the instructions here.
Due to the delay of opening submission system, we extended the Entry Period to December 31, 2018. Please take a look at the part of Winner Selection for detail.
What is the Challenge?
Learning to Play: The Multi-Agent Reinforcement Learning in MalmO Competition (“Challenge”) is a new challenge that proposes research on Multi-Agent Reinforcement Learning using multiple games. Participants would create learning agents that will be able to play multiple 3D games as deﬁned in the MalmO platform. The aim of the competition is to encourage AI research on more general approaches via multi-player games. For this, the Challenge will consist of not one but several games, each one of them with several tasks of varying difficulty and settings. Some of these tasks will be public and participants will be able to train on them. Others, however, will be private, only used to determine the ﬁnal rankings of the competition.
Organizer (Microsoft, Queen Mary University of London and crowdAI) will make the Challenge tasks and sample code available via GitHub on or before the Challenge start date. Entries will be judged by Organizer according to the criteria outlined in the “Winner Selection” section below.
The following prizes will be awarded: (1) the top 7 team will be awarded a MARLO Travel Grant with a maximum value of
$2,500 USD to join a relevant academic conference or workshop, additionally, the 1st winning team will be awarded a second MARLO Travel Grant with a maximum value of
$2,500 USD to join the Applied Machine Learning Days 2019 and (2) three winning teams will be awarded Microsoft Azure Sponsorship with a maximum value of
$10,000 USD for the 1st place,
$5,000 USD for the 2nd place and
$3,000 USD for the 3rd place. Please see the “Challenge Prizes” section below for further details.
What are the start and end dates?
The Challenge starts at 00:01 Pacific Standard Time, on July 27, 2018, and the Qualify Round ends at 23:59 Pacific Standard Time, on December 31, 2018 (“Entry Period”). Entries must be received within the Entry Period to be eligible.
The Kick-off Tournament is held in MARLO workshop of AIIDE’18, the 14th AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment held at the University of Alberta in Edmonton, AB, Canada on November 14, 2018.
The Final Tournament will be held offline in a week after deadline to decide winners.
Games and Tasks
One of the main features of this competition is that agents play in multiple games. Therefore, several tasks are proposed for this contest. For the purpose of this document and the competition itself, we deﬁne:
Game: each one of the diﬀerent scenarios in which agents play.
Task: each instance of a game. Tasks, within a game, may be diﬀerent to each other in terms of level layout, size, diﬃculty and other game-dependent settings.
Figure 1: Sketch of how games and tasks are organized in the Challenge.
As can be seen, tasks will be of public nature and accessible by the participants, while others are secret and will be used to evaluate the submitted entries at the end of the competition.
Tasks are distributed across sets: Figure 2: On the left: Build Battle, where players need to recreate a structure (in this case, the structure is shown on the ground). On the right: Pig Chase, where players collaborate to corner the pig.
To participate in the Challenge, you will first need to register as a user on crowdai.org and entry the Challenge. Then you start playing with the Challenge by simply cloning the starter-kit of the Challenge on GitHub. Your participation in the Challenge happens through a self-hosted GitLab instance of crowdAI. You need to create a “private” repository with your initial code (or simply code copied from the starter kit). You can make a submission by creating a new git tag and pushing the tag on your repository. crowdAI bot identifies your push tag, automatically clone your repository into a docker image and run an evaluation. After evaluation, the leaderboard is updated. You can find more detail about the actual submission process in the starter-kit.
Your code (and also the corresponding issues for every submission) will stay private during the Challenge. You are encouraged to add an opensource license of your choice. At the end of the Challenge, you will be provided a time period, within which you can decide to keep your code private and not making it publicly available. Else your repository will be made public by default. If the participant has not already added a License to the repository, a MIT license will be automatically added to the repository before making it public. Those who object to having their code made public, will be excluded from the automated process of making all the Challenge specific repositories public.
We are not responsible for entries that we do not receive for any reason, or for entries that we receive but are not decipherable for any reason. We will automatically disqualify any incomplete or illegible entries.
Round 1 – until October 21, 2018
- Warm up round. Submitted agent to work for a single task.
Round 2 – October 21, 2018 – December 31, 2018
- Submitted agent to work with fixed random agent as its opponent.
At the close of the Entry Period (on December 31, 2018), the Organizer will select 8-32 qualifying teams from eligible entries based upon the overall performance on the Competition tasks (i.e. game score) to invite them to the Final Tournament.
The decisions of the Organizer are final and binding. If we do not receive a sufficient number of entries meeting the entry requirements, we may, at our discretion, select fewer qualifying teams to the tournament.
Kick-off Tournament is organized in a form of live competition at the workshop in AIIDE 2018. A few Invited teams are broken into groups. Teams on each group play among themselves as the same format with Final Tournament as below.
The Final Tournament is organized offline in a week after the deadline of December 31, 2018. The 8-32 invited teams are broken into 8 groups. Teams on each group play among themselves (in which we here called a “league”) to determine a ranking, and the top 2 teams of these players progresses to the next round. Each league (P players in a group) is played across the same N games, with T repetitions played per game. Each game has its own leaderboard of agents ranked by the quality of the players. These leaderboards award ranking points to the entries, following a Formula 1-like scheme: 25 points for the 1st ranked entry, 18 for the 2nd, 15, 12, 10, 8, 6, 4, 2 and 1 for positions in the ranking from 3rd to 10th respectively. No points are awarded for positions 11th and below. The winner of the league is determined by adding all ranking points obtained in the diﬀerent games of the challenge. Figure 3: MARLO Final Tournament
If any team member is a potential winner and is 18 years of age or older, but is considered a minor in their place of legal residence, we may require that person’s parent or legal guardian to sign all required forms on their behalf. If that person does not complete the required forms as instructed and/or return the required forms within the time period listed on the winner notification message, we may disqualify the relevant team and select a runner-up.
If your team is confirmed as a winner of the Challenge:
It may not designate another party as the winner. If your team is unable or unwilling to accept its prize, we may award it to a runner-up; and
Team members will be solely responsible for any taxes that may be payable in connection with the award of any prize.
MARLO Travel Grant Prize, Microsoft Azure Sponsorship Prize
To be eligible to win the MARLO Travel Grant prize or Microsoft Azure Sponsorship Prize, your team must meet the following criteria at the time of entry and at the time of award of any prize:
Each team member must be enrolled in undergraduate or post-graduate studies (that is, PhD or Masters level) at an accredited college or university and the team lead must be registered in a PhD program at an accredited college or university. Faculty or post-docs can be included in your team as a supervisor. But the supervisor is not eligible for the prizes.
A team consists of up to five participants. To participate in the Challenge and be eligible to win either prize, each team member must meet the following criteria at the time of entry and at the time of award of any prize:
18 years of age or older. If any team member is 18 years of age or older, but is considered a minor in their place of residence, they should ask their parent’s or legal guardian’s permission prior to submitting an entry into the Challenge; and
Not an employee of Microsoft Corporation or of any Microsoft subsidiary; and
Not an immediate family (parent, sibling, spouse/domestic partner, child) or household member of a Microsoft employee, an employee of a Microsoft subsidiary, or any person involved in any part of the administration and execution of this promotion.
Subject to meeting the relevant eligibility criteria, teams are permitted to enter to the prizes. Your team lead must notify Organizer before the end of the Entry Period (Dec 31, 2018) by sending an email to MalmoAdm@microsoft.com, indicating your crowdAI team name and all every individual team members. Only emails actually received by Organizer before the end of the Entry Period will be considered.
This promotion is void where prohibited by law. No purchase necessary.
How will my team’s entry be potentially used?
Except as set out below, we are not claiming any ownership rights to your team’s entry. However, by submitting your entry, each team member: 1. Grants Organizer an irrevocable, non-exclusive, royalty-free, worldwide right and license to: (i) use, review, assess, test, and otherwise analyze your entry and all its content in connection with the Challenge; and (ii) feature your entry and all content in connection with the marketing, sale, or promotion of the Challenge (including but not limited to internal and external sales meetings, conference presentations, tradeshows, and screen shots of the Challenge entry in press releases) in all media (now known or later developed); and 2. agrees to sign any necessary documentation that may be required for us and our designees to make use of the rights you granted above; and 3. understands and acknowledges that Organizer may have developed or commissioned materials similar or identical to your team’s submission and you waive any claims you may have resulting from any similarities to your entry; and 4. understands that we cannot control the incoming information you will disclose to our representatives in the course of entering, or what our representatives will remember about your entry. You also understand that we will not restrict work assignments of representatives who have had access to your entry. By entering the Challenge, you agree that use of information in our representatives’ unaided memories in the development or deployment of our products or services does not create liability for us under this agreement or copyright or trade secret law; and 5. understands that you will not receive any compensation or credit for use of your entry, other than what is described in these Challenge Rules.
If your team does not want to grant us these rights to your entry, please do not enter the Challenge.
What if something unexpected happens and the Challenge can’t run as planned?
If someone cheats, or a virus, bug, bot, catastrophic event, or any other unforeseen or unexpected event that cannot be reasonably anticipated or controlled (also referred to as force majeure), affects the fairness and/or integrity of the Challenge, we reserve the right to cancel, change, or suspend the Challenge. This right is reserved whether the event is due to human or technical error. If a solution cannot be found to restore the integrity of the Challenge, we reserve the right to select winning teams from among all eligible entries received before we had to cancel, change, or suspend the Challenge. If any team member attempts to, or we have strong reason to believe that a team member has, compromised the integrity or the legitimate operation of the Challenge by cheating, or through any other illegal, unlawful, or unfair activity, we may seek damages from that team member to the fullest extent permitted by law. Further, we may disqualify the team, and ban any team member from participating in any of our future Challenges, so please play fairly.
MARLO Travel Grant Prize
The top 7 team (3 Progress Award at Kick-off Tournament and the rest of 4 Award at Final Tournament) will be awarded a MARLO Travel Grant with a maximum value of
$2,500 USD for the team members to join a relevant conference to publish their competition result. The conference is within a year after finishing the final tournament and it is decided later based on mutual agreement between Organizer and the winning team. Additionally, the 1st winning team will be awarded a second MARLO Travel Grant with a maximum value of
$2,500 USD to join the Applied Machine Learning Days 2019. The winner of Progress Award at Kick-off Tournament and the winner of Award at the Final Tournament can be overlapped as long as the Organizer identify some improvements about the submission made by the relevant teams. Organizer will reimburse the following reasonably and necessarily incurred travel expenses of team members in attending the conference: (1) economy class roundtrip airfares from your nearest airport to the city of conference; and (2) accommodation up to 3 days. Organizer will not cover meals during your attendance in the conference.
Half of these travel grants are kindly sponsored by enable, an EPFL Technology Transfer Office program aimed at facilitating the path towards commercialization of EPFL innovative technologies and stimulation innovation.
Microsoft Azure Sponsorship Prize
Three teams will each win a Microsoft Azure Sponsorship at Final Tournament with a maximum value of
$10,000 USD for the 1st place,
$5,000 USD for the 2nd place and
$3,000 USD for the 3rd place. The Microsoft Azure Sponsorship will be awarded to the team lead nominated by the relevant team. Team leads will be solely responsible for allocation of the grant between the relevant team members. The award of a Microsoft Azure Sponsorship will be subject to each team member agreeing to comply with such terms and conditions of use or other requirements that Microsoft may impose.
Azure for Students
Azure for Students gets you started with $100 in Azure credits to be used within the first 12 months plus select free services (subject to change) without requiring a credit card at sign-up. This is not mandatory for the Challenge. We recommend you take a look at the portal site if you would get additional computing resource: https://azure.microsoft.com/en-us/free/students/.
Winner’s List and Sponsor
We will notify winning teams by November 17, 2018. We will also post a leaderboard detailing the top 32 team entries online on crowdAI and under Microsoft Research website. Please note that we will only post team names and we will not post the names of any individual team members. This list will remain posted for a period of at least 12 calendar months.
If your team’s entry is in a public repository we may also post a link to that repository. Please check your GitLab and GitHub settings to ensure that this will not result in individual team members becoming identifiable, unless this is intended by the individual(s) in question.
This promotion is sponsored by Microsoft Corporation, One Microsoft Way, Redmond, WA 98052-6399, USA.
- July 27th, 2018: Competition Open
- October 21th, 2018 : Qualifying Round 1 submission deadline
- November 14th, 2018: Kick-off Tournament
- December 31th, 2018 : Qualifying Round 2 submission deadline
- Early January, 2019: Final Tournament
The organizing team comes from multiple groups — Queen Mary University of London, École Polytechnique Fédérale de Lausanne and Microsoft Research.
The organizing team consists of:
- Diego Perez-Liebana (Queen Mary University of London)
- Raluca D. Gaina (Queen Mary University of London)
- Daniel Ionita (Queen Mary University of London)
- Sharada Prasanna Mohanty (École polytechnique fédérale de Lausanne)
- Sam Devlin (Microsoft Research)
- Andre Kramer (Microsoft Research)
- Sean Kuno (Microsoft Research)
- Katja Hofmann (Microsoft Research)
- enable (EPFL)
Please visit the starter-kit page first for our comprehensive instruction to start this competition.
- Starter-kit: https://github.com/crowdAI/marLo
The original resource of Project Malmo platform is available on the GitHub.
- GitHub: https://github.com/Microsoft/malmo
- Project Malmo website: https://www.microsoft.com/en-us/research/project/project-malmo/
Follow twitter for the latest information of Project Malmo
- Twitter: @Project_Malmo
- Gitter Channel : https://gitter.im/Microsoft/malmo
- Discussion Forum : https://www.crowdai.org/challenges/marlo-2018/topics
We strongly encourage you to use the public channels mentioned above for communications between the participants and the organizers. In extreme cases, if there are any queries or comments that you would like to make using a private communication channel, then you can send us an email at: