Reinforcement learning with musculoskeletal models
Update 11/01: Google Cloud Platform will generously sponsor also the second round of the challenge. Top 50 teams from the Round 1 will be awarded 400$ cloud credits!
Update 10/31: Instructions for submission for Round-2 are available at : https://github.com/crowdAI/nips2018-ai-for-prosthetics-round2-starter-kit
Update 08/27: We’ve updated osim-rl package to version 2.1. If you joined before 08/27/2018 please run in your conda environment:
pip install git+https://github.com/stanfordnmbl/osim-rl.git -U
Update 07/27: Google Cloud Platform will sponsor participants of the challenge. Top 400 teams with positive (>0) number of points will be awarded 250$ cloud credits!
Update 07/30: Watch our webinar to learn more about biomechanics, neuroscience and reinforcement learning!
Welcome to AI for Prosthetics challenge, one of the official challenges in the NIPS 2018 Competition Track. In this competition, you are tasked with developing a controller to enable a physiologically-based human model with a prosthetic leg to walk and run. You are provided with a human musculoskeletal model, a physics-based simulation environment OpenSim where you can synthesize physically and physiologically accurate motion, and datasets of normal gait kinematics. You are scored based on how well your agent adapts to the requested velocity vector changing in real time.
Follow the instructions on our github repo to get started!
Our objectives are to:
- bring Deep Reinforcement Learning to solve problems in medicine,
- promote open-source tools in RL research (the physics simulator OpenSim, the RL environment, and the competition platform are all open-source),
- encourage RL research in computationally complex environments, with stochasticity and highly-dimensional action spaces.
Visit our github repo to get started!
What’s new compared to NIPS 2017: Learning to run?
We took into account comments from the last challenge and there are several changes:
- You can use experimental data (to greatly speed up the learning process)
- We released the 3rd dimensions OpenSim model (the model can fall sideways)
- We added a prosthetic leg – the goal is to solve a medical challenge on modeling how walking will change after getting a prosthesis. Your work can speed up design, prototying, or tuning prosthetics!
You haven’t heard of NIPS 2017: Learning to run? Watch this video!
Your task is to build a function f which takes the current state observation (a dictionary describing the current state) and returns the muscle excitations action (19-dimensional vector) maximizing the total reward. The objective is to follow the requested velocity vector. The trial ends either if the pelvis of the model falls below 0.6 meters or if you reach 1000 iterations (corresponding to 10 seconds in the virtual environment).
The total reward is 9 * s - p * p where s is the number of steps before reaching one of the stop criteria and p is the absolute difference between horizonal velocity and 3. You can interpret it as a request to run at a constat speed of 3 meters per second.
In the second round the task is also to follow a requested velocity vector. However, in this round the vector will change in time and it will be a random process. We will provide the distribution of this process in mid-July.
- Round 1: 16.06.2018 - 28.10.2018
- Round 2 (test run): 1.10.2018 - 28.10.2018
- Round 2: 29.10.2018 - 6.11.2018
Top 50 teams from Round 1 qualify to Round 2
In order to avoid overfitting to the training environment, top participants will be asked to resubmit their solutions in the second round of the challenge. The final ranking will be based on results from the second round.
- Organizers reserve the right to modify challenge rules as required.
We will provide the full list of prizes in mid-July. Prizes confirmed for now include:
- Invitation to publish articles in the NIPS competition book.
- Invitation to the 3rd Applied Machine Learning Days at EPFL in Switzerland on January 26 - 29, 2019, with travel and accommodation covered.
- Invitation to give a research talk at Stanford, with travel and accommodation covered.
- Reimbursement of travel and accommodation at NIPS 2018
Please visit osim-rl project’s website for resources on biomechanics and reinforcement learning, solutions from the NIPS 2017 and other materials. Visit OpenSim website for materials on musculoskeletal simulations.
Here are some interesting blog posts written by participants:
- Understanding the Challenge
- Understanding the Action Space
- Understanding the Observation Space
- Understanding the Reward
Here are some helper libraries written by participants:
Here are some articles and blog posts written by participants about NIPS 2017: Learning to Run Challenge (can very helpful for this challenge as well):
Use one of the public channels:
- Gitter Channel : crowdAI/NIPS-Learning-To-Run-Challenge
- Technical issues : https://github.com/stanfordnmbl/osim-rl/issues
- Discussion Forum : https://www.crowdai.org/challenges/nips-2018-ai-for-prosthetics-challenge/topics
We strongly encourage you to use the public channels mentioned above for communications between the participants and the organisers. In extreme cases, if there are any queries or comments that you would like to make using a private communication channel, then you can send us an email at :