Cookies help us deliver our services. By using our services, you agree to our use of cookies. Learn more

NIPS 2017: Learning to Run

Reinforcement learning environments with musculoskeletal models


I just want to confirm my understanding of the setting: Does the excitation of the muscles apply (angular) acceleration to the joints and body parts?

Posted by Ultron over 1 year ago

I am not sure whether this is considered a fully observed environment or not, since we only have the velocity and angular velocity.


Posted by Lukasz_  over 1 year ago |  Quote

Excitation turn the muscles on and off, but not instantaneously – you can find a brief explanation of these relations here https://simtk-confluence.stanford.edu:8443/display/OpenSim/First-Order+Activation+Dynamics Partially or fully activated muscle ‘pulls’ body parts and generates angular accelerations. Indeed accelerations are not in the observation vector, but in theory, you could model them as a function of excitations, since the mathematics of the biomechanical model are fixed. So it is fully observed. Formally, the only unknowns are positions of obstacles, so it’s not fully observed in this sense (yet, we give partial information about the next obstacle).

Posted by Yongjin  over 1 year ago |  Quote

Can we use a sequence of observations as an input to a DNN rather than a single observation at a time?

And, can we compute an acceleration using raw observations and use it as a feature or an input?

Posted by spMohanty  over 1 year ago |  Quote

Hi @Yongjin,

Yeah you can maintain a buffer of previous observations and use them all to make the prediction for the next action, if that gives you a better performance.

Cheers, Mohanty