
-
NIPS 2017: Learning to Run
Reinforcement learning environments with musculoskeletal models
Challenge ended
2154
Submissions630
Participants92.3 k
Views -
Learning How to Walk
Reinforcement learning environments with musculoskeletal models
Challenge ended
200
Submissions125
Participants18.1 k
Views
Zxc_596 hasn't authored any tutorials yet...
Learning How to Walk | Visualization | over 2 years ago
Hi,
I noticed that the geometry (at least for visualization) seems to be incorrect. I am not sure if this affects the calculation or not. What I noticed is that the geometry for left and right femur are the same:
https://github.com/stanfordnmbl/osim-rl/blob/master/osim/models/Geometry/femur.vtp https://github.com/stanfordnmbl/osim-rl/blob/master/osim/models/Geometry/l_femur.vtp
This can be seen in the attached picture
I assume that this could be fixed by applying a transformation around here or the corresponding part for the other femur…
https://github.com/stanfordnmbl/osim-rl/blob/master/osim/models/gait9dof18musc.osim#L253-L268
Another issue I recognized is that somehow, the pelvis position in the observation vector is not in X, Y, Z order but it seems Z, X, Y. Still, the supposed Z position/speed values are quite odd and I am wondering what they really are…
These plus other issues I already described make me doubt the validity model… Could you provide a little more information about this? I would be happy to help and provide fixes but I am yet not sure if I am missing something.
Thanks!
Learning How to Walk | Observations and rewards | over 2 years ago
Hi! I am trying to understand then observation vector and I found a few strange issues. Could you clarify if there is a misunderstanding on my side?
1) If my interpretation of the observation vector is correct, the rewards seems to be calculated on the y delta: I am assuming that positions are in the format [X,Y,Z]:
The pelvis position is set here: https://github.com/stanfordnmbl/osim-rl/blob/master/osim/env/human.py#L76-L78 https://github.com/stanfordnmbl/osim-rl/blob/master/osim/env/human.py#L116-116
And the reward is calculated here (using Y coordinate): https://github.com/stanfordnmbl/osim-rl/blob/master/osim/env/human.py#L28-L29
Is this a bug or where is my mistake?
2) pelvis X speed seem to be the opposite to what I expected. Positive X speed means that the skeleton is moving backwards…
3) left and right (knee, hip, ankle) seem to be swapped at least with respect to what is represented in the visualization. I am using the code in getFootL as a reference. Unless I made mistake, when the right foot is high, I get a high value in the left one.
https://github.com/stanfordnmbl/osim-rl/blob/master/osim/env/human.py#L18-L20
Thanks a lot for putting this together! It is a lot of fun!
Apparently Zxc_596 prefers to keep an air of mystery about them...
Learning How to Walk | Thank you everyone :) #Results | over 2 years ago
Thank you for organizing this! And congrats to syllogismos on your great results!
As an opportunity to learn more: It would be nice to hear a little more about how/if you modified the reward function and what approach/algorithm you used. (rllab’s DDPG or something else?)
Looking forward to the next challenge!