crowdAI is shutting down - please read our blog post for more information
Reinforcement learning environments with musculoskeletal models
tomvw hasn't authored any tutorials yet...
NIPS 2017: Learning to Run |
Observation Vector | over 2 years ago
The first indices pelvis position appears is 1 and 2 (for x,y) and next it appears again on indices 24, 25.
For the observation vector I meant whether it would possible to include other information on the state of the model in OpenSim (for example muscle state or ground reaction force). I think this would probably mean some adaptations in the methods of the opensim-rl environment. So it is probably a bit out of the scope of this challenge :).
I am very excited to start playing around with this challenge. Everything works very smoothly!
I have a remark on the observation vector - I think it holds a duplicate value of the pelvis_x and pelvis_y position. I just want to check with you to be sure I have the correct version of everything.
I also have a suggestion/question: is it in scope, maybe later during the challenge, to adapt the observation vector?
I think it would be interesting to include some information on the muscles (length/force). Geyer has developed a very cool feedback controller for a model that has dynamics that are very comparable to what is implemented in OpenSim (paper: https://www.cs.cmu.edu/~hgeyer/Publications/Geyer&Herr-ReflexModel2Column.pdf). For me it shows that information on the muscle state might enable a much faster ‘good solution’ for the controllers we are trying to develop.
Cheers a enjoy the fun!
Learning How to Walk |
What is exactly considered "walking" | over 2 years ago
@ViktorF: The activation values are indeed clipped between 0 and 1 by the OSIM code. Also their relation to forces produced by the muscles is physiologically modeled in the OSIM code and is quite nonlinear. If you are interested, the following pages provide good explanation: http://simtk-confluence.stanford.edu:8080/display/OpenSim/Muscle+Model+Theory+and+Publications.
Learning How to Walk |
Hunt Crossley contact model crashes integration | over 2 years ago
I am trying to find out in the source code of simbody why this happens. I am not sure of the cause but, I think it can be prevented that the model comes into this problematic state by increasing the stiffness of the contact.
Just to give an idea how I am running the simulation: Below is the C++ code snippet that I use to do my forward integration(it is not truncated in 10ms intervals but it has a controller implemented). The “myPeriodicController” is an implementation of the abstract periodiceventhandler. It allows me to do one forward integration and adapt anything in the states (in our case the activations) every fixed “eventinterval” (10ms in our case).
I think it will be difficult to replicate exactly what I am having (since it will depend on the controller that I test, and in a lot of cases there is no problem). But you can reproduce the error by just running a forward simulation through the OpenSim GUI. If you open the model and drag it so that the pelvis_y coordinate is at its maximum (via the coordinate window) and next do a simulation by hitting the green button next to simulate, the model will fall down and sink a little bit in the ground. The integration will from now on proceed very slow (it will look like it does not continue). The cause is the sudden very high peak in contact force that tries to accelerate the system, that is overall governed by stiff differential equations.
If you don’t get it working immediately I can also send you .mot and .sto file with the forces acting on the system and the states of the system respectively.
If you have noticed that your forward integration is in some cases going extremely slow, this is the probable cause ;-).
// ******** SimBody**********//
Real eventinterval = 0.01;
MultibodySystem& system_ref = osimModel->updMultibodySystem();
MyPeriodicController myPeriodicController = new MyPeriodicController(eventinterval, osimModel, NN_structure);
// ******************** OpenSim***********************//
State& s = osimModel->initializeState();
initialTime = TimeVec; finalTime = TimeVec;
ForceReporter* reporter = new ForceReporter(osimModel);
Manager manager(*osimModel, integrator);
I am running simulations with the model you provided, but I’m working with it in a matlab-cpp framework since I had already a learning algorithm implemented in Matlab.
In some cases when I run a forward simulation the forward integration keeps running. I tried to investigate the cause of this and I found that something strange happens in the contact forces.
At a certain moment the contact force between ground and feet drops to zero and stays zero for some integration steps (at this moment there is clearly contact though). Because there is no force the foot sinks in the ground. A couple of integration steps further, the contact force peaks from zero to a huge value (because the foot sunk in the ground). This huge peak is probably the reason why the integration fails, but the cause for the force to have an interval where it is 0 is unknown to me.
Apparently tomvw prefers to keep an air of mystery about them...