Cookies help us deliver our services. By using our services, you agree to our use of cookies. Learn more

crowdAI is shutting down - please read our blog post for more information

NeurIPS 2018: AI for Prosthetics Challenge

Reinforcement learning with musculoskeletal models


Unstable actions upon new environment

Posted by SArnold93 about 2 years ago

Hi All, This might be an inherent flaw in my model, however I am finding that when I load the model into a new environment (including the server environment) my agent become unstable and takes borderline useless actions for the first few iterations. Whilst training my model is able to take one full step and then promptly faceplants (hurray!), but in testing simply raises one leg and falls over backwards (not ideal)? Leading to me getting a far lower score with the server than I am on my local computer.

Has anyone else has had this issue? If so how did you resolve it? Any help you can provide would be greatly appreciated.

Posted by Jolly Roger  about 2 years ago |  Quote

Check if observations in the server environment correspond to those of your local one. For me the order of keys in dictionaries was different. For example, if you do some preprocessing, you should better iterate over explicit list of keys (for body_part in [“head”, “torso”, …]) rather than (for body_part in dict.keys()).


Posted by SArnold93  about 2 years ago |  Quote

Ah yeah that sounds like it could do the trick, thank you for the advice i’ll give it a go. Don’t worry i’m far away from achieving anything as functional as yours haha