Loading
Cookies help us deliver our services. By using our services, you agree to our use of cookies. Learn more

crowdAI is shutting down - please read our blog post for more information

NeurIPS 2018: AI for Prosthetics Challenge

Reinforcement learning with musculoskeletal models


Completed
4575
Submissions
477
Participants
68130
Views

Submission gets half score

Posted by qurock over 1 year ago

I don’t know whether it’s a bug or I’m doing something wrong, but the score I get from all submissions is exactly half the score I track locally summing the rewards I receive on the same episode.

Is anyone else having the same problem? Does anyone know why it happens?

4

Posted by ShawK91  over 1 year ago |  Quote

Same here

Posted by JonAMichaels  over 1 year ago |  Quote

The submission system does 2 simulations with your agent. So, if you’re getting half the score that implies you’re not letting crowdai run your agent twice, and it’s assigning a score of zero to your second run. Make sure that after you get the done signal you reset the client environment and run it until the state variable you’re getting back is empty.

Posted by spiglerg  over 1 year ago |  Quote

Solution: client.env_create automatically performs the first reset, so if you do env.reset() before the first test episode, you will skip the first submission episode. This solves the problem:

1) Instead of creating a Client object and then wrapping it with ClientToEnv (osim-rl-helper), simply use the new ClientToEnv object directly. The object creates the env on the first reset.

class ClientToEnv: def init(self, remote_base, crowdai_token): “”” Wrapper that reformats client environment to a local environment format, complete with observation_space, action_space, reset, step, submit, and time_limit. “”” self.client = Client(remote_base) self.reset_ = self.client.env_reset self.step = self.client.env_step self.submit = self.client.submit

	self.crowdai_token = crowdai_token
	self.time_limit = 1000
	self.action_space = gym.spaces.Box(low=0, high=1, shape=(19, ),
									   dtype=np.float32)
	self.first_reset = True

def reset(self):
	if self.first_reset:
		self.first_reset = False
		obs = self.client.env_create(self.crowdai_token, env_id='ProstheticsEnv')
		return obs
	else:
		obs = self.reset_()
		return obs

Posted by spiglerg  over 1 year ago |  Quote

Hmm the formatting is off.

	class ClientToEnv:
		def __init__(self, remote_base, crowdai_token):
			"""
			Wrapper that reformats client environment to a local environment format,
			complete with observation_space, action_space, reset, step, submit, and
			time_limit.
			"""
			self.client = Client(remote_base)
			self.reset_ = self.client.env_reset
			self.step  = self.client.env_step
			self.submit = self.client.submit

			self.crowdai_token = crowdai_token
			self.time_limit = 1000
			self.action_space = gym.spaces.Box(low=0, high=1, shape=(19, ),
											   dtype=np.float32)
			self.first_reset = True

		def reset(self):
			if self.first_reset:
				self.first_reset = False
				obs = self.client.env_create(self.crowdai_token, env_id='ProstheticsEnv')
				return obs
			else:
				obs = self.reset_()
				return obs

Posted by bvankeir  over 1 year ago |  Quote

If I’m not mistaking, the code below should be a env_reset, because env_create starts a new submission > def reset(self): if self.first_reset: … obs = self.client.env_reset() …

Solution: client.env_create automatically performs the first reset, so if you do env.reset() before the first test episode, you will skip the first submission episode. This solves the problem:

1) Instead of creating a Client object and then wrapping it with ClientToEnv (osim-rl-helper), simply use the new ClientToEnv object directly. The object creates the env on the first reset.

class ClientToEnv: def init(self, remote_base, crowdai_token): “”” Wrapper that reformats client environment to a local environment format, complete with observation_space, action_space, reset, step, submit, and time_limit. “”” self.client = Client(remote_base) self.reset_ = self.client.env_reset self.step = self.client.env_step self.submit = self.client.submit

	self.crowdai_token = crowdai_token
	self.time_limit = 1000
	self.action_space = gym.spaces.Box(low=0, high=1, shape=(19, ),
									   dtype=np.float32)
	self.first_reset = True

def reset(self):
	if self.first_reset:
		self.first_reset = False
		obs = self.client.env_create(self.crowdai_token, env_id='ProstheticsEnv')
		return obs
	else:
		obs = self.reset_()
		return obs

Posted by bvankeir  over 1 year ago |  Quote

hmm, my comment turned out something different than I intended to, but the baseline is that the code posted earlier should have a client.env_reset() instead of client.env_create() in the final if statement, as env_create breaks one submission to start a new one

Posted by cloudpig  over 1 year ago |  Quote

have same problems with you.