Code & Data

Space Cannons: a reinforcement learning testbed for collaborative behaviour and training by human demonstration

Space Cannons is a two-player shooting game designed as a reinforcement learning test bed. Space Canons has two crucial features: (i) it can provide separate scores to agents when they cooperate to score points, depending on the degree of cooperation displayed by the agents. This is done by counting the number of hits by each enemy and then creating a coop-factor to decide if the hit was cooperative or not; (ii) Space Cannons is designed to also collect demonstration data coming from human experts. This data can then again be used in the training process of any leaning algorithm.

Space Cannons was designed by Mehul Verma as his AI MSc thesis at the Vrije Universiteit under supervision of Erman Acar.

It was published in the paper “Learning to Cooperate with Human Evaluative Feedback and Demonstrations. Mehul Verma and Erman Acar, Hybrid Human-Artificial Intelligence Conference (HHAI) 2022, IOS Press (to appear),

The GitHub repo is at

Documentation for Space Cannons is at

Axies: A Collaborative Platform for Identifying Context-Specific Values

Value alignment is a crucial aspect of ethical multiagent systems. An important step toward value alignment is identifying values specific to an application context. However, identifying context specific values is complex and cognitively demanding

The Axies platform simplifies the complex value identification task as a guided value annotation task. Our platform successfully supported the experiments involving two contexts and two groups of annotators by providing an intuitive design that allows the annotators to visualize all components.

Axies has two key features: (1) it requires collaborative work among human annotators, who perform several high-level cognitive tasks, and (2) it exploits natural language processing (NLP) and active learning techniques to guide annotation. The interface can be used on small (e.g., smart phone) and large screens

Axies has been published in “A Collaborative Platform for Identifying Context-Specific Values“, Enrico Liscio, Michiel van der Meer, Catholijn Jonker, Pradeep Murukannaiah,, AAMAS 2021, pgs. 1773–1775, as well as a longer journal version.
There’s also a short paper at the HHAI 2022 conference and a poster on Axies.

See for a demonstration.

Axies is available on Github at and in the TU Delft code repository.