Humanoids 2013 Workshop




Benchmarking of State-of-the-Art algorithms in Generating Human-Like Robot Reaching Motions


Mohammad Khansari, Andre Lemme, Yaron Meirovitch,
Benjamin Schrauwen, Martin A. Giese, Auke. J. Ijspeert, Aude Billard and Jochen Steil


Humanoids 2013 Conference - Oct 15, 2013, Atlanta (USA)


Sept 5 : One-page abstract submission EXTENDED to Sept 9th
Sept 10 : Notification of acceptance
Sept 30 : Submission of final trained model for the Benchmark
Oct 5: Notification of Benchmark result
Oct 15 : Workshop day


Manual programming of robot motions often requires a large amount of engineering knowledge about both the task and the robot and it can become particularly non-intuitive when dealing with high Degrees of Freedom (DoF) humanoids. The advent of a new generation of humanoid robots that need to perform a wide variety of tasks in human daily lives stresses further more the importance of techniques to autonomously adapt to various situations and to be robust to various sources of perturbations/uncertainties. It is furthermore essential for social acceptance of humanoids to provide motion patterns that are more similar to human movements. In response to these concerns, many approaches have been introduced within the passed decade following different levels of modeling, from kinematic of the motion, biomechanic of the limb, planning and execution level, to neural modeling of cortical process.

This workshop aims at benchmarking the state-of-the-art learning algorithms that cover model of the kinematic of human reaching motions. It is timely as an increasing variety of different approaches in generating human-like robot motions in the field calls for standardized and systematic comparisons. This workshop provides the opportunity to understand and to compare methods through an open-source benchmark software framework that has been developed in the European project AMARSi ( This benchmark evaluates algorithms on a library of human motions based on various criteria such as "level of similarity to human motions", "accuracy in reaching the goal state", "adaptability to changes dynamic environments", "robustness to perturbations", etc. This workshop thereby allows participants to discuss strengths and drawbacks of existing approaches based on a common basis which is essential for researchers in the field.

Topics of interest:

- Reaching motions, point-to-point motions, discrete motions
- Human-like robot motions
- Standardized and systematic comparisons of different movement generation algorithms
- Dynamical systems, Movement Primitives, Motion Generations, Motion Planners
- Open-source benchmark software framework

How to contribute:

We enthusiastically invite interested researchers to participate at our workshop in order to provide the community with a standardized and systematic comparison between different state-of-the-art algorithms in motion generation. To participate, please send a one-page abstract, describing your approach by September 5 to Notification of acceptance to the benchmark will be given by September 10. All accepted participants will have access to the Benchmark Code and a library of 2D motions, including a guideline about how to use the benchmark.

The participants should then provide a (trained) model for each of the motions in the library by September 30. All trained model should be sent in a single zip file to or . The organizers will then run the benchmark for all the participants and announce the Benchmark result by October 5. The three approaches with highest score will give an oral presentation during the workshop on October 15. The remaining groups will also have the chance to present their work during the poster session.

For more information, please don't hesitate to contact the organizers. Please note that all the participants at the workshop must register for the conference. The deadline for the early registration has been set to September 15.

Benchmark Software:

The AMARSi Benchmark framework is a software package written in MATLAB that evaluates the performance of reaching motion generation methods against ten different metrics. All accepted participants will have access to the Benchmark Code and a library of 2D motions, including a guideline about how to use the benchmark. The software package and the howto manual will be sent to the participants on the 10th of September with the notification of acceptance. After the workshop the software will be published and available online on the AMARSi website.

To participate in the Benchmark, you do not need to provide us with your learning algorithm. You only need to send a piece of code (i.e. integration in a common interface class) that can be used at the execution level and also your (trained) model of the motions. Your code at the execution level should provide the next state given the current situation (i.e. current time, goal position, and current position and velocity). In case if it is necessary, you could create a content-obscured version of your code by using the MATLAB "pcode" function. More information about the default training dataset and how to prepare the your model is given here.

Tentative Program:


Welcome from the organizers,pointing out the scope of the workshop (5 minutes)


Invited speaker: Jochen Steil (30 minutes)


Invited speaker: Sylvain Calinon (30 minutes)

14:35-15:00: Introducing the open-source AMARSi benchmark software

Coffee Break (30 minutes).


Andre Lemme, Univesity of Bielefeld


Alexandros Paraschos, TUDarmstadt


Mohammad Khansari, EPFL


Sylvain Calinon, IIT



17:00-17:30 Discussion on strengths and drawbacks of the existing approaches, and closing remarks on the current benchmark scores and participant's feedback for further improving the benchmark framework (30 minutes).


Invited Speakers:

Prof. Jochen Steil

Coordinator of the FP7-EU project AMARSi
Managing director of the Research Institute for Cognition and Robotics

Dr. Sylvain Calinon

Team Leader at the Department of Advanced Robotics,
Italian Institute of Technology, Genova, Italy


Mohammad Khansari*, PhD
Ecole Polytechnique Federale de Lausanne (EPFL)

Andre Lemme*, PhD student
Bielefeld University

Yaron Meirovitch, PhD student
Weizmann Institute of Science

Benjamin Schrauwen, Prof.
Ghent University

Martin A. Giese, Prof.
University Clinic Tubingen

Auke. J. Ijspeert, Prof.
Ecole Polytechnique Federale de Lausanne (EPFL)

Aude Billard, Prof.
Ecole Polytechnique Federale de Lausanne (EPFL)

Jochen Steil, Prof.
Bielefeld University


Andre Lemme


Oct 15, 2013


Atlanta, USA


Scientific Community



Recent Publications