This MATLAB benchmark framework was developed to compare different methods for generating goal directed trajectories and extract their specificities, strengths and weaknesses. It allows each user to configure different perturbations which can occur during a movement execution and prepare their models for the given task before a baseline parameter set is used to create compareable results.
Publication:
Open-source benchmarking for learned reaching motion generation in robotics
Lemme A, Meirovitch Y, Khansari-Zadeh SM, Flash T, Billard A, Steil JJ (2015)
Paladyn, Journal of Behavioral Robotics 6(1): 30–41.
Special issue:
The special issue follows-up on the IEEE Humanoid workshop 2013 on "Benchmarking state-of-the-art algorithms in generating human-like robot reaching motions".
Coming soon.
Software download:
Version 1.0: download
Support and software updates: here
Humanoids 2013 workshop:
Software
Beta version: download
Abstracts:
Calinon IIT: download
Khansari EPFL: download
Lemme UniBi 1: download
Lemme UniBi 2: download
Paraschos TUD: download
First steps in using the Benchmark framework:
In preparation of using this framework, your movement generation algorithm needs to be wrapped in a common interface defined in the benchmark. The proposed interface is specified in a MATLAB class definition. This class is only for using the movement representation of the users model and does not provide functionality for learning from data. The idea is that the benchmark user has already MATLAB models which are trained with a dataset used in the benchmark. If that is the case only three steps are needed, and the benchmark framework can be used directly:
- Write a class definition, which inherits the interface class
- Initialize the new class with the learned model
- Store the wrapped model
Default dataset:
The demonstrations data for the handwriting motions were collected from pen input using a Tablet-PC. For each motion, the user was asked to draw 7 demonstrations of a desired pattern, by starting from different initial positions (but fairly close to each other) and ending to the same final point.
These demonstrations may intersect each other. In total a library of 30 human handwriting motions were collected, of which 26 each correspond to one single pattern, the remaining four motions each include more than one pattern (called Multi Models). For all handwriting motions (shapes), the target is by definition set at (0, 0). You can find detailed information here.
