Methods for Objective Comparison of Results in Intelligent Robotics Research

Exploring Our Focus


As Robotics continues to rise and be recognized as a "Science", there is a growing necessity to emphasize the reproducibility of experiments, a feature that's currently challenging or sometimes entirely lacking. Benchmarks, offering a qualitative means for interpreting performance, are essential for evaluating the effectiveness of an approach against a standard basis. Our primary objective is to amplify the awareness within the robotics community about the paramount importance of creating research that is both reproducible and measurable. By doing so, we strive to assist the transition of robotics research to a rigorous science, where reproducibility and benchmarking become the standard means for evaluation.

Delving into Key Topics

This workshop is designed to encompass a wide range of topics, with special focus, but not limited, to the following areas:
• Benchmarking methods for robotics research
• Statistical significance of results
• Ontologies for intelligent system
• Robotic System Classification
• Randomized Experimental study
• Benchmarking Protocol and standardization
• Performance modeling of the relationship between a task and the environment where it is performed
• Performance Metrics for Response Robotics
• Success metrics in bio-inspired Robotics
• Benchmarking mobile robot planning
• Benchmarking mobile manipulator systems
• Benchmarking and modelling humanoid and human movements
• Epistemological issues