Towards Reproducibility and Objective Performance Evaluation in Robotics and AI

    *************************************

    Virtual Workshop - all time are in CEST

    *************************************

    Part I - Examples of Good Practices in Reproducibility and Benchmarking

    May 31st 2021:

  • Session 1: Introduction
  • Session 2: Examples of Good Practices
  • Session 3: Objective Comparison and Verification

    • Part II: Reproducible Research in Robotics and AI

    June 4th 2021:

  • Session 4: Replication and Benchmarking of Research Results (I)
  • Session 5: Contributed presentations and demos
  • Session 6: Stories and panel discussion
  • Session 7: Replication and Benchmarking of Research Results (II)
  • Session 8: Discussion
    • Closing remarks

***********************************************

May 31st - all time are in CEST

***********************************************
Time Presenter Title


9:45 10:30
Fabio Bonsignorio
Introduction
The R-Article process in IEEE R&A Magazine. Where we are. Call for submissions!


10:30 11:00
Dikai
Liu
Short presentations from other TCs’ representatives
Special issues in special TC domains

11:00 11:30
 Holger Caesar NuScenes - A large scale datasets for Self-driving cars perception and prediction benchmarking
11:30 14:00
Break


14:00 14:30
Signe Redfield
Session 2: Examples of Good Practices
Shared representations for Reproducible Robotics
14:30 16:00
TOWN HALL on Reproducibility and Benchmarking open to all attendees and TC representatives
16:00 16:45 Stefano Carpin Our R-Article: Issues, opportunities and lesson learned


16:45 17:30

Luca Carlone
Session 3: Objective Comparison and Verification
Benchmarking and Reproducibility in Robotics Perceptions
17:30 18:15 Jorge Dias Benchmarking and Assessment In Robotics Acting in Dynamic Environments - Facts and Figures from MBZ International Robotic Challenge
**********************************************

June 4th - all time are in CEST

**********************************************
Time Presenter Title


8:00
8:45
Fabio Bonsignorio (Chair)
Session 4: Replication and Benchmarking of Research Results (I)
Summary Discussion about First day

8:45
9:30
Angela Faragasso
Reproducibility and Benchmarking in Robotics for Healthcare and Nuclear Decommissioning

9:30 10:15
Francesco Amigoni
Experiments, tests, and benchmarks in robotics: a demarcation is possible

10:15 10:40
Andrea Censi
The AI Driving Olympics & Duckietown approach to robotics reproducibility (I)
10:40 11:00
Relax Break


11:00 12:15
Session 5: Contributed presentations and demos
- Remote demo of AI Driving Olympics evaluation (ETH Zurich)
- Other Experimental (remote) contributions
12:15 12:30
Break


12:30 13:30
Enrica Zereik
Angelo Odetti
Session 6: Stories and panel discussion
- Experience of Reproducible Robotics on the H2Arm an open source hw platform - Live from the Lab
-Discussion
13:30 14:30
Break

14:30 15:15
Joelle Pineau
Session 7: Replication and Benchmarking of Research Results (II)
Reproducibility of Research in Machine Learning and AI

15:15 16:00
Peter Kazanzides
Shared research platform and standards for sharing of data in surgical robotics
16:00 16:20 Andrea Censi
Liam Paull		 The AI Driving Olympics & Duckietown approach to robotics reproducibility (II)


16:20 17:00
Session 8: Discussion
Starting topics:
- How to make robotics a science?
- How can we improve the R-Article process and spread the reproducibility and benchmarking mindset?

17:00 17:30
Fabio Bonsignorio
Closing remarks
Reproducible Research in Robotics: The Road Ahead

FREE Registration (max 100 ATTENDEES)

To register for this webinar


Follow our page and stay tuned to our Facebook Stories and News!

Facebook