IROS Full Day Workshop on Sep 28

Schedule Changed!

AM session starts at 9AM

PM session starts at 14PM

Embodied Brain Systems Science

–from Body Representation in Human Brain

toward Rehabilitation Technology–

09:00-09:30 Talk 1: Prof. Jun Ota (The University of Tokyo)
"Overview of embodied-brain systems science"
09:30-10:05 Talk 2: Prof. Keisuke Shima (Yokohama National University)
"EMG-Based Direct Rehabilitation Between Human and Human"
10:05-10:40 Talk 3: Prof. Yasuhisa Hasegawa (Nagoya University)
"Embodiment of Extra-Robotic Finger Using Somatosensory Feedback"
10:40-11:20 Poster Session & Coffee Break
11:20-11:55 Talk 4: Prof. Giulio Sandini (Istituto Italiano di Technologia)
"Visual Perception and Biological Motion: Which Visual Features Contribute to Body Representation?"
11:55-12:30 Talk 5: Dr. Shingo Shimoda (RIKEN)
"Role of Muscle Synergy in NeuroFeedback Rehabilitation"
12:30-14:00 Lunch Break
14:00-14:35 Talk 6: Prof. Hajime Asama (The University of Tokyo)
"The Role of Sense of Agency in Body Consciousness"
14:35-15:10 Talk 7: Prof. Paul Verschure (Universitat Pompeu Fabra)
"Bringing Embodied Theoretical Neuroscience to the Clinic: A Case Study in the Rehabilitation of Stroke"
15:10-15:50 Poster Session & Coffee Break
15:50-16:25 Talk 8: Prof. Kazuo Kiguchi (Kyusu University)
"Realtime Motion Prediction with EEG"
16:25-17:00 Talk 9: Dr. Qi An (The University of Tokyo)
"Analysis of Human Standing-up Motion from Muscle Synergy and Application for Diagnosis Motion based on Muscle Synergy"
Aim of This Workshop
In daily life, a human can recognize one’s own body and the surrounding environment adequately to achieve adaptive movement and to maintain posture despite sudden environmental changes and gradual body change. To realize these adaptive movements, we hypothesize that some cognitive mapper of the body (body representation) exists as a neural mechanism for estimating the body state and the environment around the body, utilizing information from the sensory–motor system. Body representation is developed through interaction between the human body and the surrounding environment. It is altered slowly by brain plasticity to adapt to changes of several conditions. Another key feature to develop body representation is a person’s self-consciousness and self-recognition of one’s own body (sense of ownership). This consciousness and recognition are also created, updated, and transformed through perceptual and motion experience. However when body representation is distracted as in cases of stroke, neurodegenerative disease, or amputation, it is expected to cause motor dysfunction. Alternatively, an amputee might recognize the personal body incorrectly and might even feel pain in a lost limb (i.e., phantom limb pain). These phenomena imply the possibility that humans have a body representation in the brain, and that severe problems occur in the sensory-motor system when the brain is injured. To develop effective rehabilitation and training methodologies for elderly people, post-stroke patients, and amputee populations, it is necessary to elucidate the neural mechanisms of body representation and to apply this knowledge to rehabilitation interventions.

To achieve the goals described above, we started a five-year research program in 2014 for "Understanding brain plasticity on body representations to promote their adaptive functions" funded by a grant-in-aid for Scientific Research on Innovative Areas (FY2014–2018, PI: Prof. Ota) by MEXT, Japan. In our program, we have combined brain science and rehabilitation robotics using systems engineering. We thereby intend to gain an integrated understanding of motor control and somatognosia to create a new academic discipline known as embodied-brain systems science. To date, we have elucidated some biomarkers in the human brain which preserve body representation in the brain or mechanisms to develop body representation through interaction between the human body and the environment. Now we are at the next stage of promoting our research project further to develop novel rehabilitation technologies based on neural mechanisms of body representation and slow adaptation to environmental and body change. This workshop targets to cover the recent findings in brain mechanism including mathematical models of change of body presentation and human motor control. This workshop will further present and discuss recent advancement to utilize these findings to assistive devices for neurorehabilitation such as upper limb training systems using function electrical stimulation and VR technology, exoskeleton systems, and new prosthetics device. Not merely introducing our research outcomes, this workshop will invite outstanding researchers who work in fields of rehabilitation therapy and human cognition to discuss how one can create novel rehabilitation strategies.

Talk 1 (09:00-09:30)
"Overview of embodied-brain systems science"

QiAn Speaker: Prof. Jun Ota (The University of Tokyo)

Biography: Professor Jun OTA is a Professor at Research into Artifacts, Center for Engineering (RACE), the University of Tokyo. He received B.E., M.E. and Ph.D. degrees from the Faculty of Engineering, the University of Tokyo in 1987, 1989 and 1994 respectively. From 2009, he became a Professor at Research into Artifacts, Center for Engineering (RACE), the University of Tokyo. From 2015, he is a guest professor of South China University of Technology. From 1996 to 1997, he was a Visiting Scholar at Stanford University. He received RSJ (the Robotics Society of Japan) Fellow in 2016. His research interests are multi-agent robot systems, embodied-brain systems science, design support for large-scale production/material handling systems, human behavior analysis and support.

Abstract: Japan is now super-aged society, and we are experiencing a sharp increase in the number of patients of motor paralysis and other dysfunctions resulting from motor dysfunction, stroke, and neurodegenerative diseases. Thus, establishing effective rehabilitation techniques to overcome these motor dysfunctions is of paramount importance. The key to achieving this is to elucidate the mechanisms by which the brain adapts to changes in body functions. However, abnormalities in somatognosia can occur even in diseases that do not cause motor dysfunction. This indicates that we create and maintain a model of the body in the brain, which we call body representation in the brain. Embodied-brain program is one of the programs from Japan Society for the Promotion Science, Grant-in-Aid for Scientific Research on Innovative Areas, interdisciplinary area. Those are awarded to new research areas that will lead to the upgrading and enhancement of scientific research in Japan. The official name of the program is Understanding brain plasticity on body representations to promote their adaptive functions (program director: Professor Jun Ota, the University of Tokyo). This is five-year project from 2014 to 2018. The number of researchers in this program is about 130 now. Embodied-brain systems science is a new transdisciplinary research area with the integration of brain science, systems engineering, and rehabilitation medicine. We will address the problem of impaired motor function that is prevalent in our ultra-aged society because of the locomotor and neurologic disorders of old age. We will do this through the integrated academic discipline of "embodied-brain systems science." The targets of this program is as follows: (a) to construct model-based rehabilitation that intervenes in the representation of the body in the brain, (b) to describe the structure of the major brain functions essential to the existence of somatognosia and motor control and work toward common computational principles for them, and (c) to understand the plasticity of body presentation in the brain (slow dynamics) and develop technology that allows it to be controlled. An overview of embodied-brain systems science is introduced in this presentation.

Talk 2 (09:30-10:05)
"EMG-Based Direct Rehabilitation Between Human and Human"

Speaker: Prof. Keisuke Shima (Yokohama National University)

Biography: Keisuke SHIMA received the BE, ME, and PhD degrees from Hiroshima University, Hiroshima, Japan, in 2005, 2007, and 2009, respectively. He was a Research Fellow of the Japan Society for the Promotion of Science (JSPS) from 2007 to 2008 (DC1) and from 2009 to October 2012 (PD). From November 2012 to March 2013, he has been an Assistant Professor in the Yokohama National University, Japan. He is currently an Associate Professor. His current research interests focus on biological signal analysis, neural networks, and human-machine interfaces. Dr. Shima received the Best Student Paper Award - Finalist and Honorable Mention from the IEEE International Conference on Systems, Man and Cybernetics in 2008.

Abstract: Extensive and focused physiotherapy is needed to help individuals with disabilities, such as hemiplegics, achieve natural physical movement involving the simultaneous use of various muscles. Here we outline a new approach to such work involving the use of FES (functional electrical stimulation) and EMG signals to help people with hemiplegia resulting from spinal injuries or cerebrovascular accidents (CVAs) achieve such muscle contraction and to enable related evaluation. In the presentation, we introduce that the details of the proposed concept, and some experiments in motion communication between human and human.

Talk 3 (10:05-10:40)
"Embodiment of Extra-Robotic Finger Using Somatosensory Feedback"

Hasegawa Speaker: Prof. Yasuhisa Hasegawa (Nagoya University) Biography: Yasuhisa Hasegawa received his doctor degree in engineering from Nagoya University, Japan in 2001. From 1996 to 1998, he worked for Mitsubishi Heavy Industries Ltd., Japan. He was formerly a Research Associate at Nagoya University, an Assistant Professor at Gifu University, and an Associate Professor at University of Tsukuba. He is currently a Professor of Department of Micro-Nano Mechanical Science and Engineering, Nagoya University. He is the Administrative Committee member of Robotics and Automation Society, IEEE and a Deputy Chief Editor of ROBOMECH Journal (Springer). He is mainly engaging in the research fields of human intention-based physical assistive robot, human-robot interface device and dynamic motion control. He received a Best Paper Award at the 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS2003), at the 2004 and 2015 International Symposium on Micro-Nano Mechatronics and Human Science (MHS2004, MHS2015), at the IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM2013), and at the IEEE International Conference on Robotics and Automation (ICRA2017).

Abstract: In my talk, two kinds of topics for intuitive control and embodiment of a robotic limb will be introduced. The first one is an interactive interface to intuitively control a robot or a wearable robot and to monitor the state of the robot through an electric feedback. The other is an interface to develop an illusion of embodiment. The interface gives a user to an illusion of embodiment as if an additional robotic finger such as a sixth finger could be one of own fingers. First, we are challenging to develop a new interface to control the exoskeleton robot for a paraplegic patient. Healthy parts of a patient body such as hands become alternative means to operate the assistive device since a bioelectric signal such as EMG is missing at the lower limb of the paraplegic patient. Under the terms of this paralysis, a feedback from the robot through the alternative path is very important for the user to control the assistive robot without visual confirmation. I introduce the present progressive work which uses an electric stimulation for alternative feedback of lost deep sensation. Second, we introduce the other interface to include a new finger into a body schema with a unique interface. An actual thumb of right hand controls the additional robotic finger attached on a palm of left hand. This task execution developed a temporal illusion that an actual right thumb moves to a palm of the left hand. In addition, an embodiment of the robotic limb was invoked in two conditions through a long-term experiment. A slow and fast dynamics in change and improve of body representation were found in the experiment.

Talk 4 (11:20-11:55)
"Visual Perception and Biological Motion: Which Visual Features Contribute to Body Representation?"

QiAn Speaker: Prof. Giulio Sandini (Istituto Italiano di Technologia)


Abstract: Visual perception in artificial systems has been always considered a way of measuring different aspects of the environments to support robot’s actions. However, in the context of human robot interaction, the goal of movements is not only to reach a “physical” result, such as grasping an object or navigating a room, but also to communicate to the partner(s) the goal to be jointly pursued. In this scenario the role of vision is not only to support action execution but also to stimulate our ability to imagine/predict actions of others. Our hypothesis is that an implicit contribution to this ability is mediated by the visual appearance of the regularities of biological motion typical of muscle-activated systems. Stemming from these observations the goal of the talk is to discuss the role of vision during social interaction and to present how low-level dynamic visual primitives could be used to represent biological motion and to stimulate our anthropomorphic imagination.

Talk 5 (11:55-12:30)
"Role of Muscle Synergy in NeuroFeedback Rehabilitation"

QiAn Speaker: Dr. Shingo Shimoda (RIKEN)

Biography: Shingo Shimoda received the B.S., M.S., and Ph.D. degrees in mechanical and electronic from the University of Tokyo, Tokyo, Japan, in 1999, 2001, and 2005, respectively. He spent as a visiting student at MIT in 2003-2004. He was a Research Scientist with the Biomimetic Control Research Center, RIKEN, Japan, in 2005. In 2008, he became a Unit Leader with RIKEN Brain Research Institute-TOYOTA Collaboration Center, Intelligent Behavior Control Collaboration Unit, Nagoya, Japan. He is principle chair of Technical Committee on Cognitive Robotics in IEEE Robotics and Automation Society.

Abstract: For rehabilitations, robots are usually used to support the paralyzed motions. Many of the patients, however, need the less support than the conventional robotics support but enough the specified inputs to activate the remaining motion control capability. Our concept for neurorehabilitation is to clarify the patient states from the behavior observations and provide the minimum support and try to use the patients’ motion control capability as much as possible. We show the results of several clinical test conducted based on our concept.

Talk 6 (14:00-14:35)
"The Role of Sense of Agency in Body Consciousness"

QiAn Speaker: Prof. Hajime Asama (The University of Tokyo)

Biography: Hajime Asama received his M. S. and Dr. Eng. in Engineering from the University of Tokyo, in 1984 and 1989, respectively. He was a Research Scientist, etc. in RIKEN Japan from 1986 to 2002. He became a professor of RACE, the University of Tokyo in 2002, and a professor of School of Engineering, the University of Tokyo since 2009. He received RSJ Distinguished Service Award in 2013, etc. He was the vice-president of Robotics Society of Japan in 2011-2012. an AdCom member of IEEE Robotics and Automation Society in 2007-2009. Currently, He is the president-elect of IFAC since 2017, the president of International Society for Intelligent Autonomous Systems since 2014, an associate editor of Journal of Robotics and Autonomous Systems, Control Engineering Practice, etc. He is a Fellow of JSME and RSJ. His main research interests are distributed autonomous robotic systems, smart spaces, service engineering, embodied brain systems, and service robotics.

Abstract: Body consciousness such as sense of agency and sense of ownership is generated in real time based on the body representation in brain. This process can be called “fast dynamics.” On the other hand, the body representation is created, updated and transformed through perceptional and motion experience, which can be called “slow dynamics.” In this group, these dynamics on the process creating and updating body representation in brain related to body consciousness are investigated and modelled mathematically. In this presentation, perceptive processes underlying the body consciousness is discussed including the influence of high-level cognitive processes on task performance on a low level perceptual processing of delay detection, the attention allocation during the updating of body consciousness, and the brain activity during the preparation of motion during the fast dynamics updating of body consciousness.

Talk 7 (14:35-15:10)
"Bringing Embodied Theoretical Neuroscience to the Clinic: A Case Study in the Rehabilitation of Stroke"

QiAn Speaker: Prof. Paul Verschure (Universitat Pompeu Fabra)

Biography: Paul Verschure is Catalan Institute of Advanced Studies (ICREA) Research Professor, Director of the Center for Autonomous Systems and Neurorobotics at Universitat Pompeu Fabra and director of the neuro-engineering program at the Institute for Bioengineering of Catalunya where he runs the Synthetic Perceptive, Emotive and Cognitive Systems (SPECS) Laboratory ( He is founder/CEO of Eodyne Systems S.L. (, which is commercializing a novel science grounded neurorehabilitation technology. Paul is founder/Chairman of the Future Memory Foundation ( which aims at supporting the development of new tools and paradigms for the conservation, presentation, and education of the history of the Holocaust and Nazi crimes. He received his MA and Ph.D. in Psychology, and Paul's scientific aim is to find a unified theory of mind and brain using synthetic methods and to apply it to the quality of life enhancing technologies. His theory of mind and brain, Distributed Adaptive Control, has been generalized to a range of brain structures and robotic systems and has laid the foundation for a novel neurorehabilitation approach called the Rehabilitation Gaming System ( Paul explores new methods for the simulation, visualization, and exploration of complex data to support his DAC theory and advance clinical diagnostics and intervention in neuropathologies ( Complementary to his science, Paul has developed and deployed over 25 art installations ( These include the biomimetic mixed reality space Ada experienced by over half a million visitors (2002) and more recently three virtual/augmented reality educational installations and applications for the Memorial Site Bergen-Belsen (2012 - ) which is now generalized to other sites across Europe. Paul manages a multidisciplinary team of 30 researchers ( with whom he has published over 300 articles in leading journals and conferences in a range of disciplines. Paul collaborates with a wide network of international researchers. He has represented Switzerland at the Global Science Forum of the OECD, is chair of the annual Barcelona Cognition, Brain, and Technology summer school and co-chair of the annual Convergent Science Network’s conference Living Machines. Paul also hosts a podcast ( He is the founder and academic director of the Interdisciplinary Master program Cognitive Systems and Interactive Media at University Pompeu Fabra.

Abstract: Over the last 20 years, we have developed the Distributed Adaptive Control theory of mind and brain (DAC). DAC has successfully controlled a range of flying, mobile and humanoid robots and interactive mechatronic systems (see for a review Verschure, 2012) and generalized to several brain systems (see for a review Verschure, Pennartz, & Pezzulo, 2014). In this combined convergent approach DAC validates hypotheses on behavior, function, and structure and has given rise to advanced social robotic systems that can learn to represent self and other and to use systems of memory and learning to acquire language and communication strategies. As a further validation of the DAC theory, we have turned to the understanding and treatment of neuropathologies. In this talk, I will show how principles of the DAC theory on embodied and goal oriented learning have successfully generalized to the functional rehabilitation of stroke patients. Over 800 stroke patients have successfully used this DAC based Rehabilitation Gaming System (RGS) at 20 centers across the world. We are currently validating the use of RGS in other neuropathologies including multiple sclerosis, Parkinson’s disease, cerebral palsy and Alzheimer’s disease. This translation from the robotics lab to the clinic demonstrates that the DAC principles that guide robot perception, cognition and action can directly be of value towards recovering and rescuing these functions in humans.

Talk 8 (15:50-16:25)
"Realtime Motion Prediction with EEG"

KazuoKiguchi Speaker: Prof. Kazuo Kiguchi (Kyusu University)

Biography: Kazuo Kiguchi received the Bachelor of Engineering degree in mechanical engineering from Niigata University, Japan in 1986, the Master of Applied Science degree in mechanical engineering from the University of Ottawa, Canada in 1993, and the Doctor of Engineering degree in Mechano-Informatics from Nagoya University, Japan in 1997. He was a Research Engineer with Mazda Motor Co. between 1986-1989, and with MHI Aerospace Systems Co. between 1989-1991. He worked for the Dept. of Industrial and Systems Engineering, Niigata College of Technology, Japan between 1994-1999 and for Dept. of Advanced Technology Fusion, Graduate School of Science and Engineering, Saga University, Japan between 1999-2012. He is currently a professor in the Dept. of Mechanical Engineering, Faculty of Engineering, Kyushu University, Japan. He received JSME Funai Award in 2010, Lifetime Achievement Award at WAC2014, JSME Medal for Outstanding Paper in 2017, and JSME Medal for Distinguished Engineers in 2017. His research interests include human assist robots, rehabilitation robots, biorobotics, human-robot interface, and application of robotics in medicine. He is Fellow of the Japan Society of Mechanical Engineers, and a member of IEEE (R&A, SMC, and EMB Societies), the Robotics Society of Japan, the Society of Instrument and Control Engineers, and the Japan Society of Computer Aided Surgery.

Abstract: Human assist robots such as power-assist exoskeletons or robotic limb are expected to play important roles to assist daily living motion of physically weak persons such as elderly, injured, or disabled persons. Those kinds of human assisting robotic systems must estimate the users' motion intention in real-time. Electroencephalogram (EEG) is an important brain signal which contains information of human intention. In this presentation, EEG-based real-time motion intention methods are introduced in order to control the human assisting robotic systems based on the user’s motion intention.

Talk 9 (16:25-17:00)
"Analysis of Human Standing-up Motion from Muscle Synergy and Application for Diagnosis"

QiAn Speaker: Qi An (Department of Precision Engineering, The University of Tokyo)

Biography: Qi received his B.E., M.E., and PhD from the University of Tokyo, Japan, in 2009, 2011, and 2014. From 2010 to 2011, he joined Yoky Matsuoka’s Lab as a visiting student at University of Washington, USA. From Nov.2014 to Mar.2015, he studied Martin Buss’s Lab as a visiting researcher at Technische Universitat Munchen, Germany. He is currently an assistant professor at the University of Tokyo. His research interest is to understand how humans modulate their muscle synergies to adapt different environments.

Abstract: In order to utilize the concept of muscle synergy for online rehabilitation, it is necessary to know impaired structure of muscle synergy from patient movements and to decide the direction of rehabilitation. However, muscle synergy structure varied among patients, and it is not always easy to measure their muscle activity to identify synergy structure. In order to solve this problem, our study has focused on human standing-up motion and constructed a database of resultant movement from impaired muscle synergy structure. This database is constructed from forward dynamic simulation of human musculoskeletal model to calculate how standing-up motion changes according to the different structure of muscle synergy. It can therefore elucidate that what muscle synergies resulted in failure motion (e.g. falling down or unable to lift up their body) and how each synergy contributes to success of the standing-up motion. The database enables health care provider to detect impaired motor function from observed movement and it also suggests the rehabilitation direction to improve body function.

Poster Talk (10:40-11:20, 15:10-15:50)
Poster 1: "A comprehensive procedure for evaluating the midline of the transverse plane in humans"

Author: Arito Yozu, Yuki Mataki, Kei Nakai, Akira Matsushita, Ryoko Takeuchi, Yukiyo Shimizu, Hiroshi Kishimoto, Kayo Tokeji, Hirotaka Mutsuzaki, Yutaka Kohno, Nobuaki Iwasaki (Ibaraki Prefectural University of Health Sciences Hospital)

Abstract: Patients with severe disabilities (e.g., cerebral palsy) often have multiple impairments including visual problems, auditory problems, muscle weakness, limited range of motion, morphological deformity, etc. Not only their body but also their perception and cognition become asymmetric. When we prescribe a seating system or an orthosis, we need to decide the orientation of their midline, however, asymmetries in multiple systems complicate our decision. Here, we propose a comprehensive procedure for evaluating the midline of the transverse plane for multiple systems. First, we evaluate the eye-centered midline by testing the visual field and the ocular range of motion. Second, we evaluate the auditory-centered midline by testing the auditory field. Third, we evaluate the vestibular-centered midline by testing the equilibrium. Fourth, we evaluate the midline for multiple body segments: head, thorax, pelvis, and lower extremities. We also evaluate the range of rotation for each joint. Finally, we describe the difference between the midlines using the eye-centered midline as a basis. This is because patients’ daily activities are usually oriented by visual perception. By using our method, we can examine the patients’ posture or their locomotion adequately. We can apply this method in prescribing an orthosis or a seating system for patients.

Poster 2: "Mirror Descent based Reinforcement Learning"
Author: Megumi Miyashita, Shiro Yano, Toshiyuki Kondo (Tokyo University of Agriculture and Technology)

Abstract: Reinforcement learning is one of learning method and can be applied to various domains. In recent years, some researches have dealt with the relation between reinforcement learning and black-box optimization. In the problem of black-box optimization, we can get only the evaluated value for an objective function. Thus we can not know the form of the objective function. This setting is similar to policy and reward in reinforcement learning. We propose reinforcement learning algorithm as black-box optimization algorithm from mirror descent algorithm which is general optimization algorithm. We call the proposed method Mirror Descent Search. Using Mirror Descent Search, we can also derive the extension algorithm by applying the accelerated method of mirror descent. In other words, we can see reinforcement learning widely through the connection to mirror descent. Then we solve the task that the robot arm passes over a point by using proposed methods. The results tell us that the extension algorithm learns faster than Mirror Descent Search.

Poster 3: "Bayesian Learning and Sense of Agency"
Author: Shiro Yano (Tokyo University of Agriculture and Technology), Hiroshi Imamizu (The University of Tokyo), Toshiyuki Kondo (Tokyo University of Agriculture and Technology), Takaki Maeda (Keio University)

Abstract: Sense of agency is the subjective sense about the sensory information such that the information originates from my own action. Past researches have employed Comparator model as the computational model of the Sense of agency. Comparator model was proposed in the motor learning domain. Although probabilistic perspectives bring success in motor learning domain recently, there are not enough research to rewrite the model based on probabilistic representation. In this research, I rewrite the model based on the probabilistic perspective. As the part of this approach, we define the sense of agency by the likelihood, and proposed both the Bayesian learning model and the stochastic gradient learning(SGD) model. We took an experiment to compare whether Bayesian model or SGD model are suitable to explain basic situations.