Vancouver, BC, CAN. Entering the world, we reach out to our environment, with a brain map of essential pathways for exploration. But what really goes on between our brains, eyes and appendages? And how do we process the information necessary to connect with our most immediate environment, ourselves?

 

“It is still a mystery, really,” says Pai, computer science professor at the University of British Columbia (UBC). "No one has ever completely mapped out the processes at the level of specific neurons, muscles and tendons.”

 

---

Musculotendon Simulation for Hand Animation. Shinjiro Sueda, Andrew Kaufman and Dinesh K. Pai. Proceedings of SIGGRAPH 2008: ACM Transaction on Graphics.  [ Download PDF ]

---

 

Prof. Dinesh Pai researches the core principles which enable humans and robots to use sensory information as they move and interact with their environment. Such sensorimotor computation forms the bridge between abstract information processing in the human brain and the concrete reality of the physical world.

 

The human brain uses exteroceptive sensors (such as vision and touch) to perceive the state of its external environment. The brain uses proprioceptive sensors (such as muscle spindles and the vestibular organs) to understand the state of its own body. In the end, we take action by controlling our muscles.

 

Our brains and bodies have evolved over hundreds of millions of years to perform complex sensorimotor tasks without much conscious thought. These actions appear so simple to us that it is easy to overlook the extraordinary sophistication behind ordinary actions such as looking at an object with our eyes and picking it up with our hand. The sophistication only becomes apparent when we try to reproduce these “ordinary” skills in robots, or when we observe the development of these skills in childhood and their loss in the elderly.

 

Pai now is part of a UBC team leading an international initiative to look under the hood of that apparent simplicity.

 

“Essentially, we are reverse engineering the brain to produce the first working computational model of the complex interplay between our minds and our bodies,” says Pai.

 

The project builds on a highly successful workshop, Sensorimotor Computation: Bits, Bodies, and Brains. [C1] Hosted by The Peter Wall Institute for Advanced Studies at the University of British Columbia, the workshop helped to identify and refine the project themes. 

 

Overall, the project is an interdisciplinary effort that seeks a deep understanding of how the human brain and spinal cord interact with the real world through the senses and muscles, using techniques from many fields ranging from neurobiology to computer science. It provides a window into the workings of the human brain, because it connects empirical observations of biological systems — from psychophysical studies to neuron electrophysiology — to the functional constraints on any system that successfully interacts with the physical world.

 

This connection to insights from robotics, computer science, and engineering could lead to a deeper understanding of biological systems. Pai's own work involves computational and physical modeling, with applications to auditory displays, biomechanics, computer graphics, and haptic interfaces. The latter is a computer communication interface that achieves tactile contact through a device that senses body movement (such as a data glove, or even a complete suit of clothes).

 

Using magnetic resonance imaging (MRI), the team catalogs body parts and functions, tracing their interactions with the brain. This information is used to create a working three-dimensional computer model of all these functions. “We are in uncharted territory, in terms of computing,” says Pai. “It’s not like you can find software like this at your local Future Shop or Best Buy. So we have been creating our own as we go along.”

 

The scientific goal of this project is to model the complex computations, sensing, and motor actions that are required to control our eyes and hand when we look at. or reach out for, an object of interest. The team will construct computational models of how the eyes and head are moved to direct gaze to objects of interest in the environment, and how the hand manipulates objects. The researchers base their models on neurobiological measurements of how humans actually perform these tasks.

 

Although the project is just ramping up, the team’s mapping and modeling expedition is already producing some of the world’s most realistic computer simulations of the human body. “Our research is really guided by a desire to determine and model exactly what is happening under our skin, first and foremost.” … “There will be many exciting outcomes from this project, but it really falls under the category of pure research.”

 

Current robots have as much in common with human movements as helicopters do with seagulls.

“Current robots have as much in common with human movements as helicopters do with seagulls,” Pai adds. “The challenges are similar, but they use completely different solutions.”

 

In time, the results will have important implications for applied clinical research and therefore for human health in the long term. The project could produce great leaps forward in medicine, industry and robotics.

 

Down the road, the team’s findings will enable doctors to test surgical outcomes before picking up a scalpel. "There is an amazing amount of variance between humans — skeletons, organs, muscles can all differ in size from person to person,” says Pai. “That means there is always some guesswork involved in surgery.”

 

“But if you can give someone an MRI and create a personalized computer model, suddenly a doctor has more information to work with,” he says. “They can say, ‘If I cut this tendon, what exactly is going to happen, given this patient’s unique body.’”

 

Advances in the field of neuroprosthetics — devices that replace or improve the function of an impaired nervous system — is another desired research outcome, Pai says.

 

“With a better understanding of mind-body connections, we hope to be able to use electrodes in the brain or spinal cord to restore some functions in people who have experienced strokes or some other disability.”

 

While these applications are still years away, the field of digital animation is taking note of their research. The upcoming prestigious computer science conference ACM SIGGRAPH will publish research by Pai and PhD candidate Shinjiro Sueda that outlines how the team’s modeling of body movements can help to make digital animations of humans more realistic [C2].

 

The Peter Wall Institute for Advanced Studies at the University of British Columbia provides funding for this research.

The multidisciplinary research team is an international effort that includes personnel from the countries of:

• Canada: McGill University, the University of British Columbia (UBC).
• Japan: Digital Human Research Centre.
• Italy: Santa Lucia Foundation.
• United States: Northwestern University, Smith Kettlewell Eye Research Institute), UCLA, University of Washington (UW).

The project co-investigators at the University of British Columbia (UBC) include

• Tony Hodgson, Mechanical Engineering.
• Tim Ingliss, School of Human Kinetics.
• Alan Mackworth, Computer Science.
• Martin McKeown of the Pacific Parkinson’s Research Centre.
• Dinesh Pai, Sensorimotor Systems Laboratory.
• John Steeves, Director of International Collaboration on Repair Discoveries (ICORD).

Basil Waugh contributed to this article.

 

[C1] Sensorimotor Computation: Bits, Bodies, and Brains. D. K. Pai (PI), A. J. Hodgson, J. T. Inglis, A. K. Mackworth, M. J. McKeown, J. D. Steeves. Peter Wall Institute for Advanced Studies (PWIAS): Report on PWIAS Exploratory Workshop (9-11 February 2007).  [ Download PDF ]

Executive Summary (excerpt). Can we develop a deep, constructive understanding of how the human brain and spinal cord interact with the real world through its senses and muscles? This was the topic of the highly multidisciplinary PWIAS workshop on sensorimotor computation. It examined three specific sensorimotor subsystems: 1. Eye movements, Hands and dexterity, and 3. Balance. Each was examined from biological, behavioral, physical, and computational perspectives, focusing on two themes: Modeling and Learning.

The workshop identified the essential role for computational models in understanding the complexity of sensorimotor systems in biology. The specific themes that were identified for further exploration include the role of biomechanisms, control architectures, context dependence, and composition of sensorimotor programs.

[C2] Musculotendon Simulation for Hand Animation. Shinjiro Sueda, Andrew Kaufman and Dinesh K. Pai. Proceedings of SIGGRAPH 2008: ACM Transaction on Graphics.  [ Download PDF ]

Abstract

We describe an automatic technique for generating the motion of tendons and muscles under the skin of a traditionally animated character. This is achieved by integrating the traditional animation pipeline with a novel biomechanical simulator capable of dynamic simulation with complex routing constraints on muscles and tendons. We also describe an algorithm for computing the activation levels of muscles required to track the input animation. We demonstrate the results with several animations of the human hand.

 

 

Name

Email

Website

Title

Comment

smaller | bigger

I have read and agree to the Terms of Usage.

Write the displayed characters

Add Comment