Complexity Of Eye-Hand Coordination
Marlene Cimons, National Science Foundation
Research helps understand how brain systems interact to carry out cognitive processes
People not only use their eyes to see, but also to move. It takes less than a fraction of a second to execute the loop that travels from the brain to the eyes, and then to the hands and/or arms. Bijan Pesaran is trying to figure out what occurs in the brain during this process.
“Eye-hand coordination is the result of a complex interplay between two systems of the brain, but there are many regions where this interaction takes place,” says Pesaran, an associate professor of neural science at New York University. “One of the things about the current state of knowledge is that it is focused on the different pieces of the brain and how each works individually. Relatively little work has been done to link how they work together at the cellular level.”
The thrust of his research involves studying how neurons in these parts of the brain communicate with one another.
“The cerebral cortex contains a mosaic of brain areas that are connected to form distributed networks,” says the National Science Foundation (NSF)-funded scientist. “In the frontal and parietal cortex, these networks are specialized for movements such as saccadic (voluntary) eye movements and reaches, that is, hand and arm movements. Before each movement we decide to make, these areas contain specific patterns of neural activity which can be used to predict what we will do.”
A more sophisticated understanding of the brain’s role in eye-hand coordination can be an important model for discovering how brain systems interact to carry out cognitive processes in general, he says. Such insights could lead to new neural technologies that translate thoughts into actions, for example, to control a robotic arm or prompt speech.
“There is a whole new set of technologies called neural prostheses,” Pesaran says. “In the future, there could be devices in the brain that will help people remember, to think more clearly, and to help them move.”
Using eye movements to prompt hand and arm movements involves building a spatial representation, “which is improved by moving our eyes,” he says. “The command that is sent to the eyes moves the eyes, which effectively measure space when they move, and that is used to improve the accuracy of the reach. We move our eyes to improve our movement, not just to see better.”
He often describes the behavior of high level ping pong players to explain how it works.
“You keep your eye on the ball so you know where it is, so you can hit it,” he says. “But right up until the minute you hit the ball, something important is happening, which is that your brain is sending a command to your arm to hit the ball. But the visual signals are delayed. At the time you hit the ball, the vision of the ball won’t enter your brain for another fraction of a second, so there is no point in looking at the ball. You can look all you want, but your arm already has moved.
“When ping pong players are playing at a high level, they look at the ball up to the point where they hit it. As soon as the paddle makes contact with the ball, you can see their eyes and head turn to now look at their opponent. They think they are looking at their opponent when they are hitting the ball, but they are looking at ball. Their eyes are tracking the ball, even though they are aware of their opponent.
“This helps the brain keep a very high resolution of space to make the stroke more accurate,” he continues. “It’s not about seeing the ball, because by then it’s too late. It’s about moving the eyes with the ball so that the stroke is more accurate. And the brain orchestrates this complicated pattern of behavior.”