CMU study on brain-hand interaction could improve prosthetics

When people sit down for a meal, they don’t think about what to do with their hands when eating. It’s just second nature.

Yet, their hands are constantly adapting posture and grip to the cutlery, food and drinks on the table, in order to enable the goal of eating.

“Our brains have an amazing ability to automate highly complex processes,” said Brad Mahon, psychology professor at Carnegie Mellon University.

A recent development from Carnegie Mellon researchers found that those types of actions with hands are built from a vocabulary of basic building blocks, and that process is supported by the brain’s supramarginal gyrus, or SMG.

That study, in turn, could possibly lead to better developments with prosthetics and brain-computer interfaces.

The research, led by Leyla Caglar while she was a postdoctoral fellow at CMU and the University of Coimbra, Portugal, found the brain organizes hand movements by combining a small set of simple motions to create a vocabulary of everyday actions.

Actions like writing a letter with a pen, grasping a tool or unlocking a door and turning a key are built in the brain in a systemic way, said Caglar, who is now a postdoctoral fellow at Mount Sinai Medical Center.

Caglar said the SMG operates like an “assembly hub.” Instead of storing movements separately, it combines finger, hand, arm and wrist patterns called “kinematic synergies” to build the range of actions those body parts perform.

Like how native speakers don’t have to think about how to say the words they want to use, the SMG runs processes automatically in the background, according to the study.

Although there are many different types of interactions between objects and people, and despite differences in manual dexterity across people, all humans have a common neural system that supports complex interactions, the study showed.

Caglar said the study results move researchers a step closer to understanding the principles of brain organization that make human tool use possible.

“We’re interested in understanding how the brain supports all the amazing things we can do with our hands,” Mahon said.

Modern neuroprosthetics struggle to translate neural activity into natural, multi-joint movement, Caglar said.

The study shows the brain’s left SMG encodes complex, object-directed actions as linear combinations of basic, patterned coordinated movement across fingers, hands, wrists and arms; and provides a generative model of action.

“By identifying such synergy-level neural representations, prosthetic controllers could operate on these low-dimensional bases instead of individual muscle activations, which might allow for more stable, interpretable, and generalizable control,” Caglar said.

“This parallels how speech prosthetics map neural activity to phonological feature combinations rather than to entire words and how kinematic synergies could serve as the motor ‘phonemes’ for prosthetic hand and arm control by providing a low-dimensional, compositional vocabulary for movement.”

Because the SMG integrates visual and functional information about objects, it links and integrates information about what an object is, how it should be grasped and why it is used that way, Caglar said. Similarly, effective artificial limbs must integrate intention and object recognition.

Having a better understanding of how the SMG recognizes and recombines that information could offer a blueprint for designing prosthetics that fuse visual object recognition with motor planning, she said.