ShareThis Page
Technology

CMU researchers are teaching a computer to understand body language

Aaron Aupperlee
| Thursday, July 6, 2017, 4:21 p.m.
Carnegie Mellon University researchers have developed methods to detect the body pose, including facial expressions and hand positions, of multiple individuals. This enables computers to not only identify parts of the body but to understand how they are moving and positioned.
Courtesy Carnegie Mellon University
Carnegie Mellon University researchers have developed methods to detect the body pose, including facial expressions and hand positions, of multiple individuals. This enables computers to not only identify parts of the body but to understand how they are moving and positioned.

Robots might soon be able to tell if you flash a peace sign or flip the bird.

Researchers at Carnegie Mellon University have developed a computer that can understand and track body movements in real time, including individual fingers.

The computer could allow robots to read body language, helping them better perceive your mood and interact with you in social settings, said Yaser Sheikh, an associate professor of robotics leading the project. Robots could better understand what we mean when we point at something or do other hand gestures like putting our finger to lips to tell someone to keep quiet.

"We communicate almost as much with the movement of our bodies as we do with our voice," Sheikh said in a statement. "But computers are more or less blind to it."

The university believes it is the first time a computer can track individual fingers. The computer can also track several people at once.

Self-driving cars could detect when a person might step off the curb to cross a street by monitoring the person's body language. Coaches will be able to track every player on the field, court or ice and tell what they are doing with their arms, legs, heads and hands.

The researchers released the code publicly to allow other scientists to conduct more research and find more applications. Sheikh said more than 20 companies and commercial groups, including automotive companies, have expressed interest in licensing the technology.

The technology was developed using CMU's Panoptic Studio, a two-story dome that has about 500 video cameras. The studio can capture humans interacting and analyze the data.

CMU's Panoptic Studio

Aaron Aupperlee is a Tribune-Review staff writer. Reach him at aaupperlee@tribweb.com, 412-336-8448 or via Twitter @tinynotebook.

TribLIVE commenting policy

You are solely responsible for your comments and by using TribLive.com you agree to our Terms of Service.

We moderate comments. Our goal is to provide substantive commentary for a general readership. By screening submissions, we provide a space where readers can share intelligent and informed commentary that enhances the quality of our news and information.

While most comments will be posted if they are on-topic and not abusive, moderating decisions are subjective. We will make them as carefully and consistently as we can. Because of the volume of reader comments, we cannot review individual moderation decisions with readers.

We value thoughtful comments representing a range of views that make their point quickly and politely. We make an effort to protect discussions from repeated comments either by the same reader or different readers

We follow the same standards for taste as the daily newspaper. A few things we won't tolerate: personal attacks, obscenity, vulgarity, profanity (including expletives and letters followed by dashes), commercial promotion, impersonations, incoherence, proselytizing and SHOUTING. Don't include URLs to Web sites.

We do not edit comments. They are either approved or deleted. We reserve the right to edit a comment that is quoted or excerpted in an article. In this case, we may fix spelling and punctuation.

We welcome strong opinions and criticism of our work, but we don't want comments to become bogged down with discussions of our policies and we will moderate accordingly.

We appreciate it when readers and people quoted in articles or blog posts point out errors of fact or emphasis and will investigate all assertions. But these suggestions should be sent via e-mail. To avoid distracting other readers, we won't publish comments that suggest a correction. Instead, corrections will be made in a blog post or in an article.

click me