'Watson' has serious limitations
By Stephen Baker
Published: Sunday, Feb. 20, 2011,
A mere three years ago, Watson, the IBM computer that researchers at Carnegie Mellon University helped develop, was a "Jeopardy!"-playing fool. And that's putting it mildly.
Watson had the verbal skills of a toddler. It botched the solutions to the game-show clues with howlers that filled IBM's research lab with laughter -- and raised deep concern.
Once, when queried about a famous French bacteriologist, Watson skipped right past Louis Pasteur and responded instead: "What is, 'How tasty was my little Frenchman?'" (the title of a 1971 Brazilian movie about cannibals).
Even worse, Watson churned away for nearly two hours to come up with such nonsense.
Things have changed.
The former bionic idiot, making its man-versus-machine debut on "Jeopardy!" last week, is now smart and fast, with a startling command of idiomatic English. This vastly improved Watson drubbed "Jeopardy!" legends Ken Jennings and Brad Rutter.
The question now is whether the computer has unfair advantages. Is its speed on the buzzer too fast, its trove of resource documents too big• Those debates are sure to heat up after Watson's big win.
The smart machines in our life -- from Watson and Google's supercomputer to the iPhone -- are advancing relentlessly. By most counts, our cave-painting ancestors had brains comparable to our own. But in the 40,000 years between us and them, our computational tools have evolved from piles of stones to the abacus, the calculator and now this machine that can make life miserable for humans on the "Jeopardy!" set.
And every time Watson puts its digital foot in its mouth -- it still happens at least once or twice a game -- researchers can tweak the machine's instructions and its performance improves. Other programs, meanwhile, lead it to make automatic adjustments in search of more accurate answers. This is known as machine learning, a vital component within Watson.
It's all too easy to see Watson do its thing and conclude that legions of such machines will soon relieve us of our brainwork and our jobs, if not our souls. In fact, machines like Watson will no doubt displace people who are paid to answer questions, probably starting with telephone call centers.
But humans will adjust, as we always have. When our inventions -- from tractors and cotton gins to spell-checking software -- take over certain chores, we move to niches beyond the range of these tools. And believe me, after watching Watson in action for a year, I can assure you that there's plenty of room in the work world for the still-peerless human mind.
You see, Watson isn't nearly as smart as it looks on TV. Outside of its specialty of answering questions, the computer remains largely clueless. It knows nothing. When it comes up with an answer, such as "What is 'Othello'?," the name of Shakespeare's play is simply the combination of ones and zeros that correlates with millions of calculations it has carried out. Statistics tell it that there is a high probability that the word "Othello" matches with a "tragedy," a "captain" and a "Moor."
But Watson doesn't understand the meaning of those words any more than Google does or, for that matter, a parrot raised in a household of Elizabethan scholars.
Watson is incapable of coming up with fresh ideas, much less creating theories, cracking jokes, telling a story or carrying on a conversation. Its ability is simply to make sense of questions and then scour a trove of data for the most likely answers.
It represents a dramatic advance in artificial intelligence but like another famous IBM computer, Deep Blue, Watson excels on a limited playing field, in a game defined by clear, rigid rules.
In 1997, Deep Blue beat the reigning world chess champion, Garry Kasparov. Chess remains forever tied to the format the computer aced: The same queens, knights and pawns moving in knowable combinations -- albeit trillions of them -- on the 64 squares of the board. The best players are now machines.
But consider "Jeopardy!" The game is written by humans for humans; it can easily be adapted. In coming seasons, writers could quell the rise of Watson and its ilk by shifting toward clues that favor the subtleties of the human mind.
This means fewer of the factoids that Watson feasts on: the conquest that occurred in 1066 ("What is Norman?"), the biblical figure who lived in a whale ("Who is Jonah?"). A revised "Jeopardy!" would feature more implicit clues, based on judgment, innuendo and the human experience.
This clue, for example, ties Watson into knots:
"Look in this direction and you'll see the wainscoting." The answer is rooted in human experience, not data. Only a "Jeopardy!" contestant with a body is likely to understand it and come up with the right response: "What is down?"
More than base knowledge, that simple clue requires thought. That's an area where humans still have an edge. The rest of us will adapt to the invasion of question-answering Watsons by focusing on work at which the human brain excels -- and will leave the rest to machines.
We've already outsourced long division, spelling and much of our highway navigation to machines. Now we'll look to them more and more to dig through mountains of data and come up with answers for us.
This should free us up to do what remains uniquely human, at least for now -- generating fresh ideas.
Stephen Baker is the author of the about-to-be-published "Final Jeopardy: Man vs. Machine and the Quest to Know Everything."
Show commenting policy
TribLive commenting policy
You are solely responsible for your comments and by using TribLive.com you agree to our Terms of Service.
We moderate comments. Our goal is to provide substantive commentary for a general readership. By screening submissions, we provide a space where readers can share intelligent and informed commentary that enhances the quality of our news and information.
While most comments will be posted if they are on-topic and not abusive, moderating decisions are subjective. We will make them as carefully and consistently as we can. Because of the volume of reader comments, we cannot review individual moderation decisions with readers.
We value thoughtful comments representing a range of views that make their point quickly and politely. We make an effort to protect discussions from repeated comments either by the same reader or different readers.
We follow the same standards for taste as the daily newspaper. A few things we won't tolerate: personal attacks, obscenity, vulgarity, profanity (including expletives and letters followed by dashes), commercial promotion, impersonations, incoherence, proselytizing and SHOUTING. Don't include URLs to Web sites.
We do not edit comments. They are either approved or deleted. We reserve the right to edit a comment that is quoted or excerpted in an article. In this case, we may fix spelling and punctuation.
We welcome strong opinions and criticism of our work, but we don't want comments to become bogged down with discussions of our policies and we will moderate accordingly.
We appreciate it when readers and people quoted in articles or blog posts point out errors of fact or emphasis and will investigate all assertions. But these suggestions should be sent via e-mail. To avoid distracting other readers, we won't publish comments that suggest a correction. Instead, corrections will be made in a blog post or in an article.