FirstStar Systems | The Leaders in Strong AI

Why I believe IBM Will Re-Brand Watson August 8, 2016

By John W. Bottoms, FirstStar Systems, Concord, MA

Watson, IBM’s Artificially Intelligent system has been in the news once again. The first news of Watson came with the Jeopardy game show win over Jeopardy Champions Ken Jennings and Brad Rutter. According to Ken, “I felt obsolete,” he reveals. “I felt like a Detroit factory worker in the ‘80’s seeing a robot that could now do his job on the assembly line.”

Ken shouldn’t feel bad; the process of machines overtaking humans has long been a subject for folklore. Before Ken’s battle with Watson, there was the battle between the world’s fastest typists and a computer typist. And going further back we have the legendary story of coal miner John Henry’s fatal battle with a steam driven engine. What we learn from this is that machines are becoming more capable and they will generally surpass humans in their designed function. The happy entrepreneur is the one who can see when they are closely skilled, and that is when the contest battle can be covered for the press, or the music publishers.

In the case of Watson, IBM has aggressively sold the ability of the system without much discussion about how the system works. The press headlines describe how smart Watson is while downplaying the role of those who provide the assistance and guidance. In the case of the Jeopardy match, certain rules were disallowed to provide an advantage to Watson that would have otherwise been fatal. Specifically, certain types of questions containing images or sound were disallowed. Perhaps the match should have been called “Jeopardy2” vs Ken and Brad.

We see headlines such as, “IBM computer Watson wins Jeopardy Clash”. Then, “Artificial Intelligence Course Creates AI Teaching Assistant” and “Watson correctly diagnoses woman after doctors were stumped.”

Perhaps these zealous headlines were written by overly enthusiastic editors. Or, they may have been suggested by IBM’s marketing department. Instead, we have to take the responsibility of drilling down to get the real story.

Here’s the gist of why “Why I believe IBM Will Re-Brand Watson.” My argument derives from a number of points, all of which should be taken seriously if Watson is to be accepted as a refined system capable of working alongside professionals.

Issues Concerning Watson’s Image

Governments around the world take doctors’ credentials seriously. Recently a German court sentenced a man who falsely claimed to be a doctor and treated patients on a cruise ship to three years in prison (http://www.travelweek.ca/news/bogus-cruise-ship-doctor-sentenced-three-years-prison/).

Further, if Watson has passed the medical boards, then publish the results or the accreditation and license. Is it working in a specialty or is it capable of working at a GP? If it has not passed boards and licensing requirements then it is not appropriate to attribute the skills of a doctor to an AI system.

Of concern by humans worldwide is that while the medical version of Watson is a described as being able to diagnose certain diseases, there have as yet, been no reports of the successful diagnosis rate vs. false positives or false negatives. This information should be provided along with a set of reproducible results test data. Then others can verify in double blind experiments or A/B tests that Watson performs as described. Some are even calling for parts of the programming of Watson to be made open source so that it can be examined.

Finally, there are those who ask, “When Watson is described as performing a task, who is actually doing the work? Is it Watson or the humans behind the scenes? In the case of Jill at Georgia Tech, there were eight humans working in cooperation with Watson. For any given response by the Jill system, the answer could have come from one of the behind-the-scenes graduate students, or even the course professor. Are we to believe Jill answered 100% of the queries submitted or was it 12%? We have not seen the breakdown of what work was done by humans and what was provided by Jill, and yet Jill gets the credit for the project.

Of all the scenarios discussed here, the medical one is the most troubling. We take our medical diagnoses seriously. It was IBM who termed the system “Cognitive Computing”, without much detail provided to accompany that term.

Some employees of IBM have gone public” one has observed that, “As someone who’s worked on/with teams building various Watson services/solutions over the past 3 years, I can confirm Watson is almost entirely a branding exercise without (from the engineering perspective) common leadership, coherent inter-team objectives, etc. There isn’t a ‘unified approach’ to anything here beyond the use of various open source machine learning libraries with huge amounts of medical data either licensed or acquired over the past 2 years.”(https://news.ycombinator.com/item?id=12240253, Krona)

Why “Cooperative Computing” is better than “Cognitive Computing”

Please do not mistake my intentions here. All of these arguments are valid concerns about how Watson is portrayed in the press. We are being told that Watson is better than Ken and Brad at Jeopardy, and better than doctors in diagnosing certain cancers. All of these seek to place Watson on a pedestal, while it would be more beneficial to describe Watson as a cooperative team member who listens to and advises doctors.

They have moved from the description of a human augmented system to an AI. without qualification. Similar techniques were used in Watson’s configuration of the Jeopardy playing field. The goalposts were moved by removing the human advantage of vision and music processing. This was done to assure that Watson would win the Jeopardy challenge.

Please don’t mistake my criticisms for disparaging remarks. To the contrary, I believe that supervised learning is the most valuable approach to improvements in knowledge systems. That is exactly how we ground and train children as they grow. I just wish that IBM had called it “Cooperative Computing” rather than “Cognitive Computing.”

Perhaps the current approach is simply a marketing decision that was planned all along. The plan was to develop Watson to capture a market, and then the gods could come down from Olympus and interact with humans. On the other hand, maybe IBM is afraid that the mixing of the blood of the gods with those of humans will result in demigods that just cause more problems than they solve. Either way you look at it, it will continue being a good show.