Tag Archives: IBM Watson

Evaluating Amazon Lex – AI Digital Agent / Assistant Implementation

June 1, 2017 Ian Roseman Leave a comment

Evaluating AI chatbot solutions for:

Simple to Configure – e.g. Wizard Walkthrough
Flexible, and Mature Platform e.g. Executing backend processes
Cost Effective and Competitive Solutions
Rapid Deployment to XYZ platforms

The idea is almost anyone can build and deploy a chat bot for your business, small to midsize organizations.

Amazon Lex

Going through the Amazon Lex build chat process, and configuration of the Digital Assistant was a breeze. AWS employs a ‘wizard’ style interface to help the user build the Chatbot / Digital Agent. The wizard guides you through defining Intents, Utterances, Slots, and Fulfillment.

Intents – A particular goal that the user wants to achieve (e.g. book an airline reservation)
Utterances – Spoken or typed phrases that invoke your intent
Slots – Data the user must provide to fulfill the intent
Prompts – Questions that ask the user to input data
Fulfillment – The business logic required to fulfill the user’s intent (i.e. backend call to another system, e.g. SAP)

The Amazon Lex Chatbot editor is also extremely easy to use, and to update / republish any changes.

The challenge with Amazon Lex appears to be a very limiting ability for chatbot distribution / deployment. Your Amazon Lex Chatbot is required to use one of three methods to deploy: Facebook, Slack, or Twilio SMS. Facebook is limiting in a sense if you do not want to engage your customers on this platform. Slack is a ‘closed’ framework, whereby the user of the chat bot must belong to a Slack team in order to communicate. Finally, Twilio SMS implies use of your chat bot though a mobile phone SMS.

I’ve reached out to AWS Support regarding any other options for Amazon Lex chatbot deployment. Just in case I missed something.

There is a “Test Bot” in the lower right corner of the Amazon Lex, Intents menu. The author of the business process can, in real-time, make changes to the bot, and test them all on the same page.

Key Followups

Is there a way to leverage the “Test Bot” as a “no frills” Chatbot UI, and embed it in an existing web page? Question to AWS Support.
One concern is for large volumes of utterances / Intents and slots. An ideal suggestion would allow the user a bulk upload through an Excel spreadsheet, for example.
I’ve not been able to utilize the Amazon Lambda to trigger server side processing.
Note: there seem to be several ‘quirky’ bugs in the Amazon Lex UI, so it may take one or two tries to workaround the UI issue.

IBM Watson Conversation also contends for this Digital Agent / Assistant space, and have a very interesting offering including dialog / workflow definition.

Both Amazon Lex and IBM Watson Conversation are FREE to try, and in minutes, you could have your bots created and deployed. Please see sites for pricing details.

Business, Economy, Education, Entertainment, Healthcare, Technology, Work

AI Digital Assistant verse Search Engines

March 8, 2017 Ian Roseman Leave a comment

Aren’t AI Digital Assistants just like Search Engines? They both try to recognize your question or human utterance as best as possible to serve up your requested content. E.g.classic FAQ. The difference in the FAQ use case is the proprietary information from the company hosting the digital assistant may not be available on the internet.

Another difference between the Digital Assistant and a Search Engine is the ability of the Digital Assistant to ‘guide’ a person through a series of questions, enabling elaboration, to provide the user a more precise answer.

The Digital Assistant may use an interactive dialog to guide the user through a process, and not just supply the ‘most correct’ responses. Many people have flocked to YouTube for instructional type of interactive medium. When multiple workflow paths can be followed, the Digital Assistant has the upper hand.

The Digital Assistant has the capability of interfacing with 3rd parties (E.g. data stores with API access). For example, there may be a Digital Assistant hosted by Medical Insurance Co that has the ability to not only check the status of a claim, but also send correspondence to a medical practitioner on your behalf. A huge pain to call the insurance company, then the Dr office, then the insurance company again. Even the HIPPA release could be authenticated in real time, in line during the chat. A digital assistant may be able to create a chat session with multiple participants.

Digital Assistants overruling capabilities over Search Engines are the ability to ‘escalate’ at any time during the Digital Assistant interaction. People are then queued for the next available human agent.

There have been attempts in the past, such as Ask.com (originally known as Ask Jeeves) is a question answering-focused e-business. Google Questions and Answers (Google Otvety, Google Ответы) was a free knowledge market offered by Google that allowed users to collaboratively find good answers, through the web, to their questions (also referred as Google Knowledge Search).

My opinions are my own, and do not reflect my employer’s viewpoint.

Business, Development, Economy, Entertainment, Finance, Media, Mobility, Music, Technology

AI Personal Assistant Needs Remedial Guidance for their Users

September 6, 2016 Ian Roseman Leave a comment

Providing Intelligent ‘Code’ Completion

At this stage in the application platform growth and maturity of the AI Personal Assistant, there are many commands and options that common users cannot formulate due to a lack of knowledge and experience. Using Natural Language to formulate questions has gotten better over the years, but assistance / guidance formulating the requests would maximize intent / goal accuracy.

A key usability feature for many integrated development environments (IDE) are their capability to use “Intelligent Code Completion” to guide their programmers to produce correct, functional syntax. This feature also enables the programmer to be unburdened by the need to look up syntax for each command reference, saving significant time. As the usage of the AI Personal Assistant grows, and their capabilities along with it, the amount of commands and their parameters required to use the AI Personal Assistant will also increase.

AI Leveraging Intelligent Command Completion

For each command parameter [level\tree], a drop down list may appear giving users a set of options to select for the next parameter. A delimiter such as a period(.) indicates to the AI Parser another set of command options must be presented to the person entering the command. These options are typically in the form of drop down lists concatenated to the right of the formulated commands. Vocally, parent / child commands and parameters may be supplied in a similar fashion.

AI Personal Assistant Language Syntax

Adding another AI parser on top of the existing syntax parser may allow commands like these to be executed:

Abstraction (e.g. no application specified)
- Order.Food.Focacceria.List123
- Order.Food.FavoriteItalianRestaurant.FavoriteLunchSpecial
Application Parser
- Seamless.Order.Food.Focacceria.Large Pizza

These AI command examples uses a hierarchy of commands and parameters to perform the function. One of the above commands leverages one of my contacts, and a ‘List123’ object. The ‘List123’ parameter may be a ‘note’ on my Smartphone that contains a list of food we would like to order. The command may place the order either through my contact’s email address, fax number, or calling the business main number and using AI Text to Speech functionality.

All personal data, such as Favorite Italian Restaurant, and Favorite Lunch Special could be placed in the AI Personal Assistant ‘Settings’. A group of settings may be listed as Key-Value pairs, that may be considered short hand for conversations involving the AI Assistant.

A majority of users are most likely unsure of many of the options available within the AI Personal assistant command structure. Intelligent command [code] completion empowers users with visibility into the available commands, and parameters.

For those without a programming background, Intelligent “Command” Completion is slightly similar to the autocomplete in Google’s Search text box, predicting possible choices as the user types. In the case of the guidance provided by an AI Personal Assistant the user is guided to their desired command; however, the Google autocomplete requires some level or sense of the end result command. Intelligent code completion typically displays all possible commands in a drop down list next to the constructor period (.). In this case the user may have no knowledge of the next parameter without the drop down choice list. An addition feature enables the AI Personal Assistant to hover over one of the commands\parameters to show a brief ‘help text’ popup.

Note, Microsoft’s Cortana AI assistant provides a text box in addition to speech input. Adding another syntax parser could be allowed and enabled through the existing User Interface. However, Siri seems to only have voice recognition input, and no text input.

Is Siri handling the iOS ‘Global Search’ requests ‘behind the scenes’? If so, the textual parsing, i.e. the period(.) separator would work. Siri does provide some cursory guidance on what information the AI may be able to provide, “Some things you can ask me:”

With only voice recognition input, use the Voice Driven Menu Navigation & Selection approach as described below.

Voice Driven, Menu Navigation and Selection

The current AI personal assistant, abstraction layer may be too abstract for some users. The difference between these two commands:

Play The Rolling Stones song Sympathy for the Devil.
- Has the benefit of natural language, and can handle simple tasks, like “Call Mom”
- However, there may be many commands that can be performed by a multitude of installed platform applications.

Verse

Spotify.Song.Sympathy for the Devil
- Enables the user to select the specific application they would like a task to be performed by.
Spotify Help
- A voice driven menu will enable users to understand the capabilities of the AI Assistant. Through the use of a voice interactive menu, users may ‘drill down’ to the action they desire to be performed. e.g. “Press # or say XYZ”
- Optionally, the voice menu, depending upon the application, may have a customer service feature, and forward the interaction to the proper [calling or chat] queue.

Update – 9/11/16

I just installed Microsoft Cortana for iOS, and at a glance, the application has a leg up on the competition
- The Help menu gives a fair number of examples by category. Much better guidance that iOS / Siri
- The ability to enter\type or speak commands provides the needed flexibility for user input.
  - Some people are uncomfortable ‘talking’ to their Smartphones. Awkward talking to a machine.
  - The ability to type in commands may alleviate voice command entry errors, speech to text translation.
  - Opportunity to expand the AI Syntax Parser to include ‘programmatic’ type commands allows the user a more granular command set, e.g. “Intelligent Command Completion”. As the capabilities of the platform grow, it will be a challenge to interface and maximize AI Personal Assistant capabilities.

Business, Development, Education, Technology, Work

Microsoft OneDrive Cloud Repository for MS Office Files with Metadata \ Tags

June 21, 2016 Ian Roseman Leave a comment

Review of Microsoft OneDrive Cloud Repository. It may be an easy tool and service(s) to save files. If you know what you roughly want to find, most Cloud repositories are easy and straight forward to use. Over time, if not managed appropriately, the cloud repository becomes burdensome to manage, e.g. access and find files. If stuck in the “file folder organization storage” mentality of organizing our content, our Cloud storage solution will become quickly unyielding. Getting into habits like tagging your content should help us to access files beyond the “Folder Borders”. To the contrary, there are huge opportunities to leverage and grow existing platforms, specifically around the process service of [file] Ingestion.

Challenges:

Bulk file loading, e.g. photos from our smartphones, maybe the entire family uploads to the same storage repository
If performed by the “Ingestion Service”, manual user “tagging” of a group of photos, or individual images may be available.
Geotagging may be available either at the time of image capture , or upon the start of the “Ingestion Service”
Facial Recognition, compared to the likes of services such as Facebook, based on my experience, are not readily available to personal Cloud Storage repositories.
Auto tagging pictures upon ingestion, if performed, may leverage “Extracted Text” from images. Images become searchable with little human intervention.

Cloud File Repository: Storing Content

I created modified existing Microsoft Office files”tags”, in this case MS Word and PowerPoint file types were used. I opened the Word file, and selected “File” menu, “Save As” menu, then “More Options” under the list of file types. I was then presented with the classic “Save As” form. Just below the “Save as type” list box, there were 3 “metadata” fields to describe the file:

Authors:
Tags:
Title:

The first two fields are semi colon ; delimited and multiple values are allowed. In this test case, I added to the “Tags” field “CV;resume;career”. I then used the MS Windows Snipping Tool that comes with the OS to document the step. I called the file MSWordTags.PNG and saved this screen capture to my OneDrive. Then I saved the document itself on my OneDrive.

Cloud File Repository: Finding Content

I then started up Internet Explorer, and went to the https://onedrive.live.com site to access my cloud content. On the top left corner of the screen, there is a field called “Search Everything”, and I typed in CV.

Results:

The search results included ONLY the image screenshot file that contained the letters CV, and not the MS Word file that explicitly had the Tag field with the text value CV.
Looking at the file properties as defined by OneDrive, there was ALSO a field called “Tags” with no values populated. For example, the Cloud “Ingestion” service did not read the file for metadata, and abstract it to the Cloud level. just two separate sets of metadata describing the same file. To view the Cloud file data, select the file, and there is an i with a circle around it. Too many ways to store the same data, and may lead to inconsistent data.
For the Cloud file information / properties, the image file had a field called “Extracted Text”, and this is how the search picked up the CV value in the Cloud Search for my files with the “CV” tag.
Oddly, the MS Word file attributes in OneDrive did not offer “tags” as a field to store meta data in the cloud. The “tags” field was available when looking at the PNG file. However, the user may add a “Description” in a multiline text field. Tags metadata on images and not MS Word files? Odd.
Future State (?): If the Cloud Ingestion process can perform an “Extracted Text” process, it may also have other “Ingestion services”, such as “Facial Recognition” from “known good” faces already tagged. e.g. I tag a face from within the OneDrive browser UI, and now when other images are ingested, there can be a correlation between the files.

As a business model, are we going to add a tier just after Cloud File ingestion, maybe exercise a third party suite of cognitive APIs, such as facial recognition? For example, Microsoft OneDrive Ingests a file, and if it’s an image file, routes through to the appropriate IBM Watson API, processes the file, and returns [updated] metadata, and a modified file? Maybe.

Update: Auto Tagging Objects Upon Ingestion

On an image with no tags, I selected the “Edit tags” menu from the Properties pane on the right side of the screen. As a scrolling menu, the option to “Add existing tag” appeared. There were dozens of tags already created with a word, thumbnail image, and the number of times used. Wow. Awesome. The current implementation seems to automatically, upon ingestion, identify objects in the image, and tag the images with those objects, e.g. Building, Beach, Horse, etc.

Presumption that Microsoft OneDrive performs object recognition on images upon file ingestion into the cloud (as opposed to in the Photos app).

“Extracted Text ” Metadata Field from within Microsoft OneDrive Image PNG File Properties:

Presumption that Microsoft OneDrive performs OCR on images upon file ingestion into the cloud (as opposed to the Photos app).

OneDrive Search Results - Extracted Text — OneDrive Search Results – Extracted Text

Advertising, Business, Child Development, Education, Entertainment, Internal Information, Legal, Media, Reviews, Technology

AI Assistant Summarizing Email Threads and Complex Documents

June 6, 2016 Ian Roseman Leave a comment

“Give me the 50k foot level on that topic.”

“Just give us the cliff notes.”
“Please give me the bird’s eye view.”

AI Email Thread Abstraction and Summarization

A daunting, and highly public email has landed in your lap..top to respond. The email thread goes between over a dozen people all across the globe. All of the people on the TO list, and some on the CC list, have expressed their points about … something. There are junior technical and very senior business staff on the email. I’ll need to understand the email thread content from the perspective of each person that replied to the thread. That may involve sifting through each of the emails on the thread. Even though the people on the emails are English fluent, their response styles may be different based on culture, or seniority of staff (e.g. abstractly written). Also, the technical folks might want to keep the conversation of the email granular and succinct.

Let’s throw a bit of [AI] automation at this problem.

Another step in our AI personal assistant evolution, email thread aggregation and summarization utilizing cognitive APIs | tools such as what IBM Watson has implemented with their Language APIs. Based on the documentation provided by their APIs, the above challenges can be resolved for the reader. A suggestion to an IBM partner for the Watson Cognitive cloud, build an ’email plugin’ if the email product exposes their solution to customization.

A plugin built on top of an email application, flexible enough to allow customization, may be a candidate for Email Thread aggregation and summarization. Email clients may include IBM Notes, Gmail, (Apple) Mail, Microsoft Outlook, Yahoo! Mail, and OpenText FirstClass.

Add this capability to the job description of AI assistants, such as Cortana, Echo, Siri, and Google Now. In fact, this plug-in may not need the connectivity and usage of an AI assistant, just the email plug-in interacting with a suite of cognitive cloud API calls.

AI Document Abstraction and Summarization

A plug in may also be created for word processors such as Microsoft Word. Once activated within a document, a summary page may be created and prefixed to the existing document. There are several use cases, such as a synopsis of the document.

With minimal effort from human input, marking up the content, we would still be able to derive the contextual metadata, and leverage it to create new sentences, paragraphs of sentences.

Update:

I’ve not seen an AI Outlook integration in the list of MS Outlook Add-ins that would bring this functionality to users.

Business, Development, Economy, Education, Finance, Mobility, Retail, Science, Science Theory, Technology, Work

Building AI Is Hard—So Facebook Is Building AI That Builds AI

May 12, 2016 Ian Roseman Leave a comment

“…companies like Google and Facebook pay top dollar for some really smart people. Only a few hundred souls on Earth have the talent and the training needed to really push the state-of-the-art [AI] forward, and paying for these top minds is a lot like paying for an NFL quarterback. That’s a bottleneck in the continued progress of artificial intelligence. And it’s not the only one. Even the top researchers can’t build these services without trial and error on an enormous scale. To build a deep neural network that cracks the next big AI problem, researchers must first try countless options that don’t work, running each one across dozens and potentially hundreds of machines.”

Source: Building AI Is Hard—So Facebook Is Building AI That Builds AI

This article represents a true picture of where we are today for the average consumer and producer of information, and the companies that repurpose information, e.g. in the form of advertisements.

The advancement and current progress of Artificial Intelligence, Machine Learning, analogously paints a picture akin to the 1970s with computers that fill rooms, and accept punch cards as input.

Today’s consumers have mobile computing power that is on par to the whole rooms of the 1970s; however, “more compute power” in a tinier package may not be the path to AI sentience. How AI algorithm models are computed might need to take an alternate approach.

In a classical computation system, a bit would have to be in one state or the other. However quantum mechanics allows the qubit to be in a superposition of both states at the same time, a property which is fundamental to quantum computing.

The construction, and validation of Artificial Intelligence, Machine Learning, algorithm models should be engineered on a Quantum Computing framework.

Advertising, Business, Child Development, Development, Economy, Education, Entertainment, Finance, Help, Mobility, Retail, Technology

AI Personal Assistants are “Life Partners”

April 21, 2016 Ian Roseman Leave a comment

Artificial Intelligent (AI) “Assistants”, or “Bots” are taken to the ‘next level’ when the assistant becomes a proactive entity based on the input from human intelligent experts that grows with machine learning.

Even the implication of an ‘Assistant’ v. ‘Life Partner’ implies a greater degree of dynamic, and proactive interaction. The cross over to becoming ‘Life Partner’ is when we go ‘above and beyond’ to help our partners succeed, or even survive the day to day.

Once we experience our current [digital, mobile] ‘assistants’ positively influencing our lives in a more intelligent, proactive manner, an emotional bond ‘grows’, and the investment in this technology will also expand.

Practical Applications Range:

Alcoholics Anonymous Coach , Mentor – enabling the human partner to overcome temporary weakness. Knowledge, and “triggers” need to be incorporated into the AI ‘Partner’; “Location / Proximity” reminder if person enters a shopping area that has a liquor store. [AI] “Partner” help “talk down”
Understanding ‘data points’ from multiple sources, such as alarms, and calendar events, to derive ‘knowledge’, and create an actionable trigger.
- e.g. “Did you remember to take your medicine?” unprompted; “There is a new article in N periodical, that pertains to your medicine. Would you like to read it?”
- e.g. 2 unprompted, “Weather calls for N inches of Snow. Did you remember to service your Snow Blower this season?”
FinTech – while in department store XYZ looking to purchase Y over a certain amount, unprompted “Your credit score indicates you are ‘most likely’ eligible to ‘sign up’ for a store credit card, and get N percentage off your first purchase” Multiple input sources used to achieve a potential sales opportunity.

IBM has a cognitive cloud of AI solutions leveraging IBM’s Watson. Most/All of the 18 web applications they have hosted (with source) are driven by human interactive triggers, as with the “Natural Language Classifier”, which helps build a question-and-answer repository.

There are four bits that need to occur to accelerate adoption of the ‘AI Life Partner’:

Knowledge Experts, or Subject Matter Experts (SME) need to be able to “pass on” their knowledge to build repositories. IBM Watson Natural Language Classifier may be used.
The integration of this knowledge into an AI medium, such as a ‘Digital Assistant’ needs to occur with corresponding ‘triggers’
Our current AI ‘Assistants’ need to become [more] proactive as they integrate into our ‘digital’ lives, such as going beyond the setting of an alarm clock, hands free calling, or checking the sports score. Our [AI] “Life Partner” needs to ‘act’ like buddy and fan of ‘our’ sports team. Without prompting, proactively serve up knowledge [based on correlated, multiple sources], and/or take [acceptable] actions.
1. E.g. FinTech – “Our schedule is open tonight, and there are great seats available, Section N, Seat A for ABC dollars on Stubhub. Shall I make the purchase?”
  1. Partner with vendors to drive FinTech business rules.
Take ‘advantage’ of more knowledge sources, such as the applications we use that collect our data. Use multiple knowledge sources in concert, enabling the AI to correlate data and propose ‘complex’ rules of interaction.

Our AI ‘Life Partners’ may grow in knowledge, and mature the relationship between man and machine. Incorporating derived rules leveraging machine learning, without input of a human expert, will come with risk and reward.

Advertising, Business, Development, Economy, Education, Entertainment, Fashion, Finance, Fun, Law, Retail, Science, Technology

The Race Is On to Control Artificial Intelligence, and Tech’s Future

March 25, 2016 Ian Roseman Leave a comment

Amazon, Google, IBM and Microsoft are using high salaries and games pitting humans against computers to try to claim the standard on which all companies will build their A.I. technology.

In this fight — no doubt in its early stages — the big tech companies are engaged in tit-for-tat publicity stunts, circling the same start-ups that could provide the technology pieces they are missing and, perhaps most important, trying to hire the same brains.

For years, tech companies have used man-versus-machine competitions to show they are making progress on A.I. In 1997, an IBM computer beat the chess champion Garry Kasparov. Five years ago, IBM went even further when its Watson system won a three-day match on the television trivia show “Jeopardy!” Today, Watson is the centerpiece of IBM’s A.I. efforts.

Today, only about 1 percent of all software apps have A.I. features, IDC estimates. By 2018, IDC predicts, at least 50 percent of developers will include A.I. features in what they create.

Source: The Race Is On to Control Artificial Intelligence, and Tech’s Future – The New York Times

The next “tit-for-tat” publicity stunt should most definitely be a battle with robots, exactly like BattleBots, except…

Use A.I. to consume vast amounts of video footage from previous bot battles, while identifying key elements of bot design that gave a bot the ‘upper hand’. From a human cognition perspective, this exercise may be subjective. The BattleBot scoring process can play a factor in 1) conceiving designs, and 2) defining ‘rules’ of engagement.
Use A.I. to produce BattleBot designs for humans to assemble.
Autonomous battles, bot on bot, based on Artificial Intelligence battle ‘rules’ acquired from the input and analysis of video footage.

Roseman Solutions LLC