Category Archives: Digital Assistant

Agile Advisor Plugin for Microsoft Teams

Estimated reading time: 3 minutes

Advisory Role in Microsoft Team Communications

Agile Advisor Plugin for #Microsoft Teams is able to observe team interactions, such as conference calls within Microsoft Teams. The Advisor can derive “dialog intents” and provide recommendations for improvement. A retrospective on communications, such as Scrum ceremonies

Voice Recognition During Teams Meetings

Technology that leverages voice recognition, such as Interactive voice response (IVR) solutions are fraught with failed recognition. IVRs are used to answer calls in just about every company, which prompts for either a phrase from the user on what they want and the ability to enter a numeric value correlating to the desired intent. Challenge #1.

Dialog and Intent Identification

Beyond trying to identify the user’s intent from a phrase or sentence, a dialog, a series of interactions between two or more team members is even more complex. Current AI models that identify intent from a sentence or phrase have a mixed variable of accuracy, which is why these models must be tuned over time. A collective of interactions, a dialog between two or more team members, has a much higher level of complexity to identify intents. Challenge #2. Once a dialog intent(s) has an “N”% level of accuracy, rules may be fired with any number of outcomes, such as unintrusive logging of Agile suggestions for best practices, and next steps: e.g. a retrospective of the scrum ceremony.

Dynamically Identify Roles in Teams Meetings

Who participates in Microsoft Teams meetings and team chats can be associated with Microsoft Teams’ member profiles, such as Scrum Master, and Product Owners.

Enhance the Adherence to Agile Principles

12 Principles Behind the Agile Manifesto, and opportunities for rules to be trigger based on conversations, the interactive dialogs.

  1. Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
  2. Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
  3. Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
  4. Business people and developers must work together daily throughout the project.
  5. Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
  6. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
  7. Working software is the primary measure of progress.
  8. Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
  9. Continuous attention to technical excellence and good design enhances agility.
  10. Simplicity–the art of maximizing the amount of work not done–is essential.
  11. The best architectures, requirements, and designs emerge from self-organizing teams.
  12. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

Dialog Intent Rules for Agile Guidance

From the above agile principles, we can derive the following dialog intents and precise recommendations for improvement.

Barriers to Implementation

At the current level of Artificial Intelligence (AI) Digital Assistants, i.e. chatbots, even the “best in breed”, has “difficulty”, i.e. lower probability with intent recognition, with a single sentence or phrase. Multiply that by interpreting an interactive dialog with multiple sentences, multiple participants, and exchange of responses, feasibility is highly speculative.

And Still More Opportunity: Recognition of Facial Emotional Expressions 

Expressions of people may be able to be determined, and opportunities for suggestive posture can be advised. Even body posture folded arms as an example, can imply a guarded opinion, and not open to compromise.

Reference article – Emotion recognition using facial expressions

Caution and Opportunities

This plugin output could be used for annual employee evaluations.

Help Wanted: Civil War Reenactment Soldiers to Improve AI Models

I just read an article on Digital PC Magazine, “Human Help Wanted: Why AI Is Terrible at Content Moderation” which started to get my neurons firing.

Problem Statement

Every day, Facebook’s artificial intelligence algorithms tackle the enormous task of finding and removing millions of posts containing spam, hate speech, nudity, violence, and terrorist propaganda. And though the company has access to some of the world’s most coveted talent and technology, it’s struggling to find and remove toxic content fast enough.

Ben Dickson
July 10, 2019 1:36PM EST

I’ve worked at several software companies which leveraged Artifical Intelligence, Machine Learning to recognize patterns, correlations. The larger the data sets, in general, the higher the accuracy of the predictions. The outliers in the data, the noise, “falls out” of the data set. Without quality, large training data, Artificial Intelligence makes more mistakes.

In terms of speech recognition, image classification, and natural language processing (NLP), in general, programs like chatbots, digital assistants, are becoming more accurate because of their sample size, training data sets are large, and there is no shortage of these data types. For example, there are many ways I can ask my digital assistant for something, like “Get the movie times”. Training a digital assistant, at a high level, would be to catalog how many ways can I ask for “something”, achieve my goal. I can go and create that list. I could write a few dozen questions, but still, my sample data set would be too small. Amazon has a crowdsourcing platform, Amazon Mechanical Turk, which I can request they build me the data sets, thousands of questions, and correlated goals.

MTurk enables companies to harness the collective intelligence, skills, and insights from a global workforce to streamline business processes, augment data collection and analysis, and accelerate machine learning development.

Amazon Mechanical Turk: Access a global, on-demand, 24×7 workforce

Video “Scene” Recognition – Annotated Data Sets for a Wide Variety of Scene Themes

In silent films, the plot was conveyed by the use of title cards, written indications of the plot and key dialogue lines. Unfortunately, silent films are not making a comeback. In order to achieve a high rate of successful identification of activities within a given video clip, video libraries of metadata need to be created, that capture:

  • Media / Video Asset, Unique Identifier
  • Scene Clip IN and OUT timecodes
  • Scene Theme(s), similar to Natural language processing (NLP), Goals = Utterances / Sentences
    • E.g. Man drinking water; Woman playing Tennis
  • Image recognition, in the context of machine vision, is the ability of software to identify objects, places, people, writing and actions in images. Image recognition is used to perform a large number of machine-based visual tasks, such as labeling the content of images with meta-tags

Not Enough Data

Here is an example of how Social Media, such as Facebook, attempts to deal with video deemed inappropriate for their platform:

In March, a shooter in New Zealand live-streamed the brutal killing of 51 people in two mosques on Facebook. But the social-media giant’s algorithms failed to detect the gruesome video. It took Facebook an hour to take the video down, and even then, the company was hard-pressed to deal with users who reposted the video.

Ben Dickson
July 10, 2019 1:36PM EST

…in many cases, such as violent content, there aren’t enough examples to train a reliable AI model. “Thankfully, we don’t have a lot of examples of real people shooting other people,” Yann LeCun, Facebook’s chief artificial-intelligence scientist, told Bloomberg.

Ben Dickson
July 10, 2019 1:36PM EST

Opportunities for Actors and Curators of Video Content: Dramatizations

All those thousands of people who perform, creating videos of content that range the gamut from playing video games to “unboxing” collectible items. The actors who perform dramatizations could add tags to their videos indicating as per above, documenting themes for a given skit. If actors post their videos on YouTube or proprietary crowdsourcing platforms, they would be entitled to some revenue for the use of their licensed video.

Disclosure Regarding Flag Controversy

I now realize there are politics around Nike “tipping their hat” toward the Betsy Ross flag. However, when I referenced the flag in this blog post, I was thinking of the American Revolution, and the 13 colonies flag. I didn’t think the title would resonate with readers, “Help Wanted: Amerian Revolutionary war Reenactment Soldiers to Improve AI Models.”, so I took some creative liberty.

As your Digital Assistant, Siri Will Answer Incoming Calls

Voice mail is so LAST Century. It’s a static communications interface to address your incoming phone calls. It’s a dinosaur in terms of communications protocol. Yes, a digital assistant, or chat bots should “field” your incoming calls, providing your callers a higher level of service.

Business or Personal?

Why not both? There are use cases which highlight the value of a Digital Assistant answering your phone calls when you’re unavailable.

Trusted Friends and Business Pins

Level of available services may change based upon the level of trusted access, such as:

  • Friends Seeking Your Availability for a Hockey Game Next Week
  • Business Partners Sharing Information access such as invoices

Untrusted Caller Access

  • The Vetting of Unsolicited Calls, such as robocalls

Defining and Default Dialogs

Users can define dialogs through drop and drag workflow diagram tools making it easy to “build” conversations / dialogs flows. In addition, out of the box flows can provide administrators with opportunities and discover the ways in which AI digital assistant may be leveraged.

Canned / Default dialog templates to handle the most common dialogs / workflows will empower users to the implement rapidly.

Any Acquisitions in the Pipeline?

Are the big names in the Digital Assistant space looking to partner or acquire tools that can easily transform workflows to be leveraged by digital assistant?

  • IBM’s Conversations – chatbot dialog definition tool
  • Interactive Voice Response (IVR) solutions

APIs available on Mobile OS SDKs?

Are the components available for third party product companies to extend the Mobile OS capabilities as of now? Or are the mobile OS companies the only ones in a possession of performing these upgrades?

People Turn Toward “Data Banks” to Commoditize on their Purchase and User Behavior Profiles

Anyone who is anti “Big Brother”, this may not be the article for you, in fact, skip it. 🙂

 

The Pendulum Swings Away from GDPR

In the not so distant future, “Data Bank” companies consisting of Subject Matter Experts (SME) across all verticals,  may process your data feeds collected from your purchase and user behavior profiles.  Consumers will be encouraged to submit their data profiles into a Data Bank who will offer incentives such as a reduction of insurance premiums to cash back rewards.

 

Everything from activity trackers, home automation, to vehicular automation data may be captured and aggregated.    The data collected can then be sliced and diced to provide macro and micro views of the information.    On the abstract, macro level the information may allow for demographic, statistical correlations, which may contribute to corporate strategy. On a granular view, the data will provide “data banks” the opportunity to sift through data to perform analysis and correlations that lead to actionable information.

 

Is it secure?  Do you care if a hacker steals your weight loss information? May not be an issue if collected Purchase and Use Behavior Profiles aggregate into a Blockchain general ledger.  Data Curators and Aggregators work with SMEs to correlate the data into:

  • Canned, ‘intelligent’ reports targeted for a specific subject matter, or across silos of data types
  • ‘Universes’ (i.e.  Business Objects) of data that may be ‘mined’ by consumer approved, ‘trusted’ third party companies, e.g. your insurance companies.
  • Actionable information based on AI subject matter rules engines and consumer rule transparency may be provided.

 

 “Data Banks” may be required to report to their customers who agreed to sell their data examples of specific rows of the data, which was sold on a “Data Market”.

Consumers may have the option of sharing their personal data with specific companies by proxy, through a ‘data bank’ granular to the data point collected.  Sharing of Purchase and User Behavior Profiles:

  1. may lower [or raise] your insurance premiums
  2. provide discounts on preventive health care products and services, e.g. vitamins to yoga classes
  3. Targeted, affordable,  medicine that may redirect the choice of the doctor to an alternate.  The MD would be contacted to validate the alternate.

 

The curriated data collected may be harnessed by thousands of affinity groups to offer very discrete products and services.  Purchase and User Behavior Profiles,  correlated information stretches beyond any consumer relationship experienced today.

 

At some point, health insurance companies may require you to wear a tracker to increase or slash premiums.  Auto Insurance companies may offer discounts for access to car smart data to make sure suggested maintenance guidelines for service are met.

 

You may approve your “data bank” to give access to specific soliciting government agencies or private firms looking to analyze data for their studies. You may qualify based on the demographic, abstracted data points collected for incentives provided may be tax credits, or paying studies.

Purchase and User Behavior Profiles:  Adoption and Affordability

If ‘Data Banks’ are allowed to collect Internet of Things (IoT) device profile and the devices themselves are cost prohibitive.  here are a few ways to increase their adoption:

  1.  [US] tax coupons to enable the buyer, at the time of purchase, to save money.  For example, a 100 USD discount applied at the time of purchase of an Activity Tracker, with the stipulation that you may agree,  at some point, to participate in a study.
  2. Government subsidies: the cost of aggregating and archiving Purchase and Behavioral profiles through annual tax deductions.  Today, tax incentives may allow you to purchase an IoT device if the cost is an itemized medical tax deduction, such as an Activity Tracker that monitors your heart rate, if your medical condition requires it.
  3. Auto, Life, Homeowners, and Health policyholders may qualify for additional insurance deductions
  4. Affinity branded IoT devices, such as American Lung Association may sell a logo branded Activity Tracker.  People may sponsor the owner of the tracking pedometer to raise funds for the cause.

The World Bank has a repository of data, World DataBank, which seems to store a large depth of information:

World Bank Open Data: free and open access to data about development in countries around the globe.”

Here is the article that inspired me to write this article:

http://www.marketwatch.com/story/you-might-be-wearing-a-health-tracker-at-work-one-day-2015-03-11

 

Privacy and Data Protection Creates Data Markets

Initiatives such as General Data Protection Regulation (GDPR) and other privacy initiatives which seek to constrict access to your data to you as the “owner”, as a byproduct, create opportunities for you to sell your data.  

 

Blockchain: Purchase, and User Behavior Profiles

As your “vault”, “Data Banks” will collect and maintain your two primary datasets:

  1. As a consumer of goods and services, a Purchase Profile is established and evolves over time.  Online purchases are automatically collected, curated, appended with metadata, and stored in a data vault [Blockchain].  “Offline” purchases at some point, may become a hybrid [on/off] line purchase, with advances in traditional monetary exchanges, and would follow the online transaction model.
  2. User Behavior (UB)  profiles, both on and offline will be collected and stored for analytical purposes.  A user behavior “session” is a use case of activity where YOU are the prime actor.  Each session would create a single UB transaction and are also stored in a “Data Vault”.   UB use cases may not lead to any purchases.

Not all Purchase and User Behavior profiles are created equal.  Eg. One person’s profile may show a monthly spend higher than another.  The consumer who purchases more may be entitled to more benefits.

These datasets wholly owned by the consumer, are safely stored, propagated, and immutable with a solution such as with a Blockchain general ledger.

Hostess with the Mostest – Apple Siri, Amazon Alexa, Microsoft Cortana, Google Assistant

Application Integration Opportunities:

  • Microsoft Office, Google G Suite, Apple iWork
    • Advice is integrated within the application, proactive and reactive: When searching in Microsoft Edge, a blinking circle representing Cortana is illuminated.  Cortana says “I’ve collected similar articles on this topic.”  If selected, presents 10 similar results in a right panel to help you find what you need.
  • Personal Data Access and Management
    • The user can vocally access their personal data, and make modifications to that data; E.g. Add entries to their Calendar, and retrieve the current day’s agenda.

Platform Capabilities: Mobile Phone Advantage

Strengthen core telephonic capabilities where competition, Amazon and Microsoft, are relatively week.

  • Ability to record conversations, and push/store content in Cloud, e.g. iCloud.  Cloud Serverless recording mechanism dynamically tags a conversations with “Keywords” creating an Index to the conversation.  Users may search recording, and playback audio clips +/- 10 seconds before and after tagged occurrence.
Calls into the User’s Smartphones May Interact Directly with the Digital Assistant
  • Call Screening – The digital assistant asks for the name of the caller, purpose of the call, and if the matter is “Urgent”
    • A generic “purpose” response, or a list of caller purpose items can be supplied to the caller, e.g. 1) Schedule an Appointment
    • The smartphone’s user would receive the caller’s name, and the purpose as a message back to the UI from the call, currently in a ‘hold’ state,
    • The smartphone user may decide to accept the call, or reject the call and send the caller to voice mail.
  • A  caller may ask to schedule a meeting with the user, and the digital assistant may access the user’s calendar to determine availability.  The digital assistant may schedule a ‘tentative’ appointment within the user’s calendar.
    • If calendar indicates availability, a ‘tentative’ meeting will be entered. The smartphone user would have a list of tasks from the assistant, and one of the tasks is to ‘affirm’ availability of the meetings scheduled.
  • If a caller would like to know the address of the smartphone user’s office, the Digital Assistant may access a database of “generally available” information, and provide it. The Smartphone user may use applications like Google Keep, and any note tagged with a label “Open Access” may be accessible to any caller.
  • Custom business workflows may be triggered through the smartphone, such as “Pay by Phone”.  When a caller is calling a business user’s smartphone, the call goes to “voice mail” or “digital assistant” based on smartphone user’s configuration.  If the user reaches the “Digital Assistant”, there may be a list of options the user may perform, such as “Request for Service” appointment.  The caller would navigate through a voice recognition, one of many defined by the smartphone users’ workflows.

Platform Capabilities: Mobile Multimedia

Either through your mobile Smartphone, or through a portable speaker with voice recognition (VR).

  • Streaming media / music to portable device based on interactions with Digital Assistant.
  • Menu to navigate relevant (to you) news,  and Digital Assistant to read articles through your portable media device (without UI)

Third Party Partnerships: Adding User Base, and Expanding Capabilities

In the form of platform apps (abstraction), or 3rd party APIs which integrate into the Digital Assistant, allowing users to directly execute application commands, e.g. Play Spotify song, My Way by Frank Sinatra.

  • Any “Skill Set” with specialized knowledge: direct Q&A or instructional guidance  – e.g Home Improvement, Cooking
  • eCommerce Personalized Experience – Amazon
  • Home Automation – doors, thermostats
  • Music – Spotify
  • Navigate Set Top Box (STB) – e.g. find a program to watch
  • Video on Demand (VOD) – e.g. set to record entertainment

 

Evaluating fobi.io Chatbot Powered By Google Forms: AI Digital Agent?

Interesting approach to an AI Chatbot implementation.  The business process owner creates one or more Google Forms containing questions and answers, and converts/deploys to a chatbot using fobi.io.  All the questions for [potential] customers/users are captured in a multitude of forms.  Without any code, and within minutes, an interactive chatbot can be produced and deployed for client use.

The trade off for rapid deployment and without coding is a rigid approach of triggering user desired “Goal/Intents”.  It seems a single goal/intent is mapped to a single Google Form.  As opposed to a digital agent, which leverages utterances to trigger the user’s intended goal/intent.  Before starting the chat, the user must select the appropriate Google Form, with the guidance of the content curator.

Another trade off is, it seems, no integration on the backend to execute a business process, essential to many chatbot workflows. For example, given an Invoice ID, the chatbot may search in a transactional database, then retrieve and display the full invoice.  Actually, I may be incorrect. On the Google Forms side, there is a Script Editor. Seems powerful and scary all at the same time.

Another trade off that seems to exist, more on the Google Forms side, is building not just a Form with a list of Questions, but a Consumer Process Workflow, that allows the business to provide an interactive dialog based on answers users provide.  For example, a Yes/No or multichoice answer may lead to alternate sets of questions [and actions].  It doesn’t appear there is any workflow tool provided to structure the Google Forms / fobi.io chatbot Q&A.

However, there are still many business cases for the product, especially for small to mid size organizations.

* Business Estimates – although there is no logic workflow to guide the Q&A sessions with [prospective] customers, the business still may derive the initial information they require to make an initial assessment.  It seems a Web form, and this fobi.io / Google Forms solution seems very comparable in capability, its just a change in the median in which the user interacts to collect the information.

One additional note, Google Forms is not a free product.  Looks like it’s a part of the G Suite. Free two week trial, then the basic plan is $5 per month, which comes with other products as well.  Click here for pricing details.

Although this “chatbot” tries to quickly provide a mechanism to turn a form to a chatbot, it seems it’s still just a form at the end of the day.  I’m interested to see more products from Zoi.ai soon