The holiday season brings lots of people to your front door. If you have a front door camera, you may be getting many alerts from your front door that let you know there is motion at the door. It would be great if the front doorbell cameras could take the next step and incorporate #AI facial/image recognition and notify you through #iOS notifications WHO is at the front door and, in some cases, which “uniformed” person is at the door, e.g. FedEx/UPS delivery person.
This facial recognition technology is already baked into Microsoft #OneDrive Photos and Apple #iCloud Photos. It wouldn’t be a huge leap to apply facial and object recognition to catalog the people who come to your front door as well as image recognition for uniforms that they are wearing, e.g., UPS delivery person.
iCloud/OneDrive Photos identify faces in your images, group by likeness, so the owner of the photo gallery can identify this group of faces as Grandma, for example. It may take one extra step for the camera owner to login into the image/video storage service and classify a group of videos converted to stills containing the face of Grandma. Facebook Meta also can tag the faces within pictures you upload and share. The Facebook app also can “guess” faces based on previously uploaded images.
No need to launch the Ring app and see who’s at the front door. Facial recognition can remove the step required to find out what is the motion at the front door and just post the iOS notification with the “who’s there”.
One less step to launching the Ring app and see who is at the front door.
“There is a lack of precision in thought,” said Robert D. Atkinson, the president of the Information Technology and Innovation Foundation, a research group that supports U.S. government funding for essential technologies including computer chips. (The group gets funding from telecom and tech companies, including the U.S. computer chip giant Intel.)
Atkinson told me that he backed the proposals winding through Congress for government help for tech research and development, and for taxpayer subsidies for U.S. chip factories. But he also said that there was a risk of U.S. policy treating all domestic technology manufacturing as equally important. “Maybe it would be nice if we made more solar panels, but I don’t think that’s strategic,” he said.
Atkinson and people whom I spoke to in the computer chip industry say that there are important ways that computer chips are not like iPhones, and that it would be helpful if more were made on U.S. soil. About 12 percent of all chips are manufactured in the U.S.
In their view, manufacturing expertise is tied to tech innovation, and it’s important for America to keep sharp skills in computer chip manufacturing.“
We are one of the three nations on Earth that can do this,” Al Thompson, the head of U.S. government affairs for Intel, told me. “We don’t want to lose this capacity.” (South Korea and Taiwan are the other two countries with top-level chip manufacturing expertise.)
The idea for the “What If…” Business and Technology series for me, comes directly out of Marvel comics. The comics ranged from “fictional” battles or plots that were so abstract yet tangent that it was nearly impossible to happen other than in a “one-off”, alternate reality, comic book in the Marvel Universe. On the same premise, I will spin several stories that will most likely not happen in the “real world”, but we will bring them to light.
Apple Adopts the Palm OS Business Model
Apple spins off its mobile hardware business and focuses on the iOS operating system. The mobile OS business unit, in theory, will have a robust and direct focus to drive revenue to their area. We may see partnerships that would have never developed if these units are continued to be tied together. Mobile iOS on OEM, 3rd party devices? Multiboot mobile hardware for Android, Linux, and Mac iOS out of the box. Competing and evolving lines, between the iOS tablets. How about mobile hardware from Motorola, Nokia, or Blackberry using iOS?
Allow Customers to Buy Gasoline as Units at Current Price on Loyalty Debit Cards
A legitimate reason IS NEEDED why in this day and age of commodities trading, storage, pipelines, trains, tankers, and trucks, why if I can buy stocks at current market value, or go on Ebay and buy 50 yards of antique bobbed wire at an auction, why can’t I go to a gas station and buy 50 UNITS of gas at the current price of gas that day? Upon return to the same brand gas station, I should be able to use my same loyalty debit card and subtract units of gas from the card, instead of the current price of gas that day.
EVs “in the field” Use Existing Home Energy Provider, with Transportation Charges Applied
Similar logic can be applied to Electric, and EVs but with a twist, incorporating the use of your home “energy provider”, and when charging ‘on the go’, only pay local “line usage/distribution” fees.
Fundraising Using Public WiFi, and an UL/DL MB Meter Reading
What if commercial, public WiFi Hotspots partnered with a fundraising cause, and every MB exchanged (up and/or down), a donation of N cents would go to a charity-sponsored by you with Paypal. The business providing the WiFi would match the donation.
Greece Prosperity / Tourism: The World will Come See You in Augmented Reality (AR)
While perusing through all of the ruins, looking at the sites, watching the tour groups, and the tour guides explaining these empty ruins, I pondered, wouldn’t it be amazing to see the people of ancient times dressed in their clothing of the times, interacting with each other through the ruins as though the tourists were not even there. In effect, acting out scenes that perhaps took place thousands of years ago, echoes of the past. I thought why wouldn’t for starters, the government pays the people of the Arts and Sciences to go through scenes, such as basic interactions with a Librarian, studying in one corner, ignoring the world, and just reading and thinking, and in another corner of this library, there might be a quiet debate going on, in Greek of course, and through a translation application, any foreigner could hear their native language the interactions. It could be ancient commoners, to known ancient people of the past acting out scenes like echoes of the past, while tours just come up close and personal, pass by, even wave their hand in front of an actors hand, and he continues to act as if the tourists were not even there. All of these Greek actors would be prerecorded and play out in Augmented Reality (AR).
Voice mail is so LAST Century. It’s a static communications interface to address your incoming phone calls. It’s a dinosaur in terms of communications protocol. Yes, a digital assistant, or chat bots should “field” your incoming calls, providing your callers a higher level of service.
Business or Personal?
Why not both? There are use cases which highlight the value of a Digital Assistant answering your phone calls when you’re unavailable.
Trusted Friends and Business Pins
Level of available services may change based upon the level of trusted access, such as:
Friends Seeking Your Availability for a Hockey Game Next Week
Business Partners Sharing Information access such as invoices
Untrusted Caller Access
The Vetting of Unsolicited Calls, such as robocalls
Defining and Default Dialogs
Users can define dialogs through drop and drag workflow diagram tools making it easy to “build” conversations / dialogs flows. In addition, out of the box flows can provide administrators with opportunities and discover the ways in which AI digital assistant may be leveraged.
Canned / Default dialog templates to handle the most common dialogs / workflows will empower users to the implement rapidly.
Any Acquisitions in the Pipeline?
Are the big names in the Digital Assistant space looking to partner or acquire tools that can easily transform workflows to be leveraged by digital assistant?
Are the components available for third party product companies to extend the Mobile OS capabilities as of now? Or are the mobile OS companies the only ones in a possession of performing these upgrades?
Advice is integrated within the application, proactive and reactive: When searching in Microsoft Edge, a blinking circle representing Cortana is illuminated. Cortana says “I’ve collected similar articles on this topic.” If selected, presents 10 similar results in a right panel to help you find what you need.
Personal Data Access and Management
The user can vocally access their personal data, and make modifications to that data; E.g. Add entries to their Calendar, and retrieve the current day’s agenda.
Platform Capabilities: Mobile Phone Advantage
Strengthen core telephonic capabilities where competition, Amazon and Microsoft, are relatively week.
Ability to record conversations, and push/store content in Cloud, e.g. iCloud. Cloud Serverless recording mechanism dynamically tags a conversations with “Keywords” creating an Index to the conversation. Users may search recording, and playback audio clips +/- 10 seconds before and after tagged occurrence.
Calls into the User’s Smartphones May Interact Directly with the Digital Assistant
Call Screening – The digital assistant asks for the name of the caller, purpose of the call, and if the matter is “Urgent”
A generic “purpose” response, or a list of caller purpose items can be supplied to the caller, e.g. 1) Schedule an Appointment
The smartphone’s user would receive the caller’s name, and the purpose as a message back to the UI from the call, currently in a ‘hold’ state,
The smartphone user may decide to accept the call, or reject the call and send the caller to voice mail.
A caller may ask to schedule a meeting with the user, and the digital assistant may access the user’s calendar to determine availability. The digital assistant may schedule a ‘tentative’ appointment within the user’s calendar.
If calendar indicates availability, a ‘tentative’ meeting will be entered. The smartphone user would have a list of tasks from the assistant, and one of the tasks is to ‘affirm’ availability of the meetings scheduled.
If a caller would like to know the address of the smartphone user’s office, the Digital Assistant may access a database of “generally available” information, and provide it. The Smartphone user may use applications like Google Keep, and any note tagged with a label “Open Access” may be accessible to any caller.
Custom business workflows may be triggered through the smartphone, such as “Pay by Phone”. When a caller is calling a business user’s smartphone, the call goes to “voice mail” or “digital assistant” based on smartphone user’s configuration. If the user reaches the “Digital Assistant”, there may be a list of options the user may perform, such as “Request for Service” appointment. The caller would navigate through a voice recognition, one of many defined by the smartphone users’ workflows.
Platform Capabilities: Mobile Multimedia
Either through your mobile Smartphone, or through a portable speaker with voice recognition (VR).
Streaming media / music to portable device based on interactions with Digital Assistant.
Menu to navigate relevant (to you) news, and Digital Assistant to read articles through your portable media device (without UI)
Third Party Partnerships: Adding User Base, and Expanding Capabilities
In the form of platform apps (abstraction), or 3rd party APIs which integrate into the Digital Assistant, allowing users to directly execute application commands, e.g. Play Spotify song, My Way by Frank Sinatra.
Any “Skill Set” with specialized knowledge: direct Q&A or instructional guidance – e.g Home Improvement, Cooking
eCommerce Personalized Experience – Amazon
Home Automation – doors, thermostats
Music – Spotify
Navigate Set Top Box (STB) – e.g. find a program to watch
Video on Demand (VOD) – e.g. set to record entertainment
Businesses already exist which have developed and sell Virtual Receptionist, that handle many caller needs (e.g. call routing).
However, AI Digital Assistants such as Alexa, Cortana, Google Now, and Siri have an opportunity to stretch their capabilities even further. Leveraging technologies such as Natural language processing (NLP) and Speech recognition (SR), as well as APIs into the Smartphone’s OS answer/calling capabilities, functionality can be expanded to include:
Call Screening – The digital executive assistant asks for the name of the caller, purpose of the call, and if the matter is “Urgent”
A generic “purpose” response or a list of caller purpose items can be supplied to the caller, e.g. 1) Schedule an Appointment
The smartphone’s user would receive the caller’s name, and the purpose as a message back to the UI from the call, currently in a ‘hold’ state,
The smartphone user may decide to accept the call, or reject the call and send the caller to voicemail.
Call / Digital Assistant Capabilities
The digital executive assistant may schedule a ‘tentative’ appointment within the user’s calendar. The caller may ask to schedule a meeting, the digital executive assistant would access the user’s calendar to determine availability. If calendar indicates availability, a ‘tentative’ meeting will be entered. The smartphone user would have a list of tasks from the assistant, and one of the tasks is to ‘affirm’ availability of the meetings scheduled.
Allow recall of ‘generally available’ information. If a caller would like to know the address of the smartphone user’s office, the Digital Assistant may access a database of generally available information, and provide it. The Smartphone user may use applications like Google Keep, and any notes tagged with a label “Open Access” may be accessible to any caller.
Join the smartphone user’s social network, such as LinkedIn. If the caller knows the phone number of the person but is unable to find the user through the social network directory, an invite may be requested by the caller.
Custom business workflows may also be triggered by the smartphone, such as “Pay by Phone”.
The Digital Executive Assistant capabilities:
Able to gain control of your Smartphone’s incoming phone calls
Able to interact with the 3rd party, dial in caller, on a set of business dialog workflows defined by you, the executive.
Google may attempt to leapfrog their Digital Assistant competition by taking advantage of their ability to search against all Google products. The more personal data a Digital Assistant may access, the greater the potential for increased value per conversation.
As a first step, Google’s “Personal” Search tab in their Search UI has access to Google Calendar, Photos, and your Gmail data. No doubt other Google products are coming soon.
Big benefits are not just for the consumer to search through their Personal Goggle data, but provide that consolidated view to the AI Assistant. Does the Google [Digital] Assistant already have access to Google Keep data, for example. Is providing Google’s “Personal” search results a dependency to broadening the Digital Assistant’s access and usage? If so, these…
interactions are most likely based on a reactive model, rather than proactive dialogs, i.e. the Assistant initiating the conversation with the human.
“What you need, before you ask. Stay a step ahead with Now cards about traffic for your commute, news, birthdays, scores and more.”
I’m not sure how proactive the Google AI is built to provide, but most likely, it’s barely scratching the service of what’s possible.
Modeling Personal, AI + Human Interactions
Starting from N number of accessible data sources, searching for actionable data points, correlating these data points to others, and then escalating to the human as a dynamic or predefined Assistant Consumer Workflow (ACW). Proactive, AI Digital Assistant initiates human contact to engage in commerce without otherwise being triggered by the consumer.
Actionable data point correlations can trigger multiple goals in parallel. However, the execution of goal based rules would need to be managed. The consumer doesn’t want to be bombarded with AI Assistant suggestions, but at the same time, “choice” opportunities may be appropriate, as the Google [mobile] App has implemented ‘Cards’ of bite size data, consumable from the UI, at the user’s discretion.
As an ongoing ‘background’ AI / ML process, Digital Assistant ‘server side’ agent may derive correlations between one or more data source records to get a deeper perspective of the person’s life, and potentially be proactive about providing input to the consumer decision making process.
The proactive Google Assistant may suggest to book your annual fishing trip soon. Elevated Interaction to Consumer / User.
The Assistant may search Gmail records referring to an annual fishing trip ‘last year’ in August. AI background server side parameter / profile search. Predefined Assistant Consumer Workflow (ACW) – “Annual Events” Category. Building workflows that are ‘predefined’ for a core set of goals/rules.
AI Assistant may search user’s photo archive on the server side. Any photo metadata could be garnished from search, including date time stamps, abstracted to include ‘Season’ of Year, and other synonym tags.
Photos from around ‘August’ may be earmarked for Assistant use
Photos may be geo tagged, e.g. Lake Champlain, which is known for its fishing.
All objects in the image may be stored as image metadata. Using image object recognition against all photos in the consumer’s repository, goal / rule execution may occur against pictures from last August, the Assistant may identify the “fishing buddies” posing with a huge “Bass fish”.
In addition to the Assistant making the suggestion re: booking the trip, Google’s Assistant may bring up ‘highlighted’ photos from last fishing trip to ‘encourage’ the person to take the trip.
This type of interaction, the Assistant has the ability to proactively ‘coerce’ and influence the human decision making process. Building these interactive models of communication, and the ‘management’ process to govern the AI Assistant is within reach.
Predefined Assistant Consumer / User Workflows (ACW) may be created by third parties, such as Travel Agencies, or by industry groups, such as foods, “low hanging fruit” easy to implement the “time to get more milk” . Or, food may not be the best place to start, i.e. Amazon Dash
Aren’t AI Digital Assistants just like Search Engines? They both try to recognize your question or human utterance as best as possible to serve up your requested content. E.g.classic FAQ. The difference in the FAQ use case is the proprietary information from the company hosting the digital assistant may not be available on the internet.
Another difference between the Digital Assistant and a Search Engine is the ability of the Digital Assistant to ‘guide’ a person through a series of questions, enabling elaboration, to provide the user a more precise answer.
The Digital Assistant may use an interactive dialog to guide the user through a process, and not just supply the ‘most correct’ responses. Many people have flocked to YouTube for instructional type of interactive medium. When multiple workflow paths can be followed, the Digital Assistant has the upper hand.
The Digital Assistant has the capability of interfacing with 3rd parties (E.g. data stores with API access). For example, there may be a Digital Assistant hosted by Medical Insurance Co that has the ability to not only check the status of a claim, but also send correspondence to a medical practitioner on your behalf. A huge pain to call the insurance company, then the Dr office, then the insurance company again. Even the HIPPA release could be authenticated in real time, in line during the chat. A digital assistant may be able to create a chat session with multiple participants.
Digital Assistants overruling capabilities over Search Engines are the ability to ‘escalate’ at any time during the Digital Assistant interaction. People are then queued for the next available human agent.