The holiday season brings lots of people to your front door. If you have a front door camera, you may be getting many alerts from your front door that let you know there is motion at the door. It would be great if the front doorbell cameras could take the next step and incorporate #AI facial/image recognition and notify you through #iOS notifications WHO is at the front door and, in some cases, which “uniformed” person is at the door, e.g. FedEx/UPS delivery person.
RIng iOS Notification
This facial recognition technology is already baked into Microsoft #OneDrive Photos and Apple #iCloud Photos. It wouldn’t be a huge leap to apply facial and object recognition to catalog the people who come to your front door as well as image recognition for uniforms that they are wearing, e.g., UPS delivery person.
iCloud/OneDrive Photos identify faces in your images, group by likeness, so the owner of the photo gallery can identify this group of faces as Grandma, for example. It may take one extra step for the camera owner to login into the image/video storage service and classify a group of videos converted to stills containing the face of Grandma. Facebook Meta also can tag the faces within pictures you upload and share. The Facebook app also can “guess” faces based on previously uploaded images.
No need to launch the Ring app and see who’s at the front door. Facial recognition can remove the step required to find out what is the motion at the front door and just post the iOS notification with the “who’s there”.
One less step to launching the Ring app and see who is at the front door.
Information architecture (IA) focuses on organizing, structuring, and labeling content in an effective and sustainable way. The goal is to help users find information and complete tasks.
There must be a common consensus, an understanding of each data point collected, and the appropriate labeling and cataloging of the Information Asset. Information assets may have a score attributed to the asset and leveraged in a multitude of ways, such as guidelines for the purging of archives, sensitivity of the information, and the levels of trust.
For each data point collected, correlations/relationships can be added either manually, or through an Induction Engine (AI) leveraging a history of relationships. The definition of hierarchical relationships between data points, and link types (e.g. processor, successor, child, or generally related) further to bolster a larger lexicon.
What are Information Assets?
For example, your phone number is an information asset. Your phone number is provided to everyone you know and is a primary point of reference to contact you. Traditionally, the “phone companies” manage that resource for you. However, in this “new” day and age, we see companies like Google providing a phone number, and as a result providing features not generally available, such as Google Voice, with Call Forwarding, and obfuscation.
Common, Consumer, Information Assets Include:
Documents of ALL Types, e.g. text, spreadsheets, presentations, etc.
Domain Names and Email Addresses are Information Assets.
Twitter, Facebook, Instagram, and Other Social Media Platforms Assets, such as User Names, Post Text, Images, Video, and Profile details.
Skype, WhatsApp, and other VoIP Info Assets such as Phone Number, User Profile information
Windows Teams, Slack, and other Team Collaboration, Information Assets, such as the historical, ongoing posted information in the Team Chat, including the integration of 3rd party apps, such as Whiteboard collaborative drawings.
Passwords, Passwords, Passwords
Common, Corporate, Information Assets Include:
All of the Consumer, Information Assets PLUS
Documents of ALL Types, e.g. Solution Architecture docs, Database Models, HR Policies, Org Charts, Corp. Network Topography, etc.
Disaster Recovery for Information Assets
What happens when the technology managing information assets become “unavailable”? What is your impact assessment? Is there a centralized data/information catalog or repository that contains a partial or complete set of Information Assets?
Information Assets are also passwords, and we have a plethora of “secure” password managers, such as Norton Antivirus provides a mechanism to hold passwords in a virtual “safe”.
Insurance Policies for [digital] Information Assets
What is the cost of securing these Information Assets, verse the payment of recuperating the information assets, if even possible?
What about Hackers that “hold your data/information” hostage?
How to price out “Insurance” for your information, just like safeguarding any other personal articles insurance policies today? Are there “Personal Articles, Insurance Policies” that can currently add a rider to your existing policies? Need to price out “Information Assets”, and the recuperation values?
Norton Life Lock [Personal / Business]
Norton LifeLock reimburses funds stolen due to identity theft up to the limit of the plan total not exceeding $1 Million USD.
Notes Repositories
Notepads like Notepad++, Microsoft OneNote, and Google Keep are tools that allow their authors to quickly take notes and organize them. A wide array of Information Assets are contained within these applications, such as text, and photos with some data describing the information captured (i.e. metadata). Gathering and exporting this information to reference Information Assets could be a lengthy and laborious process without automation, rules for sorting, and tagging info.
AI Induction and Rules Engines
Dynamically labeling Information Assets as they are “discovered”, an auto curation process. For example, the Microsoft Outlook rules engine has a robust library of canned AI rules for sorting, forwarding, formatting as emails arrive in your inbox, as well as a host of other rules “triggers”. An Induction engine is a predictive instrument that “observes” behavior over time, and then creates/suggests new rules on the basis of the history of user behavior. For example, if MS Outlook had an AI Induction engine, and observed a user ‘almost’ always moving an email with the same subject to folder N, the AI Induction engine could create the rule to anticipate the user’s behavior.
Data Lakes or Sea of Information Assets
Structured, Semi-Structured, and Unstructured data.
Labeling/tagging Information Assets in a consistent fashion.
Retrieval of data, and cross-referenced data types
Description: Alation is a complete repository for enterprise data, providing a single point of reference for business glossaries, data dictionaries, and Wiki articles. The product profiles data and monitors usage to ensure that users have accurate insight into data accuracy. Alation also provides insight into how users are creating and sharing information from raw data. Customers tout the product for its expansive partner ecosystem, and Alation has focused on increasing data literacy when metadata is distributed across business and IT.
Very novel concept, applying Augmented Reality and Artificial Intelligence (i.e. Machine Learning) to solving puzzles, such as Sudoko. Maybe not so novel considering AR uses in manufacturing.
Next, we’ll be using similar technology for human to human negotiations, reading body language, understanding logical arguments, reading human emotion, and to rebut remarks in a debate.
Over the last two decades, I’ve been involved in several solutions that incorporated artificial intelligence and in some cases machine learning. I’ve understood at the architectural level, and in some cases, a deeper dive.
I’ve had the urge to perform a data trending exercise, where not only do we identify existing trends, similar to “out of the box” Twitter capabilities, we can also augment “the message” as trends unfold. Also, probably AI 101. However, I wanted to submerge myself in understanding this Data Science project. My Solution Statement: Given a list of my interests, we can derive sentence fragments from Twitter, traverse the tweet, parsing each word off as a possible “breadcrumb”. Then remove the Stop Words, and voila, words that can identify trends, and can be used to create/modify trends.
Finally, to give the breadcrumbs, and those “words of interest” greater depth, using the Oxford Dictionaries API we can enrich the data with things like their Thesaurus and Synonyms.
Gotta Have a Hobby
It’s been a while now that I’ve been hooked on Microsoft Power Automate, formerly known as Microsoft Flow. It’s relatively inexpensive and has the capabilities to be a tremendous resource for almost ANY project. There is a FREE version, and then the paid version is $15 per month. No brainer to pick the $15 tier with bonus data connectors.
I’ve had the opportunity to explore the platform and create workflows. Some fun examples, initially, using MS Flow, I parsed RSS feeds, and if a criterion was met, I’d get an email. I did the same with a Twitter feed. I then kicked it up a notch and inserted these records of interest into a database. The library of Templates and Connectors is staggering, and I suggest you take a look if you’re in a position where you need to collect and transform data, followed by a Load and a notification process.
What Problem are we Trying to Solve?
How are trends formed, how are they influenced, and what factors influence them? The most influential people providing input to a trend? Influential based on location? Does language play a factor on how trends are developed? End Goal: driving trends, and not just observing them.
Articles I’ve read regarding Data Science projects revolved around 5 steps:
Obtain Data
Scrub Data
Explore Data
Model Data
Interpreting Data
The rest of this post will mostly revolve around steps 1 and 2. Here is a great article that goes through each of the steps in more detail: 5 Steps of a Data Science Project Lifecycle
Capturing and Preparing the Data
The data set is arguably the most important aspect of Machine Learning. Not having a set of data that conforms to the bell curve and consists of all outliers will produce an inaccurate reflection of the present, and poor prediction of the future.
First, I created a table of search criteria based on topics that interest me.
Then I created a Microsoft Flow for each of the search criteria to capture tweets with the search text, and insert the results into a database table.
MS Flow – Twitter: Ingestion of Learning Tweets
Out of the total 7450 tweets collected from all the search criteria, 548 tweets were from the Search Criteria “Learning” (22).
Data Ingestion – Twitter
After you’ve obtained the data, you will need to parse the Tweet text into “breadcrumbs”, which “lead a path” to the Search Criteria.
Machine Learning and Structured Query Language (SQL)
This entire predictive trend analysis could be much easier with a more restrictive syntax language like SQL instead of English Tweets. Parsing SQL statements would be easier to make correlations. For example, the SQL structure can be represented such as: SELECT Col1, Col2 FROM TableA where Col2 = ‘ABC’. Based on the data set size, we may be able to extrapolate and correlate rows returned to provide valuable insights, e.g. projected impact performance of the query to the data warehouse.
R language and R Studio
Preparing Data Sets Using Tools Designed to Perform Data Science.
R language and R Studio seems to be very powerful when dealing with large data sets, and syntax makes it easy to “clean” the data set. However, I still prefer SQL Server and a decent query tool. Maybe my opinion will change over time. The most helpful thing I’ve seen from R studio is to create new data frames and the ability to rollback to a point in time, i.e. the previous version of the data set.
Changing column data type on the fly in R studio is also immensely valuable. For example, the data in the column are integers but the data table/column definition is a string or varchar. The user would have to drop the table in SQL DB, recreate the table with the new data type, and then reload the data. Not so with R.
First, there was Spell Check, next Thesaurus, Synonyms, contextual grammar suggestions, and now Persona, Point of View Reviews. Between the immensely accurate and omnipresent #Grammarly and #Google’s #Gmail Predictive Text, I starting thinking about the next step in the AI and Human partnership on crafting communications.
Google Gmail Predictive Text
Google gMail predictive text had me thinking about AI possibilities within an email, and it occurred to me, I understand what I’m trying to communicate to my email recipients but do I really know how my message is being interpreted?
Google gMail has this eerily accurate auto suggestive capability, as you type out your email sentence gMail suggests the next word or words that you plan on typing. As you type auto suggestive sentence fragments appear to the right of the cursor. It’s like reading your mind. The most common word or words that are predicted to come next in the composer’s eMail.
Personas
In the software development world, it’s a categorization or grouping of people that may play a similar role, behave in a consistent fashion. For example, we may have a lifecycle of parking meters, where the primary goal is the collection of parking fees. In this case, personas may include “meter attendant”, and “the consumer”. These two personas have different goals, and how they behave can be categorized. There are many such roles within and outside a business context.
In many software development tools that enable people to collect and track user stories or requirements, the tools also allow you to define and correlate personas with user stories.
As in the case of email composition, once the email has been written, the composer may choose to select a category of people they would like to “view from their perspective”. Can the email application define categories of recipients, and then preview these emails from their perspective viewpoints?
What will the selected persona derive from the words arranged in a particular order? What meaning will they attribute to the email?
Use Personas in the formulation of user stories/requirements; understand how Personas will react to “the system”, and changes to the system.
Finally the use of the [email composer] solution based on “actors” or “personas”. What personas’ are “out of the box”? What personas will need to be derived by the email composer’s setup of these categories of people? Wizard-based Persona definitions?
There are already software development tools like Azure DevOps (ADO), which empower teams to manage product backlogs and correlate “User Stories”, or “Product Backlog Items” with Personas. These are static personas, that are completely user-defined, and no intelligence to correlate “user stories” with personas”. Users of ADO must create these links.
Now, technology can assist us to consider the intended audience, a systematic, biased perspective using Artificial Intelligence to inspect your email based on selected “point of view” (a Persons) of the intended email. Maybe your email will be misconstrued as abrasive, and not the intended response.
Machine learning and deep learning are two subsets of artificial intelligence which have garnered a lot of attention over the past two years. If you’re here looking to understand both the terms in the simplest way possible, there’s no better place to be..
— Read on morioh.com/p/78e1357f65b0
Every day, Facebook’s artificial intelligence algorithms tackle the enormous task of finding and removing millions of posts containing spam, hate speech, nudity, violence, and terrorist propaganda. And though the company has access to some of the world’s most coveted talent and technology, it’s struggling to find and remove toxic content fast enough.
I’ve worked at several software companies which leveraged Artifical Intelligence, Machine Learning to recognize patterns, correlations. The larger the data sets, in general, the higher the accuracy of the predictions. The outliers in the data, the noise, “falls out” of the data set. Without quality, large training data, Artificial Intelligence makes more mistakes.
In terms of speech recognition, image classification, and natural language processing (NLP), in general, programs like chatbots, digital assistants, are becoming more accurate because of their sample size, training data sets are large, and there is no shortage of these data types. For example, there are many ways I can ask my digital assistant for something, like “Get the movie times”. Training a digital assistant, at a high level, would be to catalog how many ways can I ask for “something”, achieve my goal. I can go and create that list. I could write a few dozen questions, but still, my sample data set would be too small. Amazon has a crowdsourcing platform, Amazon Mechanical Turk, which I can request they build me the data sets, thousands of questions, and correlated goals.
MTurk enables companies to harness the collective intelligence, skills, and insights from a global workforce to streamline business processes, augment data collection and analysis, and accelerate machine learning development.
Amazon Mechanical Turk: Access a global, on-demand, 24×7 workforce
Video “Scene” Recognition – Annotated Data Sets for a Wide Variety of Scene Themes
In silent films, the plot was conveyed by the use of title cards, written indications of the plot and key dialogue lines. Unfortunately, silent films are not making a comeback. In order to achieve a high rate of successful identification of activities within a given video clip, video libraries of metadata need to be created, that capture:
Image recognition, in the context of machine vision, is the ability of software to identify objects, places, people, writing and actions in images. Image recognition is used to perform a large number of machine-based visual tasks, such as labeling the content of images with meta-tags
Not Enough Data
Here is an example of how Social Media, such as Facebook, attempts to deal with video deemed inappropriate for their platform:
In March, a shooter in New Zealand live-streamed the brutal killing of 51 people in two mosques on Facebook. But the social-media giant’s algorithms failed to detect the gruesome video. It took Facebook an hour to take the video down, and even then, the company was hard-pressed to deal with users who reposted the video.
…in many cases, such as violent content, there aren’t enough examples to train a reliable AI model. “Thankfully, we don’t have a lot of examples of real people shooting other people,” Yann LeCun, Facebook’s chief artificial-intelligence scientist, told Bloomberg.
Opportunities for Actors and Curators of Video Content: Dramatizations
All those thousands of people who perform, creating videos of content that range the gamut from playing video games to “unboxing” collectible items. The actors who perform dramatizations could add tags to their videos indicating as per above, documenting themes for a given skit. If actors post their videos on YouTube or proprietary crowdsourcing platforms, they would be entitled to some revenue for the use of their licensed video.
Disclosure Regarding Flag Controversy
I now realize there are politics around Nike “tipping their hat” toward the Betsy Ross flag. However, when I referenced the flag in this blog post, I was thinking of the American Revolution, and the 13 colonies flag. I didn’t think the title would resonate with readers, “Help Wanted: Amerian Revolutionary war Reenactment Soldiers to Improve AI Models.”, so I took some creative liberty.
This short example may confound man and machine. How does a rules engine work, how does it make correlations to derive an answer to this and other riddles? If AI, a rules engine is wrong trying to solve this riddle, how does it use machine learning to adjust, and tune its “model” to draw an alternate conclusion to this riddle?
Training rules engines using machine learning and complex riddles may require AI to define relationships not previously considered, analogously to how a boy or man consider solving riddles. Man has more experiences than a boy, widening their model to increase the possible answer sets. But how to conclude the best answer? Question sentence fragments may differ over a lifetime, hence the man may have more context as to the number of ways the question sentence fragment may be interpreted.
Adding Context: Historical and Pop Culture
There are some riddles thousands of years old. They may have spawned from another culture in another time and survived and evolved to take on a whole new meaning. Understanding the context of the riddle may be the clue to solving it.
Layers of historical culture provide context to the riddle, and the significance of a word or phrase in one period of history may wildly differ. When you think of “periods of history”, you might think of the pinnacle of the Roman empire, or you may compare the 1960s, the 70s, 80s, etc.
Asking a question of an AI, rules engine, such as a chatbot may need contextual elements, such as geographic location, and “period in history”, additional dimensions to a data model.
Many chatbots have no need for additional context, a referential subtext, they simply are “Expert Systems in a box”. Now digital assistants may face the need for additional dimensions of context, as a general knowledge digital agent spanning expertise without bounds.
Written in the fifth century B.C., Oedipus the King is one of the most famous pieces of literature of all time, so it makes sense that it gave us one of the most famous riddles of all time.
What goes on four legs in the morning, on two legs at noon, and on three legs in the evening?
A human.
Humans crawl on hands and knees (“four legs”) as a baby, walk on two legs in mid-life (representing “noon”) and use a walking stick or can (“three legs”) in old age.
A modern interpretation of the riddle may not allow for the correlation and solving the riddle. As such “three legs”, i.e. a cane, may be elusive, as we think of the elderly on four wheels on a wheelchair.
In all sincerity, this article is not about an AI rules engine “firing rules” using a time dimension, such as:
Not letting a person gain entry to a building after a certain period of time, or…
Providing a time dimension to “Parental Controls” on a Firewall / Router, the Internet is “cut off” after 11 PM.
Adding a date/time dimension to the question may produce an alternate question. The context of the time changes the “nature” of the question, and therefore the answer as well.
Smart Solutions
Definition: Product Owner (PO)
The Product Owner (PO) is a member of the Agile Team responsible for defining Stories and prioritizing the Team Backlog to streamline the execution of program priorities while maintaining the conceptual and technical integrity of the Features or components for the team.
