Tag Archives: Search Engines

AI, Artificial Intelligence, Business, Development, Technology

Google Search Enables Users to Upload Images for Searching with Visual Recognition. Yahoo and Bing…Not Yet

July 29, 2017 Ian Roseman Leave a comment

The ultimate goal, in my mind, is to have the capability within a Search Engine to be able to upload an image, then the search engine analyzes the image, and finds comparable images within some degree of variation, as dictated in the search properties. The search engine may also derive metadata from the uploaded image such as attributes specific to the image object(s) types. For example, determine if a person [object] is “Joyful” or “Angry”.

As of the writing of this article, search engines Yahoo and Microsoft Bing do not have the capability to upload an image and perform image/pattern recognition, and return results. Behold, Google’s search engine has the ability to use some type of pattern matching, and find instances of your image across the world wide web. From the Google Search “home page”, select “Images”, or after a text search, select the “Images” menu item. From there, an additional icon appears, a camera with the hint text “Search by Image”. Select the Camera icon, and you are presented with options on how Google can acquire your image, e.g. upload, or an image URL.

Select the “Upload an Image” tab, choose a file, and upload. I used a fictional character, Max Headroom. The search results were very good (see below). I also attempted an uncommon shape, and it did not meet my expectations. The poor performance of matching this possibly “unique” shape is mostly likely due to how the Google Image Classifier Model was defined, and correlating training data that tested the classifier model. If the shape is “Unique” the Google Search Image Engine did it’s job.

Google Image Search Results – Max Headroom

Google Image Search Results – Odd Shaped Metal Object

Google Search Results - Odd Shaped Metal Object — Google Search Results – Odd Shaped Metal Object

The Google Search Image Engine was able to “Classify” the image as “metal”, so that’s good. However I would have liked to see better matches under the “Visually Similar Image” section. Again, this is probably due to the image classification process, and potentially the diversity of image samples.

A Few Questions for Google

How often is the Classifier Modeling process executed (i.e. training the classifier), and the model tested? How are new images incorporated into the Classifier model? Are the user uploaded images now included in the Model (after model training is run again)? Is Google Search Image incorporating ALL Internet images into Classifier Model(s)? Is an alternate AI Image Recognition process used beyond Classifier Models?

Behind the Scenes

In addition, Google has provided a Cloud Vision API as part of their Google Cloud Platform.

I’m not sure if the Cloud Vision API uses the same technology as Google’s Search Image Engine, but it’s worth noting. After reaching the Cloud Vision API starting page, go to the “Try the API” section, and upload your image. I tried a number of samples, including my odd shaped metal, and I uploaded the image. I think it performed fairly well on the “labels” (i.e. image attributes)

Using the Google Cloud Vision API, to determine if there were any WEB matches with my odd shaped metal object, the search came up with no results. In contrast, using Google’s Search Image Engine produced some “similar” web results.

Odd Shaped Metal Sample Image Web Results

Finally, I tested the Google Cloud Vision API with a self portrait image. THIS was so cool.

Google Vision API - Face Attributes — Google Vision API – Face Attributes

The API brought back several image attributes specific to “Faces”. It attempts to identify certain complex facial attributes, things like emotions, e.g. Joy, and Sorrow.

Google Vision API - Labels — Google Vision API – Labels

The API brought back the “Standard” set of Labels which show how the Classifier identified this image as a “Person”, such as Forehead and Chin.

Google Vision API - Web — Google Vision API – Web

Finally, the Google Cloud Vision API brought back the Web references, things like it identified me as a Project Manager, and an obscure reference to Zurg in my Twitter Bio.

The Google Cloud Vision API, and their own baked in Google Search Image Engine are extremely enticing, but yet have a ways to go in terms of accuracy %. Of course, I tried using my face in the Google Search Image Engine, and looking at the “Visually Similar Images” didn’t retrieve any images of me, or even a distant cousin (maybe?)

Google Image Search Engine: Ian Face Image

Business, Economy, Education, Entertainment, Healthcare, Technology, Work

AI Digital Assistant verse Search Engines

March 8, 2017 Ian Roseman Leave a comment

Aren’t AI Digital Assistants just like Search Engines? They both try to recognize your question or human utterance as best as possible to serve up your requested content. E.g.classic FAQ. The difference in the FAQ use case is the proprietary information from the company hosting the digital assistant may not be available on the internet.

Another difference between the Digital Assistant and a Search Engine is the ability of the Digital Assistant to ‘guide’ a person through a series of questions, enabling elaboration, to provide the user a more precise answer.

The Digital Assistant may use an interactive dialog to guide the user through a process, and not just supply the ‘most correct’ responses. Many people have flocked to YouTube for instructional type of interactive medium. When multiple workflow paths can be followed, the Digital Assistant has the upper hand.

The Digital Assistant has the capability of interfacing with 3rd parties (E.g. data stores with API access). For example, there may be a Digital Assistant hosted by Medical Insurance Co that has the ability to not only check the status of a claim, but also send correspondence to a medical practitioner on your behalf. A huge pain to call the insurance company, then the Dr office, then the insurance company again. Even the HIPPA release could be authenticated in real time, in line during the chat. A digital assistant may be able to create a chat session with multiple participants.

Digital Assistants overruling capabilities over Search Engines are the ability to ‘escalate’ at any time during the Digital Assistant interaction. People are then queued for the next available human agent.

There have been attempts in the past, such as Ask.com (originally known as Ask Jeeves) is a question answering-focused e-business. Google Questions and Answers (Google Otvety, Google Ответы) was a free knowledge market offered by Google that allowed users to collaboratively find good answers, through the web, to their questions (also referred as Google Knowledge Search).

My opinions are my own, and do not reflect my employer’s viewpoint.

Business, Science, Technology

Artificial Intelligence: Tuning with a Content Index And Predictive Models

November 13, 2012 iroseman Leave a comment

As I was reading an All Things Digital article, Artificial-Intelligence Professor Makes a Search App to Outsmart Siri, there was a statement that made:

“We memorize the dictionary to read the Library of Congress,” he said. “Siri is trying to memorize the Library of Congress.”

A tool more commonly used in the past, in books, at the end of a book, an index of where the words appeared in the book was noted with page numbers. Classic rules engines is ‘data in a black box’, searchable within the context they appear. The more put into the black box, users can search on ‘rules’ or content, and in precedence, an action occurs, or the content that is searched appears. If there are cross-references with an associated category or tag with ‘each line of data’ or ‘rule’ that will enable the Artificial Intelligence engine to be more efficient. Therefore the correlations of ‘data to other data’ with similar or like tags enables an Artificial Intelligence will be more intelligent. In theory, categorized or tagged content indexed to references of the data points should fine tune the engine.

An addition theory, allows for predictive models to produce refined searches, or rules. You can make a predictive model, where the intelligence of the user actually refines the engine. The user can ask a question, and as they refine their question, a predictive model, may allow for refined user output. If a user is allowed to participate and tag data search output, the search output could be more granular, like a refined Business Intelligence drill down. The output of a search, for example, can contain a title, brief summary, and tags that can be added or removed (by the user), which allows for a more robust search, and predictive model; however, you are relying on the user to a) not be malicious, and b) have understanding of what information he is search for within the data. If web crawlers, or if the webmaster submits URLs with tags, the meta data tags of the page, the black box or Artificial Intelligence rules engine will, if properly submitted, or indexed, correlate the data. To most people, this is AI or Search Engine 101. Some people cheat, and add pages with false meta data tags because they want their site to appear in a higher order, or precedence and they may make more revenue with advertising dollars.

There are multiple ways around trying to cheat an Artificial Intelligence Content Index:

Hit Ratio: People searching on the same question over and over increase the ‘score’ ratio, thus pushing the false results downward on the list, or removing them entirely.
Enlist ‘quality’ users, who are known quantities, such as like Twitter ‘certifies’ certain users. You may apply for ‘relatively’ unbiased, certification status, such as people who have reputations and certifications in the field, are qualified to ‘enhance’ tags, and improve upon your result outputs. e.g Professors, Statisticians,
Enlist users who will actually derive revenue, if their ‘hit ratio’ score delta increases exponentially some N number. These tags are classified as unverified, however, the people are monetarily motivated to increase peoples’ probably of success to find what people are looking for when other users search the tags become qualified as the results of the tags attract users to their content. If they are using, let’s say, a browser, which the search engine company owns, such as Chrome, a little plug in can appear and say, was this what you were looking to find, yes or no.

Roseman Solutions LLC