Category Archives: Augmented Reality

Platform Independent AI Model for Images: AI Builder, Easily Utilized by 3rd Party Apps

With all the discourse on OpenAI’s ChatGPT and Natural language processing (NLP), I’d like to steer the conversation toward images/video and object recognition. This is another area in artificial intelligence primed for growth with many use cases. Arguably, it’s not as shocking, bending our society at its core, creating college papers with limited input, but Object Recognition can seem “magical.” AI object recognition may turn art into science, as easy as AI reading your palm to tell your future. AI object recognition will bring consumers more data points from which Augmented Reality (AR) overlays digital images within an analog world of tangible objects.

Microsoft’s AI Builder – Platform Independent

Microsoft’s Power Automate AI [model] Builder has the functionality to get us started on the journey of utilizing images, tagging them with objects we recognize, and then training the AI model to recognize objects in our “production” images. Microsoft provides tools to build AI [image] models (library of images with human, tagged objects) quickly and easily. How you leverage these AI models is the foundation of “future” applications. Some applications are already here, but not mass production. The necessary ingredient: taking away the proprietary building of AI models, such as in social media applications.

In many social media applications, users can tag faces in their images for various reasons, mostly who to share their content/images with. In most cases, images can also be tagged with a specific location. Each AI image/object model is proprietary and not shared between social media applications. If there was a standards body, an AI model could be created/maintained outside of the social media applications. Portable AI object recognition models with a wide array of applications that support it’s use, such as social media applications. Later on, we’ll discuss Microsoft’s AI Model builder, externalized from any one application, and because it’s Microsoft, it’s intuitive. 🙂

An industry standards body could collaborate and define what AI models look like their features, and most importantly, the portability formats. Then the industry, such as social media apps, can elect to adopt features that are and are not supported by their applications.

Use Cases for Detecting Objects in Images

Why doesn’t everyone have an AI model containing tagged objects within images and videos of the user’s design? Why indeed.

1 – Brands / Product Placement from Content Creators

Just about everyone today is a content creator, producing images and videos for their own personal and business social media feeds, Twitter, Instagram, Snap, Meta, YouTube, and TikTok, to name a few. AI models should be portable enough to integrate with social media applications where tags could be used to identify branded apparel, jewelry, appliances, etc. Tags could also contain metadata, allowing content consumers to follow tagged objects to a specified URL. Clicks and the promotion of products and services.

2 – Object Recognition for Face Detection

Has it all been done? Facebook/Meta, OneDrive, iCloud, and other services have already tried or are implementing some form of object detection in the photos you post. Each of these existing services implements object detection at some level:

  • Identify the faces in your photos, but need you to tag those faces and some “metadata” will be associated with these photos
  • Dynamically grouping/tagging all “Portrait” pictures of a specific individual or events from a specific day and location, like a family vacation.
  • Some image types, JPEGs, PNG, GIF, etc., allow you to add metadata to the files on your own, e.g. so you can search for pictures on the OS level of implementation.
3 – Operational Assistance through object recognition using AR
  • Constructing “complex” components in an assembly line where Augmented Reality (AR) can overlay the next step in assembly with the existing object to help transition the object to the next step in assembly.
  • Assistance putting together IKEA furniture, like the assembly line use case, but for home use.
  • Gaming, everything from Mario Kart Live to Light Saber duels against the infamous Darth Vader.
4 – Palm Reading and other Visual Analytics
  • Predictive weather patterns
5 – Visual Search through Search Engines and Proprietary Applications with Specific Knowledge Base Alignment
  • CoinSnap iPhone App scans both sides of the coin and then goes on to identify the coin, building a user’s collection.
  • Microsoft Bing’s Visual Search and Integration with MSFT Edge
  • Medical Applications, Leveraging AI, e.g., Image Models – Radiology
Radiology – Reading the Tea Leaves

Radiology builds a model of possible issues throughout the body. Creating images with specific types of fractures can empower the autodetection of any issues with the use of AI. If it was a non-proprietary model, radiologists worldwide could contribute to that AI model. The displacement of radiology jobs may inhibit the open non-proprietary nature of the use case, and the AI model may need to be built independently of open input from all radiologists.

Microsoft’s AI Builder – Detect Objects in Images

Microsoft’s AI model builder can help the user build models in minutes. Object Detection, Custom Model, Detect custom objects in images is the “template” you want to use to build a model to detect objects, e.g. people, cars, anything, rather quickly, and can enable users to add images (i.e. train model) to become a better model over time.

Many other AI Model types exist, such as Text Recognition within images. I suggest exploring the Azure AI Models list to fit your needs.

Current, Available Data Sources for Image Input

  • Current Device
  • SharePoint
  • Azure BLOB

Wish List for Data Sources w/Trigger Notifications

When a new image is uploaded into one of these data sources, a “trigger” can be activated to process the image with the AI Model and apply tags to the images.

  • ADT – video cam
  • DropBox
  • Google Drive
  • Instagram
  • Kodak (yeah, still around)
  • Meta/Facebook
  • OneDrive
  • Ring -video cam
  • Shutterfly
  • Twitter

Get Started: Power Automate, Premium Account

Login to Power Automate with your premium account, and select “AI Builder” menu, then the “Models” menu item. The top left part of the screen, select “New AI Model,” From the list of model types, select “Custom Model, Object Detection”Detect Custom Objects in Images.”

AI Builder - Custom Model
AI Builder – Custom Model

It’s a “Premium” feature of Power Automate, so you must have the Premium license. Select “Get Started”,. The first step is to “Select your model’s domain”, there are three choices, so I selected “Common Objects” to give me the broadest opportunity. Then select “Next”.

AI Builder - Custom Model - Domain
AI Builder – Custom Model – Domain

Next, you need to select all of the objects you want to identify in your images. For demonstration purposes, I added my family’s first names as my objects to train my model to identify in images.

AI Builder - Custom Model - Objects for Model
AI Builder – Custom Model – Objects for Model

Next, you need to “Add example images for your objects.” Microsoft’s guidance is “You need to add at least 15 images for each object you want to detect.” Current data sources include:

Add Images
AI Model – Add Images

I added the minimum recommended images, 15 per object, two objects, 30 images of my family, and random pics over the last year.

Once uploaded, you need to go through each image, draw a box around the image’s objects you want to tag, and then select the object tag.

Part 2 – Completing the Model and its App usage.

Microsoft’s Plethora of Portals

As I was looking through Microsoft’s catalog of applications, it occurred to me just how many of their platforms are information-centric and seemed to overlap in functionality. Where should I go when I want to get stuff done, find information or produce it? Since the early days of AOL and AltaVista, we’ve seen the awesome power of a “Jump Page” as the starting point for our information journey.

Microsoft, which one do I choose?

From one software vendor’s perspective, we’ve got many options. What’s the best option for me? Seems like there should be opportunities to gain synergies between available Microsoft platforms.

Bing.com

Searching for information on the internet? News, images, encyclopedias, Wikipedia, whatever you need, and more is on the web. Microsoft Bing helps you find what you need regardless if you’re using text or an image to search for like for like information. It also serves up “relevant” information on the jump page, news mixed with advertisements. There is also a feature enabling you to add carousel “boxes”. for example, containing latest MS Word files used, synergy from Office.com

Office.com

Word, PowerPoint, Excel, Visio, Power BI… If you’ve created content or want to create content using Microsoft applications, Office.com is the one-stop-shop for all your Office apps and the content created using these applications.

SharePoint

Another portal to a universe of information around a centric theme, such as collaboration/interaction with product/project team members, an Intranet, SharePoint site with one or multiple teams. At the most fundamental level is the capability to collaborate/interact with teams, potentially leveraging Microsoft collaboration tools. Just one of many of its capabilities “out of the box” is a document management solution and the use of version control.

SharePoint can also be used for any type of Internet/web platform, i.e., a public-facing portal platform. However, SharePoint, in fact, is a sharing tool in which the authors of the website can share video presentations, shared calendars of public events, and a plethora of customized lists.

Yammer

Engaging your people is more critical than ever. Yammer connects leaders, communicators, and employees to build communities, share knowledge, and engage everyone. I’m thinking synonymous with a bulletin board. The implementation of Yammer looks like Facebook for the Enterprise.

  • Use the Home feed to stay on top of what matters, tap into the knowledge of others, and build on existing work.
  • Search for experts, conversations, and files.
  • Join communities to stay informed, connect with your coworkers, and gather ideas.
  • Join in the conversation, react, reply to, and share posts.
  • @ mention someone to loop them in.
  • Attach a file, gif, photo, or video to enhance your post.
  • Praise someone in your network to celebrate a success, or just to say thanks.
  • Create a virtual event that your community can ask a question and participate live or watch the recording afterwards.
  • Use polls to crowd source feedback and get answers fast.
  • Stay connected outside the office with the Yammer mobile app.
  • Use Yammer in Microsoft Teams, SharePoint, or Outlook.

“Yammer helps you connect and engage across your organization so that you can discuss ideas, share updates, and network with others.”

Microsoft Teams

For any team, there is a wealth of information varying from the group or single Chats, Teams, Calls, Files, and practically integration for almost all Microsoft applications and beyond. The extensibility of MS Teams seems relatively boundless, such as integrations with Wikis, SharePoint document folders, etc. From what I can tell, many organizations just use Teams for the group, or individual Chat channels are barely grazing the surface of MS Teams’ capabilities.

Setup of MS Teams, Teams “landing” page is a great place to start constructing your “living space” within MS Teams. From there, you can carve out space for all things related to the team. For example, in the “Team ABC” Team channel, you can add N number of “tabs” relating to everything from an embedded Wiki to specific SharePoint folders for the team’s product specifications. A team could even create an embedded Azure DevOps [Kanban] Board to show progress and essentially “live in” your MS Team, team channel.

Another porta;l overlap, Microsoft Teams Communities, seems to equate to Yammer.

Delve

What is Delve – Microsoft 365?

Use Delve to manage your Microsoft 365 profile and to discover and organize the information that’s likely to be most interesting to you right now – across Microsoft 365.

Delve never changes any permissions, so you’ll only see documents that you already have access to. Other people will not see your private documents. Learn more about privacy.

Delve is a content curation platform for the person it’s most relevant to…you. It gives the appearance of a user experience similar to carousels of video streaming apps. There are “Popular Documents” carousels and other carousels that are based on the most recent access. Based on how files are saved based on who can access content is how the platform gives you a treasure trove of documents you never knew you had access to or existed. It actually paints a potential compliance nightmare if people select the default document access as “…anyone within my organization…”.

Outlook.com / Best of MSN

Another portal of information focused around you: your email, your calendar, your To-Dos, and your contacts/people. It’s not just your communication with anyone, e.g., your project team members; it’s organizing your life on a smaller scale, e.g., To-Dos. You can also access other shared calendars, such as a team release schedule or a PTO schedule.

The Best of MSN is information, i.e., news around your interests, a digest of information relevant to you, delivered in an email format. Other digests of information from other sources may be curated and sent if subscribed.

Mediums to Traverse Information: AR, VR…

The visual paradigms used to query and access information may drastically influence the user’s capacity to digest the relevant information. For example, in an Augmented Reality (AR) experience, querying, identifying information, and then applying it, serving up the content in a way most conducive to a user’s experience is vital.

Users can’t just “Google It” and serve up the results like magic. The next evolution of querying information and serving up content in a medium to maximize its usability is key and is most evident when using Augmented Reality (AR). If you’re building something, instructions may be overlayed by the physical elements/parts in front of the user. Even the context of the step number would allow the virtual images to overlay the parts.

Automated and Manual Content Curation is a MUST for all Portals

Categories, Tags, Images, and all other associations from object A to everything else, the Meta of Existence, are essential for proper information dissemination and digestion. If you can tag any object with metadata, you can teach an AI/search engine to identify it in a relevant query. Implementing an Induction Engine, a type of Artificial Intelligence that proposes rules based on historic patterns is a must to improve query accuracy over time.

Next level, “Information applications” – Improved Living with Alzheimer’s

Next Ecosystem: Google..?

Facebook name change: what is Meta, the meaning of new name and Metaverse – Mark Zuckerberg

Mark Zuckerberg surprised the world in October when he announced his company had changed its name to Meta.

The announcement came as the Facebook founder and CEO delivered a presentation showcasing Facebook’s work on virtual reality technologies and the Metaverse – a concept which some believe could become the next version of the internet.

It’s a move that echoes what Google did when it changed the name of its parent company to Alphabet in 2015 – an alteration that represented its shift beyond simply being a search engine.

Mark Zuckerberg said he had chosen it as in Greek it means ‘beyond’.

“For me, it symbolises that there’s always more to build; there’s always a next chapter to the story,” he explained.

“Beyond the constraints of screens, beyond the limits of distance and physics and towards a future where everyone can be present with each other, create new opportunities and experience new things.”

A metaverse is an online world where people can game, work and communicate in a virtual environment.

This world already exists today through Meta-owned brand Oculus, which will itself be re-branded to Meta Quest in 2022, as well as Meta’s collaboration with glasses manufacturer Ray-Ban that allows users to see social media notifications via their glasses or sunglasses.

In his presentation, Mark Zuckerberg showed how his new metaverse concept ‘Horizon’ could apply to our future lives.

For example, he demonstrated how it could be used to hold realistic work meetings and help with education.

Source: Facebook name change: what is Meta, meaning of new name and Metaverse concept – and what Mark Zuckerberg said | NationalWorld

Another Episode of “WHAT IF…” this BIZ meets that Tech

Estimated reading time: 3 minutes

The idea for the “What If…” Business and Technology series for me, comes directly out of Marvel comics.  The comics ranged from “fictional” battles or plots that were so abstract yet tangent that it was nearly impossible to happen other than in a “one-off”, alternate reality, comic book in the Marvel Universe.   On the same premise, I will spin several stories that will most likely not happen in the “real world”, but we will bring them to light.

Apple Adopts the Palm OS Business Model

Apple spins off its mobile hardware business and focuses on the iOS operating system.  The mobile OS business unit, in theory, will have a robust and direct focus to drive revenue to their area.  We may see partnerships that would have never developed if these units are continued to be tied together.  Mobile iOS on OEM, 3rd party devices?   Multiboot mobile hardware for Android, Linux, and Mac iOS out of the box.  Competing and evolving lines, between the iOS tablets. How about mobile hardware from Motorola, Nokia, or Blackberry using iOS?

Allow Customers to Buy Gasoline as Units at Current Price on Loyalty Debit Cards

A legitimate reason IS NEEDED why in this day and age of commodities trading, storage, pipelines, trains, tankers, and trucks, why if I can buy stocks at current market value, or go on Ebay and buy 50 yards of antique bobbed wire at an auction, why can’t I go to a gas station and buy 50 UNITS of gas at the current price of gas that day? Upon return to the same brand gas station, I should be able to use my same loyalty debit card and subtract units of gas from the card, instead of the current price of gas that day.

EVs “in the field” Use Existing Home Energy Provider, with Transportation Charges Applied

Similar logic can be applied to Electric, and EVs but with a twist, incorporating the use of your home “energy provider”, and when charging ‘on the go’, only pay local “line usage/distribution” fees.

Full Article Here

Fundraising Using Public WiFi, and an UL/DL MB Meter Reading

What if commercial, public WiFi Hotspots partnered with a fundraising cause, and every MB exchanged (up and/or down), a donation of N cents would go to a charity-sponsored by you with Paypal. The business providing the WiFi would match the donation.

Greece Prosperity / Tourism: The World will Come See You in Augmented Reality (AR)

While perusing through all of the ruins, looking at the sites, watching the tour groups, and the tour guides explaining these empty ruins, I pondered, wouldn’t it be amazing to see the people of ancient times dressed in their clothing of the times, interacting with each other through the ruins as though the tourists were not even there. In effect, acting out scenes that perhaps took place thousands of years ago, echoes of the past. I thought why wouldn’t for starters, the government pays the people of the Arts and Sciences to go through scenes, such as basic interactions with a Librarian, studying in one corner, ignoring the world, and just reading and thinking, and in another corner of this library, there might be a quiet debate going on, in Greek of course, and through a translation application, any foreigner could hear their native language the interactions. It could be ancient commoners, to known ancient people of the past acting out scenes like echoes of the past, while tours just come up close and personal, pass by, even wave their hand in front of an actors hand, and he continues to act as if the tourists were not even there. All of these Greek actors would be prerecorded and play out in Augmented Reality (AR).

Full Article Here

 Tune in next time for more “What If…” episodes.

Bose AR, Audio Augmented Reality – Use Cases

I’ve been enamored with Bose products for well over a decade. However,  we’ve seen quality brands enter the hi-fidelity audio market over that time.  Beyond quality design in their classic audio products, can Bose Augmented Reality (Bose AR) be the market differentiator?

Bose: Using a Bose-AR-equipped wearable, a smartphone, and an app-enabled with Bose AR, the new platform lets you hear what you see.

It sounds like Bose may come up with an initial design, sunglasses, but turn to 3rd party hardware manufacturers of all sorts to integrate Bose AR into other wearable products.

Bose Augmented Reality isn’t just about audio. The devices will use sensors to track head motions for gesture controls and work with GPS from a paired smartphone to track location.  The company also aspires to combine visual information with the Bose AR platform.

Bose AR Use Cases

  • Bose Augmented Reality device reenact historical events or speeches from landmarks and statues as you visit them.
  • The Bose and NFL partnership could be leveraged to get these AR units into the football player’s helmets.  Audio queues from the on-field lead, quarterback, and dynamically replayed/relayed at the appropriate time of required action by the receiver.
  • Audio directions to your gate when your GPS detects that you’ve arrived at the airport, or any other destination from your calendar.  Audio queues would be richer the more inclusive you are to the access to Calendars, To Do lists, etc.
  • Combine visual information with the Bose AR platform, too, so you could hear a translation of a sign you’re looking at.
  • Hear the history of a painting in a museum.

Time until it’s in consumer’s hands?  TBD.  Bose objective is to have the developer kit, including a pair of glasses, available later this year.

When I was on vacation in Athens, Greece, I created a post which had Greek actors running tours in their ancient, native garb.  The Bose AR could be a complementary offering to the tour, which includes live, greek local actors portraying out scenes in ancient ruins.  Record the scenes, and interact with them while walking through the Greek ruins in your Bose AR (Augmented Reality) glasses.

Greece, Prosperity, and Taxes: The World Will Come See You in AR

Please take a moment to prioritize the use cases, or add your own.

Takeaway

I’m a cheerleader for Bose, among several others in this space, but I question a Bose AR headset that produces a high fidelity sound. Most of the use cases listed should be able to “get along OK” with an average quality sound.  Maybe high definition AR games with a high level of realism might benefit from the high-quality sound. However, their site reads like Bose is positioning themselves as a component to be integrated into other AR headsets, i.e. “Bose-AR-equipped wearable