The Ray-Ban Meta smart glasses are getting AI-powered visual search features

Key Takeaways:

– The Ray-Ban Meta smart glasses are receiving upgrades to their AI assistant.
– The improvements include real-time information access and multimodal capabilities.
– Previously, the AI had a knowledge cutoff, but now it can provide current information.
– Bing plays a role in powering the real-time information access.
– The multimodal AI allows the assistant to answer contextual questions based on what the user is looking at through the glasses.
– These updates aim to enhance the usefulness of the Meta AI and make it less gimmicky.
– Initially, the new capabilities will be available to a small number of people in the US through an early access beta version.
– Expanded access is expected to be available in 2024.
– Commands to engage the multimodal feature start with “Hey Meta, look and tell me.”
– Users can ask the AI about their surroundings and request creative assistance.
– Videos and screenshots provide an idea of what the new capabilities can do.


The Ray-Ban Meta smart glasses are about to get some powerful upgrades thanks to improvements to the social network’s AI assistant. The company is finally for real-time information to the onboard assistant, and it’s starting to test new “multimodal” capabilities that allow it to answer questions based on your environment.

Up to now, Meta AI had a “knowledge cutoff” of December 2022, so it couldn’t answer questions about current events, or things like game scores, traffic conditions or other queries that would be especially useful while on the go. But that’s now changing, according to Meta CTO Andrew Bosworth, who said that all Meta smart glasses in the United States will now be able to access real-time info. The change is powered “in part” by Bing, he added.

Separately, Meta is starting to test one of the more intriguing capabilities of its assistant, which it’s calling “multimodal AI.” The features, first during Connect, allow Meta AI to answer contextual questions about your surroundings and other queries based on what your looking at through the glasses.


The updates could go a long way toward making Meta AI feel less gimmicky and more useful, which was one of my top complaints in my of the otherwise impressive smart glasses. Unfortunately, it will likely still be some time before most people with the smart glasses can access the new multimodal functionality. Bosworth said that the early access beta version will only be available in the US to a “small number of people who opt in” initially, with expanded access presumably coming sometime in 2024.

Both shared a few videos of the new capabilities that give an idea of what may be possible. Based on the clips, it appears users will be able to engage the feature with commands that begin with “Hey Meta, look and tell me.” Zuckerberg, for example, asks Meta AI to look at a shirt he’s holding and ask for suggestions on pants that might match. He also shared screenshots showing Meta AI identifying an image of a piece of fruit and translating the text of a meme.

In a posted on Threads, Bosworth said that users would also be able to ask Meta AI about their immediate surroundings as well as more creative questions like writing captions for photos they just shot.

This article originally appeared on Engadget at
Source link

AI Eclipse TLDR:

Meta, the company behind Ray-Ban Meta smart glasses, is introducing powerful upgrades to its AI assistant. The social network’s AI assistant will now have access to real-time information, allowing it to provide up-to-date answers and perform tasks such as providing game scores, traffic conditions, and current event updates. The change is made possible, in part, by integrating Bing into the AI assistant. Additionally, Meta is testing a new feature called “multimodal AI,” which enables the assistant to answer contextual questions based on the user’s surroundings through the glasses. These updates aim to make Meta AI more useful and less gimmicky. However, the new multimodal functionality will initially be available to a small number of users in the US, with wider access expected in 2024. Meta CEO Andrew Bosworth also mentioned that users will be able to ask the AI assistant about their immediate surroundings and engage in creative interactions like writing captions for photos. The improvements to the Meta smart glasses are expected to enhance the overall user experience.