Google shows off how AI Gmail, Photos and shopping will work

Sundar Pichai speaks about Gemini 1.5 pro during Google I/O developer conference on May 14. (Google / CNN Newsource)

A day after OpenAI impressed with a startlingly improved ChatGPT AI model, Google showed off its vision for how AI will improve the products that billions of people use every day.

The updates, announced at its annual Google I/O developer conference, come as the company is trying to push beyond its core advertising business with new devices and AI-powered tools.

Google showed how it wants its AI products to become a bigger part of users’ lives by sharing information, interacting with others, finding objects around the house, making schedules, shopping and using an Android device. Simply put, Google wants its AI to be part of everything you do.

Google CEO Sundar Pichai showed off various new features powered by its latest AI model Gemini 1.5 Pro. One new feature, called Ask Photos, allows users to search photos for deeper insights, such as asking when your daughter learned to swim or recall what your license plate number is, by looking through saved pictures.

Google executives also demonstrated could “read” a textbook and turn it into a kind of AI lecture, featuring natural-sounding teachers that’ll answer your questions.

AI boosts are coming to Gmail, too. Pichai showed how users can ask Gemini 1.5 Pro to summarize all recent emails from your child’s school by analyzing attachments, and summarizing key points and spitting out action items.

Just one day before, OpenAI — one of the tech industry’s leaders in artificial intelligence — unveiled a new AI model that it says will make chatbot ChatGPT smarter and easier to use. GPT-4o aims to turn ChatGPT into a digital personal assistant that can engage in real-time, spoken conversations and interact using text and “vision.” This means it can view screenshots, photos, documents or charts uploaded by users and have a conversation about them.

Google also showed off Gemini’s latest abilities to take different kinds of input — “multimodal” capabilities to take in text, voice or images — as a direct response to ChatGPT’s voice mode. An executive also demoed a virtual “teammate” that can help stay on top of to-do lists, organize data and manage workflow.

The company also highlighted search improvements by allowing users to ask more natural or more focused questions, and providing various versions of the responses, such as in-depth or summarized results. For example, the AI can suggest kid friendly restaurants based on location, or what might be wrong with a gadget, such as a camera, by taking a video of it via Google Lens. The goal is essentially to take the legwork out of searching on Google, the company said.

Google also briefly teased Project Astra, developed by Google’s DeepMind AI lab, which will allow AI assistants to help users’ everyday lives by using phone cameras to interpret information about the real world, such as identifying objects and even finding misplaced items. It also hinted at how it would work on augmented reality glasses.

The company said that later this year it will integrate more AI functions into phones. For example, users will be able to drag and drop images created by AI into Google Messages and Gmail, and ask questions about YouTube videos and PDFs on an Android device. The AI tool can also help detect suspicious activity, such as scam calls trying to imitate a user’s bank.

According to analyst Jacob Bourne, from market research firm Emarketer, it’s no surprise AI took center stage at this year’s Google developer conference.

“By showcasing its latest models and how they’ll power existing products with strong consumer reach, Google is demonstrating how it can effectively differentiate itself from rivals,” he said.

He believes the reception of the new tools will be an indicator of how well Google can adapt its search product to meet the demands of the generative AI era.

“To maintain its competitive edge and satisfy investors, Google will need to focus on translating its AI innovations into profitable products and services at scale,” he said.