Monday was the kickoff of WWDC 2024, the annual event where Apple announces some of the biggest features headed to its devices, apps and software. And this year’s WWDC is a doozy. Thanks to Apple’s newfound — and heavy — investment in generative AI tech, the company had loads to showcase on the AI front, from an upgraded Siri to AI-generated emoji.
Apple announced a deal with OpenAI to bring ChatGPT, OpenAI’s AI-powered chatbot experience, to a range of its devices. It introduced new photo-editing tools to remove objects and people from photos. And it rolled out an AI capability to proofread, rewrite and summarize text across content, including notes to emails.
Here’s a roundup of a few of the more noteworthy Apple AI announcements from WWDC 2024.
New Siri
Siri got a makeover courtesy of Apple’s overarching generative AI push this year, called Apple Intelligence.
Thanks to Apple Intelligence, Siri, which has a revamped look, including a new icon and glowing indicator light around the edges of a device’s screen, can now handle stumbles in speech and better understand context. You can now also type to Siri, and it can answer questions, including questions about how to use your iPhone, iPad or Mac.
Soon, Siri will become even more capable with onscreen awareness and the ability to take action in and across apps — so you can ask Siri to, for example, “make this photo pop” and then “add this photo” to another app. During the WWDC keynote on Monday, Apple gave the example of Siri finding a photo of your license, extracting your ID number and entering it into a web form for you.
To take advantage of the new Siri, you’ll need an Apple device that supports Apple Intelligence — specifically the iPhone 15 Pro and devices with M1 or newer chips.
ChatGPT integration
Apple’s bringing ChatGPT to Siri and other first-party apps and capabilities across its operating systems.
Siri users will soon be able to route questions to ChatGPT for “expertise” where it might be helpful, Apple says. You can include photos with the questions you ask ChatGPT via Siri, or ask questions related to your docs or PDFs.
Apple’s also integrated ChatGPT into OS-wide tools such as Writing Tools (powered by Apple Intelligence), which allows you to create content with ChatGPT — including images — or ask an initial idea and send it to ChatGPT to get a revision or variation back.
ChatGPT integrations will arrive on iOS 18, iPadOS 18 and macOS Sequoia later this year, Apple says, and will be free without the need to create a ChatGPT or OpenAI account. Initially, they’ll be powered by GPT-4o, OpenAI’s recently introduced flagship generative AI model.
Genmoji and Image Playground
Genmoji, coming soon to iOS 18 on devices that support Apple Intelligence, lets you create AI emoji-like images of anyone in your photo library — or just custom emoji. Genmoji can be used as a sticker for reacting to messages with a Tapback or inline with your messages, Apple says.
A separate new image-generation capability allows iPhone users to create AI images of people they’re messaging with. Apple Intelligence will have an understanding of who you’re chatting with, Apple says — so if you want to personalize the chat with a custom AI image, you can do so on the fly.
There’s also Image Playground, a new image-generating feature that works across apps like Notes, Freeform, Keynote and Pages.
Available as a standalone app and API for developers, Image Playground allows users to create images using concepts such as themes, costumes, accessories, places and more. You select the themes you want to include and, minutes later, Image Playground creates a preview of your image.
AI photo editing
Apple’s new Clean Up tool, built into the upgraded Photos app, removes unwanted people and objects from pics.
Clean Up can be used on any picture in Photos by circling or highlighting the thing — or person — to be removed. Relying on AI, Clean Up removes the selected element and replaces it with contextually-aware pixels to try to make it seem like the thing (or person) was never there to begin with.
On the subject of Photos, it’s now better organized thanks to AI. Coming in iOS 18, Photos will show collections of photos automatically organized by topics like time, people, favorite memories, trips and more. And you’ll be able to use more specific terms to search across photos.
Transcribed calls
Coming soon to iPhone 15 Pro and newer models, iOS will optionally record and transcribe your phone calls.
The feature — which must be enabled manually, and which informs the party on the other end of the line that the call is being recorded so as to not run afoul of privacy laws — transcribes what’s said during the call and then provides a summary of the key points discussed in iOS’ Notes app.
Apple noted (no pun intended) during the keynote on Monday that you’ll also be able to record and transcribe audio from within the Notes app.
Comment