Featured Article

Why Apple is taking a small-model approach to generative AI

Apple Intelligence is more bespoke than larger models, with a focus on user experience

Comment

Apple Software Engineering SVP Craig Federighi, seen presenting Apple Intelligence at WWDC 2024
Image Credits: Apple

Among the biggest questions surrounding models like ChatGPT, Gemini and Midjourney since launch is what role (if any) they’ll play in our daily lives. It’s something Apple is striving to answer with its own take on the category, Apple Intelligence, which was officially unveiled this week at WWDC 2024.

The company led with flash at Monday’s presentation; that’s just how keynotes work. When SVP Craig Federighi wasn’t skydiving or performing parkour with the aid of some Hollywood (well, Cupertino) magic, Apple was determined to demonstrate that its in-house models were every bit as capable as the competition’s.

The jury is still out on that question, with the betas having only dropped Monday, but the company has since revealed some of what makes its approach to generative AI different. First and foremost is scope. Many of the most prominent companies in the space take a “bigger is better” approach to their models. The goal of these systems is to serve as a kind of one-stop shop to the world’s information.

Apple’s approach to the category, on the other hand, is grounded in something more pragmatic. Apple Intelligence is a more bespoke approach to generative AI, built specifically with the company’s different operating systems at their foundation. It’s a very Apple approach in the sense that it prioritizes a frictionless user experience above all.

Apple Intelligence is a branding exercise in one sense, but in another, the company prefers the generative AI aspects to seamlessly blend into the operating system. It’s completely fine — or even preferred, really — if the user has no concept of the underlying technologies that power these systems. That’s how Apple products have always worked.

Keeping the models small

The key to much of this is creating smaller models: training the systems on a customized dataset designed specifically for the kinds of functionality required by users of its operating systems. It’s not immediately clear how much the size of these models will affect the black box issue, but Apple thinks that, at the very least, having more topic-specific models will increase the transparency around why the system makes specific decisions.

Due to the relatively limited nature of these models, Apple doesn’t expect that there will be a huge amount of variety when prompting the system to, say, summarize text. Ultimately, however, the variation from prompt to prompt depends on the length of the text being summarized. The operating systems also feature a feedback mechanism into which users can report issues with the generative AI system.

While Apple Intelligence is much more focused than larger models, it can cover a spectrum of requests, thanks to the inclusion of “adapters,” which are specialized for different tasks and styles. Broadly, however, Apple’s is not a “bigger is better” approach to creating models, as things like size, speed and compute power need to be taken into account — particularly when dealing with on-device models.

ChatGPT, Gemini and the rest

Opening up to third-party models like OpenAI’s ChatGPT makes sense when considering the limited focus of Apple’s models. The company trained its systems specifically for the macOS/iOS experience, so there’s going to be plenty of information that is out of its scope. In cases where the system thinks a third-party application would be better suited to provide a response, a system prompt will ask whether you want to share that information externally. If you don’t receive a prompt like this, the request is being processed with Apple’s in-house models.

This should function the same with all external models Apple partners with, including Google Gemini. It’s one of the rare instances where the system will draw attention to its use of generative AI in this way. The decision was made, in part, to squash any privacy concerns. Every company has different standards when it comes to collecting and training on user data.

Requiring users to opt-in each time removes some of the onus from Apple, even if it does add some friction into the process. You can also opt-out of using third-party platforms systemwide, though doing so would limit the amount of data the operating system/Siri can access. You cannot, however, opt-out of Apple Intelligence in one fell swoop. Instead, you will have to do so on a feature by feature basis.

Private Cloud Compute

Whether the system processes a specific query on device or via a remote server with Private Cloud Compute, on the other hand, will not be made clear. Apple’s philosophy is that such disclosures aren’t necessary, since it holds its servers to the same privacy standards as its devices, down to the first-party silicon they run on.

One way to know for certain whether the query is being managed on- or off-device is to disconnect your machine from the internet. If the problem requires cloud computing to solve, but the machine can’t find a network, it will throw up an error noting that it cannot complete the requested action.

Apple is breaking down the specifics surrounding which actions will require cloud-based processing. There are several factors at play there, and the ever-changing nature of these systems means something that could require cloud compute today might be able to be accomplished on-device tomorrow. On-device computing won’t always be the faster option, as speed is one of the parameters Apple Intelligence factors in when determining where to process the prompt.

There are, however, certain operations that will always be performed on-device. The most notable of the bunch is Image Playground, as the full diffusion model is stored locally. Apple tweaked the model so it generates images in three different house styles: animation, illustration and sketch. The animation style looks a good bit like the house style of another Steve Jobs-founded company. Similarly, text generation is currently available in a trio of styles: friendly, professional and concise.

Even at this early beta stage, Image Playground’s generation is impressively quick, often only taking a couple of seconds. As for the question of inclusion when generating images of people, the system requires you to input specifics, rather than simply guessing at things like ethnicity.

How Apple will handle datasets

Apple’s models are trained on a combination of licensed datasets and by crawling publicly accessible information. The latter is accomplished with AppleBot. The company’s web crawler has been around for some time now, providing contextual data to applications like Spotlight, Siri and Safari. The crawler has an existing opt-out feature for publishers.

“With Applebot-Extended,” Apple notes, “web publishers can choose to opt out of their website content being used to train Apple’s foundation models powering generative AI features across Apple products, including Apple Intelligence, Services, and Developer Tools.”

This is accomplished with the inclusion of a prompt within the website’s code. With the advent of Apple Intelligence, the company has introduced a second prompt, which allows sites to be included in search results but excluded for generative AI model training.

Responsible AI

Apple released a whitepaper on the first day of WWDC titled, “Introducing Apple’s On-Device and Server Foundation Models.” Among other things, it highlights principles governing the company’s AI models. In particular, Apple highlights four things:

  1. “Empower users with intelligent tools: We identify areas where AI can be used responsibly to create tools for addressing specific user needs. We respect how our users choose to use these tools to accomplish their goals.”
  2. “Represent our users: We build deeply personal products with the goal of representing users around the globe authentically. We work continuously to avoid perpetuating stereotypes and systemic biases across our AI tools and models.”
  3. “Design with care: We take precautions at every stage of our process, including design, model training, feature development, and quality evaluation to identify how our AI tools may be misused or lead to potential harm. We will continuously and proactively improve our AI tools with the help of user feedback.”
  4. “Protect privacy: We protect our users’ privacy with powerful on-device processing and groundbreaking infrastructure like Private Cloud Compute. We do not use our users’ private personal data or user interactions when training our foundation models.”

Apple’s bespoke approach to foundational models allows the system to be tailored specifically to the user experience. The company has applied this UX-first approach since the arrival of the first Mac. Providing as frictionless an experience as possible serves the user, but it should not be done at the expense of privacy.

This is going to be a difficult balancing act the company will have to navigate as the current crop of OS betas reach general availability this year. The ideal approach is to offer up as much — or little — information as the end user requires. Certainly there will be plenty of people who don’t care, say, whether or not a query is executed on-machine or in the cloud. They’re content to have the system default to whatever is the most accurate and efficient.

For privacy advocates and others who are interested in those specifics, Apple should strive for as much user transparency as possible — not to mention transparency for publishers that might prefer not to have their content sourced to train these models. There are certain aspects with which the black box problem is currently unavoidable, but in cases where transparency can be offered, it should be made available upon users’ request.

More TechCrunch

Fisker is just a few days into its Chapter 11 bankruptcy, and the fight over its assets is already charged, with one lawyer claiming the startup has been liquidating assets…

The fight over Fisker’s assets is already heating up

A hacker is advertising customer data allegedly stolen from the Australia-based live events and ticketing company TEG on a well-known hacking forum. On Thursday, a hacker put up for sale…

Hacker claims to have 30 million customer records from Australian ticket seller giant TEG

Welcome to Startups Weekly — Haje‘s weekly recap of everything you can’t miss from the world of startups. Sign up here to get it in your inbox every Friday. Elon…

Tesla makes Musk best-paid CEO of all time and Fisker bites the dust

Dot is a new AI companion and chatbot that thrives on getting to know your innermost thoughts and feelings.

Dot’s AI really, really wants to get to know you

The e-fuels startup is working on producing fuel for aviation and maritime shipping using carbon dioxide and other waste carbon streams.

E-fuels startup Aether Fuels is raising $34.3 million, per filing

Fisker was facing “potential financial distress” as early as last August, according to a new filing in its Chapter 11 bankruptcy proceeding, which the EV startup initiated earlier this week.…

Fisker faced financial distress as early as last August

Cruise, the self-driving subsidiary of General Motors, has agreed to pay a $112,500 fine for failing to provide full information about an accident involving one of its robotaxis last year.…

Cruise clears key hurdle to getting robotaxis back on roads in California

Feel Therapeutics has a pretty original deck, with some twists we rarely see; the company did a great job telling the overall story.

Pitch Deck Teardown: Feel Therapeutics’ $3.5M seed deck

The Rockset buy fits into OpenAI’s broader recent strategy of investing heavily in its enterprise sales and tech orgs.

OpenAI buys Rockset to bolster its enterprise AI

The U.S. government announced sanctions against 12 executives and senior leaders of the Russia-based cybersecurity giant Kaspersky. In a press release, the Department of the Treasury’s Office of Foreign Assets…

US government sanctions Kaspersky executives

Style DNA, an AI-powered fashion stylist app, creates a personalized style profile from a single selfie. The app is particularly useful for people interested in seasonal color analysis, a process…

Style DNA gets a generative AI chatbot that suggests outfit ideas based on your color type

Rates of depression, anxiety and suicidal thoughts are surging among U.S. teens. A recent report from the Center of Disease Control found that nearly one in three girls have seriously…

Khosla-backed Marble, built by former Headway founders, offers affordable group therapy for teens

Cover says what sets it apart is the underlying technology it employs, which has been exclusively licensed from NASA’s Jet Propulsion Laboratory.

A new startup from Figure’s founder is licensing NASA tech in a bid to curb school shootings

Spotify is introducing a new “Basic” streaming plan in the United States, the company announced on Friday. The new plan costs $10.99 per month and includes all of the benefits…

Spotify launches a new Basic streaming plan in the US

Photographers say the social media giant is applying a ‘Made with AI’ label to photos they took, causing confusion for users.

Meta is tagging real photos as ‘Made with AI,’ say photographers

Website building platform Squarespace is selling Tock, its restaurant reservation service, to American Express in a deal worth $400 million — the exact figure that Squarespace paid for the service…

Squarespace sells restaurant reservation system Tock to American Express for $400M

Featured Article

Change Healthcare confirms ransomware hackers stole medical records on a ‘substantial proportion’ of Americans

The February ransomware attack on UHG-owned Change Healthcare stands as one of the largest-ever known digital thefts of U.S. medical records.

19 hours ago
Change Healthcare confirms ransomware hackers stole medical records on a ‘substantial proportion’ of Americans

Google said today that it globally paused its experiment that aimed to allow new kinds of real-money games on the Play Store, citing the challenges that come with the lack…

Google pauses its experiment to expand real-money games on the Play Store

Venture firms raised $9.3 billion in Q1 according to PitchBook data, which means this year likely won’t match or surpass 2023’s $81.8 billion total. While emerging managers are feeling the…

Kevin Hartz’s A* raises its second oversubscribed fund in three years

Google is making reviews of all your movies, TV shows, books, albums and games visible under one profile page starting June 24, according to an email sent to users last…

Google is making your movie and TV reviews visible under a new profile page

Zepto, an Indian quick commerce startup, has more than doubled its valuation to $3.6 billion in a new funding round of $665 million.

Zepto, a 10-minute delivery app, raises $665M at $3.6B valuation

Speak, the AI-powered language learning app, has raised new money from investors at double its previous valuation.

Language learning app Speak nets $20M, doubles valuation

SpaceX unveiled Starlink Mini, a more portable version of its satellite internet product that is small enough to fit inside a backpack.  Early Starlink customers were invited to purchase the…

SpaceX debuts portable Starlink Mini for $599

Ali Rathod-Papier has stepped down from her role as global head of compliance at corporate card expense management startup Brex to join venture firm Andreessen Horowitz (a16z) as a partner…

Brex’s compliance head has left the fintech startup to join Andreessen Horowitz as a partner

U.S. officials imposed the “first of its kind” ban arguing that Kaspersky threatens U.S. national security because of its links to Russia.

US bans sale of Kaspersky software citing security risk from Russia 

Apple has released Final Cut Pro for iPad 2 and Final Cut Camera, the company announced on Thursday. Both apps were previously announced during the company’s iPad event in May.…

Apple releases Final Cut Pro for iPad 2 and Final Cut Camera

Paris has quickly established itself as a major European center for AI startups, and now another big deal is in the works.

Poolside is raising $400M+ at a $2B valuation to build a supercharged coding co-pilot

The space industry is all abuzz about how SpaceX’s Starship, Blue Origin’s New Glenn, and other heavy-lift rockets will change just about everything. One likely consequence is that spacecraft will…

Gravitics prepares a testing gauntlet for a new generation of giant spacecraft

LTK (formerly LiketoKnow.it and RewardStyle), the influencer shopping app with 40 million monthly users, announced on Thursday the launch of a free direct message tool for creators to instantly share…

Influencer shopping app LTK gets an automatic direct message tool

YouTube appears to be taking a firm stance against Premium subscribers who attempt to use a VPN (virtual private network) to access cheaper subscription prices in other countries. This week,…

YouTube confirms crackdown on VPN users accessing cheaper Premium plans