AI

Using memes, social media users have become red teams for half-baked AI features

Comment

Image Credits: efenzi / Getty Images

“Running with scissors is a cardio exercise that can increase your heart rate and require concentration and focus,” says Google’s new AI search feature. “Some say it can also improve your pores and give you strength.”

Google’s AI feature pulled this response from a website called Little Old Lady Comedy, which, as its name makes clear, is a comedy blog. But the gaffe is so ridiculous that it’s been circulating on social media, along with other obviously incorrect AI overviews on Google. Effectively, everyday users are now red teaming these products on social media.

In cybersecurity, some companies will hire “red teams” – ethical hackers – who attempt to breach their products as though they’re bad actors. If a red team finds a vulnerability, then the company can fix it before the product ships. Google certainly conducted a form of red teaming before releasing an AI product on Google Search, which is estimated to process trillions of queries per day.

It’s surprising, then, when a highly resourced company like Google still ships products with obvious flaws. That’s why it’s now become a meme to clown on the failures of AI products, especially in a time when AI is becoming more ubiquitous. We’ve seen this with bad spelling on ChatGPT, video generators’ failure to understand how humans eat spaghetti, and Grok AI news summaries on X that, like Google, don’t understand satire. But these memes could actually serve as useful feedback for companies developing and testing AI.

Despite the high-profile nature of these flaws, tech companies often downplay their impact.

“The examples we’ve seen are generally very uncommon queries, and aren’t representative of most people’s experiences,” Google told TechCrunch in an emailed statement. “We conducted extensive testing before launching this new experience, and will use these isolated examples as we continue to refine our systems overall.”

Not all users see the same AI results, and by the time a particularly bad AI suggestion gets around, the issue has often already been rectified. In a more recent case that went viral, Google suggested that if you’re making pizza but the cheese won’t stick, you could add about an eighth of a cup of glue to the sauce to “give it more tackiness.” As it turned out, the AI is pulling this answer from an eleven-year-old Reddit comment from a user named “f––smith.”

Beyond being an incredible blunder, it also signals that AI content deals may be overvalued. Google has a $60 million contract with Reddit to license its content for AI model training, for instance. Reddit signed a similar deal with OpenAI last week, and Automattic properties WordPress.org and Tumblr are rumored to be in talks to sell data to Midjourney and OpenAI.

To Google’s credit, a lot of the errors that are circulating on social media come from unconventional searches designed to trip up the AI. At least I hope no one is seriously searching for “health benefits of running with scissors.” But some of these screw-ups are more serious. Science journalist Erin Ross posted on X that Google spit out incorrect information about what to do if you get a rattlesnake bite.

Ross’s post, which got over 13,000 likes, shows that AI recommended applying a tourniquet to the wound, cutting the wound and sucking out the venom. According to the U.S. Forest Service, these are all things you should not do, should you get bitten. Meanwhile on Bluesky, the author T Kingfisher amplified a post that shows Google’s Gemini misidentifying a poisonous mushroom as a common white button mushroom – screenshots of the post have spread to other platforms as a cautionary tale.

When a bad AI response goes viral, the AI could get more confused by the new content around the topic that comes about as a result. On Wednesday, New York Times reporter Aric Toler posted a screenshot on X that shows a query asking if a dog has ever played in the NHL. The AI’s response was yes – for some reason, the AI called the Calgary Flames player Martin Pospisil a dog. Now, when you make that same query, the AI pulls up an article from the Daily Dot about how Google’s AI keeps thinking that dogs are playing sports. The AI is being fed its own mistakes, poisoning it further.

This is the inherent problem of training these large-scale AI models on the internet: sometimes, people on the internet lie. But just like how there’s no rule against a dog playing basketball, there’s unfortunately no rule against big tech companies shipping bad AI products.

As the saying goes: garbage in, garbage out.

More TechCrunch

Fisker is just a few days into its Chapter 11 bankruptcy, and the fight over its assets is already charged, with one lawyer claiming the startup has been liquidating assets…

The fight over Fisker’s assets is already heating up

A hacker is advertising customer data allegedly stolen from the Australia-based live events and ticketing company TEG on a well-known hacking forum. On Thursday, a hacker put up for sale…

Hacker claims to have 30 million customer records from Australian ticket seller giant TEG

Welcome to Startups Weekly — Haje‘s weekly recap of everything you can’t miss from the world of startups. Sign up here to get it in your inbox every Friday. Elon…

Tesla makes Musk best-paid CEO of all time and Fisker bites the dust

Dot is a new AI companion and chatbot that thrives on getting to know your innermost thoughts and feelings.

Dot’s AI really, really wants to get to know you

The e-fuels startup is working on producing fuel for aviation and maritime shipping using carbon dioxide and other waste carbon streams.

E-fuels startup Aether Fuels is raising $34.3 million, per filing

Fisker was facing “potential financial distress” as early as last August, according to a new filing in its Chapter 11 bankruptcy proceeding, which the EV startup initiated earlier this week.…

Fisker faced financial distress as early as last August

Cruise, the self-driving subsidiary of General Motors, has agreed to pay a $112,500 fine for failing to provide full information about an accident involving one of its robotaxis last year.…

Cruise clears key hurdle to getting robotaxis back on roads in California

Feel Therapeutics has a pretty original deck, with some twists we rarely see; the company did a great job telling the overall story.

Pitch Deck Teardown: Feel Therapeutics’ $3.5M seed deck

The Rockset buy fits into OpenAI’s broader recent strategy of investing heavily in its enterprise sales and tech orgs.

OpenAI buys Rockset to bolster its enterprise AI

The U.S. government announced sanctions against 12 executives and senior leaders of the Russia-based cybersecurity giant Kaspersky. In a press release, the Department of the Treasury’s Office of Foreign Assets…

US government sanctions Kaspersky executives

Style DNA, an AI-powered fashion stylist app, creates a personalized style profile from a single selfie. The app is particularly useful for people interested in seasonal color analysis, a process…

Style DNA gets a generative AI chatbot that suggests outfit ideas based on your color type

Rates of depression, anxiety and suicidal thoughts are surging among U.S. teens. A recent report from the Center of Disease Control found that nearly one in three girls have seriously…

Khosla-backed Marble, built by former Headway founders, offers affordable group therapy for teens

Cover says what sets it apart is the underlying technology it employs, which has been exclusively licensed from NASA’s Jet Propulsion Laboratory.

A new startup from Figure’s founder is licensing NASA tech in a bid to curb school shootings

Spotify is introducing a new “Basic” streaming plan in the United States, the company announced on Friday. The new plan costs $10.99 per month and includes all of the benefits…

Spotify launches a new Basic streaming plan in the US

Photographers say the social media giant is applying a ‘Made with AI’ label to photos they took, causing confusion for users.

Meta is tagging real photos as ‘Made with AI,’ say photographers

Website building platform Squarespace is selling Tock, its restaurant reservation service, to American Express in a deal worth $400 million — the exact figure that Squarespace paid for the service…

Squarespace sells restaurant reservation system Tock to American Express for $400M

Featured Article

Change Healthcare confirms ransomware hackers stole medical records on a ‘substantial proportion’ of Americans

The February ransomware attack on UHG-owned Change Healthcare stands as one of the largest-ever known digital thefts of U.S. medical records.

13 hours ago
Change Healthcare confirms ransomware hackers stole medical records on a ‘substantial proportion’ of Americans

Google said today that it globally paused its experiment that aimed to allow new kinds of real-money games on the Play Store, citing the challenges that come with the lack…

Google pauses its experiment to expand real-money games on the Play Store

Venture firms raised $9.3 billion in Q1 according to PitchBook data, which means this year likely won’t match or surpass 2023’s $81.8 billion total. While emerging managers are feeling the…

Kevin Hartz’s A* raises its second oversubscribed fund in three years

Google is making reviews of all your movies, TV shows, books, albums and games visible under one profile page starting June 24, according to an email sent to users last…

Google is making your movie and TV reviews visible under a new profile page

Zepto, an Indian quick commerce startup, has more than doubled its valuation to $3.6 billion in a new funding round of $665 million.

Zepto, a 10-minute delivery app, raises $665M at $3.6B valuation

Speak, the AI-powered language learning app, has raised new money from investors at double its previous valuation.

Language learning app Speak nets $20M, doubles valuation

SpaceX unveiled Starlink Mini, a more portable version of its satellite internet product that is small enough to fit inside a backpack.  Early Starlink customers were invited to purchase the…

SpaceX debuts portable Starlink Mini for $599

Ali Rathod-Papier has stepped down from her role as global head of compliance at corporate card expense management startup Brex to join venture firm Andreessen Horowitz (a16z) as a partner…

Brex’s compliance head has left the fintech startup to join Andreessen Horowitz as a partner

U.S. officials imposed the “first of its kind” ban arguing that Kaspersky threatens U.S. national security because of its links to Russia.

US bans sale of Kaspersky software citing security risk from Russia 

Apple has released Final Cut Pro for iPad 2 and Final Cut Camera, the company announced on Thursday. Both apps were previously announced during the company’s iPad event in May.…

Apple releases Final Cut Pro for iPad 2 and Final Cut Camera

Paris has quickly established itself as a major European center for AI startups, and now another big deal is in the works.

Poolside is raising $400M+ at a $2B valuation to build a supercharged coding co-pilot

The space industry is all abuzz about how SpaceX’s Starship, Blue Origin’s New Glenn, and other heavy-lift rockets will change just about everything. One likely consequence is that spacecraft will…

Gravitics prepares a testing gauntlet for a new generation of giant spacecraft

LTK (formerly LiketoKnow.it and RewardStyle), the influencer shopping app with 40 million monthly users, announced on Thursday the launch of a free direct message tool for creators to instantly share…

Influencer shopping app LTK gets an automatic direct message tool

YouTube appears to be taking a firm stance against Premium subscribers who attempt to use a VPN (virtual private network) to access cheaper subscription prices in other countries. This week,…

YouTube confirms crackdown on VPN users accessing cheaper Premium plans