AI

This Week in AI: Ex-OpenAI staff call for safety and transparency

Comment

OpenAI logo with spiraling pastel colors (Image Credits: Bryce Durbin / TechCrunch)
Image Credits: Bryce Durbin / TechCrunch

Hiya, folks, and welcome to TechCrunch’s inaugural AI newsletter. It’s truly a thrill to type those words — this one’s been long in the making, and we’re excited to finally share it with you.

With the launch of TC’s AI newsletter, we’re sunsetting This Week in AI, the semiregular column previously known as Perceptron. But you’ll find all the analysis we brought to This Week in AI and more, including a spotlight on noteworthy new AI models, right here.

This week in AI, trouble’s brewing — again — for OpenAI.

A group of former OpenAI employees spoke with The New York Times’ Kevin Roose about what they perceive as egregious safety failings within the organization. They — like others who’ve left OpenAI in recent months — claim that the company isn’t doing enough to prevent its AI systems from becoming potentially dangerous and accuse OpenAI of employing hardball tactics to attempt to prevent workers from sounding the alarm.

The group published an open letter on Tuesday calling for leading AI companies, including OpenAI, to establish greater transparency and more protections for whistleblowers. “So long as there is no effective government oversight of these corporations, current and former employees are among the few people who can hold them accountable to the public,” the letter reads.

Call me pessimistic, but I expect the ex-staffers’ calls will fall on deaf ears. It’s tough to imagine a scenario in which AI companies not only agree to “support a culture of open criticism,” as the undersigned recommend, but also opt not to enforce nondisparagement clauses or retaliate against current staff who choose to speak out.

Consider that OpenAI’s safety commission, which the company recently created in response to initial criticism of its safety practices, is staffed with all company insiders — including CEO Sam Altman. And consider that Altman, who at one point claimed to have no knowledge of OpenAI’s restrictive nondisparagement agreements, himself signed the incorporation documents establishing them.

Sure, things at OpenAI could turn around tomorrow — but I’m not holding my breath. And even if they did, it’d be tough to trust it.

News

AI apocalypse: OpenAI’s AI-powered chatbot platform, ChatGPT — along with Anthropic’s Claude and Google’s Gemini and Perplexity — all went down this morning at roughly the same time. All the services have since been restored, but the cause of their downtime remains unclear.

OpenAI exploring fusion: OpenAI is in talks with fusion startup Helion Energy about a deal in which the AI company would buy vast quantities of electricity from Helion to provide power for its data centers, according to the Wall Street Journal. Altman has a $375 million stake in Helion and sits on the company’s board of directors, but he reportedly has recused himself from the deal talks.

The cost of training data: TechCrunch takes a look at the pricey data licensing deals that are becoming commonplace in the AI industry — deals that threaten to make AI research untenable for smaller organizations and academic institutions.

Hateful music generators: Malicious actors are abusing AI-powered music generators to create homophobic, racist and propagandistic songs — and publishing guides instructing others how to do so as well.

Cash for Cohere: Reuters reports that Cohere, an enterprise-focused generative AI startup, has raised $450 million from Nvidia, Salesforce Ventures, Cisco and others in a new tranche that values Cohere at $5 billion. Sources familiar tell TechCrunch that Oracle and Thomvest Ventures — both returning investors — also participated in the round, which was left open.

Research paper of the week

In a research paper from 2023 titled “Let’s Verify Step by Step” that OpenAI recently highlighted on its official blog, scientists at OpenAI claimed to have fine-tuned the startup’s general-purpose generative AI model, GPT-4, to achieve better-than-expected performance in solving math problems. The approach could lead to generative models less prone to going off the rails, the co-authors of the paper say — but they point out several caveats.

In the paper, the co-authors detail how they trained reward models to detect hallucinations, or instances where GPT-4 got its facts and/or answers to math problems wrong. (Reward models are specialized models to evaluate the outputs of AI models, in this case math-related outputs from GPT-4.) The reward models “rewarded” GPT-4 each time it got a step of a math problem right, an approach the researchers refer to as “process supervision.”

The researchers say that process supervision improved GPT-4’s math problem accuracy compared to previous techniques of “rewarding” models — at least in their benchmark tests. They admit it’s not perfect, however; GPT-4 still got problem steps wrong. And it’s unclear how the form of process supervision the researchers explored might generalize beyond the math domain.

Model of the week

Forecasting the weather may not feel like a science (at least when you get rained on, like I just did), but that’s because it’s all about probabilities, not certainties. And what better way to calculate probabilities than a probabilistic model? We’ve already seen AI put to work on weather prediction at time scales from hours to centuries, and now Microsoft is getting in on the fun. The company’s new Aurora model moves the ball forward in this fast-evolving corner of the AI world, providing globe-level predictions at ~0.1° resolution (think on the order of 10 km square).

AI model atmosphere
Image Credits: Microsoft

Trained on over a million hours of weather and climate simulations (not real weather? Hmm…) and fine-tuned on a number of desirable tasks, Aurora outperforms traditional numerical prediction systems by several orders of magnitude. More impressively, it beats Google DeepMind’s GraphCast at its own game (though Microsoft picked the field), providing more accurate guesses of weather conditions on the one- to five-day scale.

Companies like Google and Microsoft have a horse in the race, of course, both vying for your online attention by trying to offer the most personalized web and search experience. Accurate, efficient first-party weather forecasts are going to be an important part of that, at least until we stop going outside.

Grab bag

In a thought piece last month in Palladium, Avital Balwit, chief of staff at AI startup Anthropic, posits that the next three years might be the last she and many knowledge workers have to work thanks to generative AI’s rapid advancements. This should come as a comfort rather than a reason to fear, she says, because it could “[lead to] a world where people have their material needs met but also have no need to work.”

“A renowned AI researcher once told me that he is practicing for [this inflection point] by taking up activities that he is not particularly good at: jiu-jitsu, surfing, and so on, and savoring the doing even without excellence,” Balwit writes. “This is how we can prepare for our future where we will have to do things from joy rather than need, where we will no longer be the best at them, but will still have to choose how to fill our days.”

That’s certainly the glass-half-full view — but one I can’t say I share.

Should generative AI replace most knowledge workers within three years (which seems unrealistic to me given AI’s many unsolved technical problems), economic collapse could well ensue. Knowledge workers make up large portions of the workforce and tend to be high earners — and thus big spenders. They drive the wheels of capitalism forward.

Balwit makes references to universal basic income and other large-scale social safety net programs. But I don’t have a lot of faith that countries like the U.S., which can’t even manage basic federal-level AI legislation, will adopt universal basic income schemes anytime soon.

With any luck, I’m wrong.

More TechCrunch

Fisker is just a few days into its Chapter 11 bankruptcy, and the fight over its assets is already charged, with one lawyer claiming the startup has been liquidating assets…

The fight over Fisker’s assets is already heating up

A hacker is advertising customer data allegedly stolen from the Australia-based live events and ticketing company TEG on a well-known hacking forum. On Thursday, a hacker put up for sale…

Hacker claims to have 30 million customer records from Australian ticket seller giant TEG

Welcome to Startups Weekly — Haje‘s weekly recap of everything you can’t miss from the world of startups. Sign up here to get it in your inbox every Friday. Elon…

Tesla makes Musk best-paid CEO of all time and Fisker bites the dust

Dot is a new AI companion and chatbot that thrives on getting to know your innermost thoughts and feelings.

Dot’s AI really, really wants to get to know you

The e-fuels startup is working on producing fuel for aviation and maritime shipping using carbon dioxide and other waste carbon streams.

E-fuels startup Aether Fuels is raising $34.3 million, per filing

Fisker was facing “potential financial distress” as early as last August, according to a new filing in its Chapter 11 bankruptcy proceeding, which the EV startup initiated earlier this week.…

Fisker faced financial distress as early as last August

Cruise, the self-driving subsidiary of General Motors, has agreed to pay a $112,500 fine for failing to provide full information about an accident involving one of its robotaxis last year.…

Cruise clears key hurdle to getting robotaxis back on roads in California

Feel Therapeutics has a pretty original deck, with some twists we rarely see; the company did a great job telling the overall story.

Pitch Deck Teardown: Feel Therapeutics’ $3.5M seed deck

The Rockset buy fits into OpenAI’s broader recent strategy of investing heavily in its enterprise sales and tech orgs.

OpenAI buys Rockset to bolster its enterprise AI

The U.S. government announced sanctions against 12 executives and senior leaders of the Russia-based cybersecurity giant Kaspersky. In a press release, the Department of the Treasury’s Office of Foreign Assets…

US government sanctions Kaspersky executives

Style DNA, an AI-powered fashion stylist app, creates a personalized style profile from a single selfie. The app is particularly useful for people interested in seasonal color analysis, a process…

Style DNA gets a generative AI chatbot that suggests outfit ideas based on your color type

Rates of depression, anxiety and suicidal thoughts are surging among U.S. teens. A recent report from the Center of Disease Control found that nearly one in three girls have seriously…

Khosla-backed Marble, built by former Headway founders, offers affordable group therapy for teens

Cover says what sets it apart is the underlying technology it employs, which has been exclusively licensed from NASA’s Jet Propulsion Laboratory.

A new startup from Figure’s founder is licensing NASA tech in a bid to curb school shootings

Spotify is introducing a new “Basic” streaming plan in the United States, the company announced on Friday. The new plan costs $10.99 per month and includes all of the benefits…

Spotify launches a new Basic streaming plan in the US

Photographers say the social media giant is applying a ‘Made with AI’ label to photos they took, causing confusion for users.

Meta is tagging real photos as ‘Made with AI,’ say photographers

Website building platform Squarespace is selling Tock, its restaurant reservation service, to American Express in a deal worth $400 million — the exact figure that Squarespace paid for the service…

Squarespace sells restaurant reservation system Tock to American Express for $400M

Featured Article

Change Healthcare confirms ransomware hackers stole medical records on a ‘substantial proportion’ of Americans

The February ransomware attack on UHG-owned Change Healthcare stands as one of the largest-ever known digital thefts of U.S. medical records.

19 hours ago
Change Healthcare confirms ransomware hackers stole medical records on a ‘substantial proportion’ of Americans

Google said today that it globally paused its experiment that aimed to allow new kinds of real-money games on the Play Store, citing the challenges that come with the lack…

Google pauses its experiment to expand real-money games on the Play Store

Venture firms raised $9.3 billion in Q1 according to PitchBook data, which means this year likely won’t match or surpass 2023’s $81.8 billion total. While emerging managers are feeling the…

Kevin Hartz’s A* raises its second oversubscribed fund in three years

Google is making reviews of all your movies, TV shows, books, albums and games visible under one profile page starting June 24, according to an email sent to users last…

Google is making your movie and TV reviews visible under a new profile page

Zepto, an Indian quick commerce startup, has more than doubled its valuation to $3.6 billion in a new funding round of $665 million.

Zepto, a 10-minute delivery app, raises $665M at $3.6B valuation

Speak, the AI-powered language learning app, has raised new money from investors at double its previous valuation.

Language learning app Speak nets $20M, doubles valuation

SpaceX unveiled Starlink Mini, a more portable version of its satellite internet product that is small enough to fit inside a backpack.  Early Starlink customers were invited to purchase the…

SpaceX debuts portable Starlink Mini for $599

Ali Rathod-Papier has stepped down from her role as global head of compliance at corporate card expense management startup Brex to join venture firm Andreessen Horowitz (a16z) as a partner…

Brex’s compliance head has left the fintech startup to join Andreessen Horowitz as a partner

U.S. officials imposed the “first of its kind” ban arguing that Kaspersky threatens U.S. national security because of its links to Russia.

US bans sale of Kaspersky software citing security risk from Russia 

Apple has released Final Cut Pro for iPad 2 and Final Cut Camera, the company announced on Thursday. Both apps were previously announced during the company’s iPad event in May.…

Apple releases Final Cut Pro for iPad 2 and Final Cut Camera

Paris has quickly established itself as a major European center for AI startups, and now another big deal is in the works.

Poolside is raising $400M+ at a $2B valuation to build a supercharged coding co-pilot

The space industry is all abuzz about how SpaceX’s Starship, Blue Origin’s New Glenn, and other heavy-lift rockets will change just about everything. One likely consequence is that spacecraft will…

Gravitics prepares a testing gauntlet for a new generation of giant spacecraft

LTK (formerly LiketoKnow.it and RewardStyle), the influencer shopping app with 40 million monthly users, announced on Thursday the launch of a free direct message tool for creators to instantly share…

Influencer shopping app LTK gets an automatic direct message tool

YouTube appears to be taking a firm stance against Premium subscribers who attempt to use a VPN (virtual private network) to access cheaper subscription prices in other countries. This week,…

YouTube confirms crackdown on VPN users accessing cheaper Premium plans