The Daily: AI—Are investments paying off, how is it overrated, and are we approaching a bubble?

Audio by Marcus Johnson, Gadjo Sevilla, and Grace Harmon | Aug 22, 2024

Episode Transcript:

Marcus Johnson (00:00):

B2B marketing teams rely on EMARKETER Media Solutions to elevate their thought leadership and build meaningful relationships through exclusive webinars, guides, infographics and more, if you can believe it. Visit emarketer.com/advertise to learn more about our proven approach.

Grace Harmon (00:16):

So I do think part of it is just the reality of the technology is that the tech that we have isn't perfect because we can't get it there yet, and consumers/investors aren't going to wait five years for us to have the OpenAI that doesn't say scary or inaccurate things sometimes.

Marcus Johnson (00:38):

Hey, gang. It's Thursday, August 22nd. Grace, Gadjo, and listeners, welcome to the Behind the Numbers Daily, and EMARKETER Podcast. I'm Marcus. Today, I'm joined by two people. Let's welcome to the show for the first time one of our analysts who writes for our tech briefing. It's Grace Harmon.

Grace Harmon (00:55):

Hello.

Marcus Johnson (00:55):

Hello there. Welcome to the show. She's based in California on the other coast. It's her colleague who's joining us today, a senior analyst who also writes for the tech briefings. He's in New York. It's Gadjo Sevilla.

Gadjo Sevilla (01:07):

Hi, Marcus. Happy to be here once again.

Marcus Johnson (01:09):

Hey, chap. Good to have you. All right, today's fact, it's not really a fact. Today's research: One-quarter of British people think they could have qualified for the Olympics. What are we talking about? That's too high.

Grace Harmon (01:27):

For any sport?

Gadjo Sevilla (01:28):

Exactly.

Marcus Johnson (01:30):

Yes, for a sport. I've got it by sport here. So the one that people thought that would be the easiest to qualify for was a air rifle.

Gadjo Sevilla (01:39):

Cool.

Marcus Johnson (01:41):

15% of British people said they could definitely or probably qualify for that event, 13% for archery, 10% for badminton. What's going on there? 6% think they would make the 100-meter sprint. That sounds like not many people. 6% of the population in England, or the UK, is four million people. So four million people were sat at home thinking, "I could run that fast."

Grace Harmon (02:13):

When I watched, I think they were going something like 26 miles an hour. It's really fast.

Marcus Johnson (02:19):

Exactly. Who thinks they can run that fast who sat at home? You can't. Okay? Whatever you're thinking, you can't. Three million Brits fancy themselves as Olympic shot putters. The same number think they could windsurf at the Olympic level. This is too good. And two million think they missed their calling as a top-tier rhythmic gymnast.

Gadjo Sevilla (02:41):

How about breaking?

Marcus Johnson (02:46):

If more than one person said yes, it's too high. Great Britain. It's good to have ambition, but stop it because you're out of your mind. Anyway, today's real topic: Are we barrelling towards an AI bubble? In today's episode, first in the lead, we'll cover AI hype. How much of it is there? Is it too much? Not enough, maybe? No "In other news" today, so let's get to it. So folks, in some recent research, you both explored the question of, "Is AI driving Big Tech earnings or starting to fade fast?" And you note, "Since OpenAI released ChatGPT close to two years ago now," somehow, "money is poured into trying to build the smartest LLM, large language model, to power these GenAI chatbots." But we'll start with kind of just the overarching question theme of your research. Gadjo, I'll start with you. Are Big Tech's AI investments paying off?

Gadjo Sevilla (03:45):

Yeah. So as evidenced by the recent earnings reports, the answer is no. Not yet. So Big Tech has had less than two years to expand into AI and we're really just seeing the beginning of those benefits and those products, but not the profits. So the earnings reveal that AI spending is slow to yield profits. They're saying it could take significantly longer to reap from those specific AI investments. In fact, Gartner has a timeline saying that we're two years away from seeing the full potential of AI adoption. So I mean, the reality we're facing now is while businesses are investing billions into AI, they're getting tripped up by the fact that most of the employees in these businesses lack the sufficient training to implement the technology, and that takes a bit more time.

Marcus Johnson (04:39):

So I mean, Grace, is this an unfair question to even ask because it is still recently new, right? I mean, Chat Gp just came out less than 24 months ago and everyone's expecting them to be making money, to be worth the investment already. Big Tech thinks it's got no choice but to invest, it seems. CEO of Google, Sundar Pichai, is saying, in a recent earnings call, "The risk of under-investing is dramatically greater than the risk of over-investing." So it seems like it's not about whether to run the race or not, but about what pace are you running at? Because if you start out too fast, you'll burn through your cash before you reach a finish line. But if you don't run fast enough, you end up kind of behind the leader of the pack and that's going to be detrimental going forward because customers won't think of you when they think of AI. So is this even a fair question to ask?

Grace Harmon (05:25):

Well, I mean, the CEOs at Google and Meta aren't just talking about the dangers of over-investing. They've said that they have over-invested. They've said that arguably they have spent too much on AI ...

Marcus Johnson (05:26):

Okay.

Grace Harmon (05:36):

But it's just too dangerous to not do it. If you fall behind, I don't know how reasonably you can catch up. When we were looking at the number of times that executives were talking about AI on investment calls, earnings calls from 2022 to 2024, some companies we saw get on that really, really fast, like Meta. I think it was something around 30 times it had brought up AI just in Q3 of 2022, and that's ramped up a lot into the 100s. But some companies really just didn't start talking about it at all. I don't know. You can't necessarily say that doesn't translate to they weren't talking about it internally, but some companies are behind and it is really hard to catch up. So I think at this point, if you are a bit behind and you don't put the money in, I don't know how reasonably you can catch up.

Marcus Johnson (06:22):

Yeah. Big Tech, they seem committed. And based on that, looking at how many times AI was mentioned in the earnings call, which is something, in exercise, you guys did, I went and counted. So in the last two years, from two years ago, Q3 2022, it was mentioned 57 times across Meta, Google, Microsoft, Amazon, and Apple that you guys have in your research. So 57 times. This last quarter, Q2 two years later, it was mentioned nearly 500 times. So they seem like they want this to be a thing. Gadjo, one of the problems, a huge problem here, is the costs. AI costs are so high. It costs a lot to run, let alone train these models.

(06:57):

SemiAnalysis Chief Analyst Dylan Patel estimates that this year, it could cost OpenAI $700,000 a day to run Chat GPT. That's a quarter of a billion dollars just to run the product. The smarter it gets, the more it's going to cost to run. And the big question here is how long can you hemorrhage this much money for, right? The Information reporting recently that OpenAI could lose as much as $5 billion this year. That's close to 10 times what it lost in 2022. So how is this sustainable in knowing that the costs are probably only going to go up and the returns are so much further in the future?

Gadjo Sevilla (07:29):

First of all, I agree with you. The costs are only going to be going up. I mean, just look at the magnificent seven mega-caps. They're all planning more investment. How sustainable it is, I think it really depends on what product they're selling. If we're talking about general AI, which is what we have now, they're trying different things. But I think for the more focused companies that have a specific market and a specific product, the spending could taper down once they start bringing in the subscriptions and start getting those large enterprise contracts. And again, that will depend on how quickly they can really provide a competitive and compelling product.

Marcus Johnson (08:15):

Yeah. I mean, the big question that keeps getting asked, at least more and more recently in the last couple of months, is this idea of an AI bubble and whether it's going to pop. It was a question in Bryan Walsh's piece at Vox. One of the many folks who seems to think that the bill may be coming due, he writes, for Silicon Valley's huge investment in AI. Grace, is there an AI bubble? Do you agree with this take?

Grace Harmon (08:41):

I think that Gadjo mentioned this earlier, but there's billions of dollars going into AI, and that's right now, and right now, as we've said, just aren't seeing the returns. So I think it just depends on how many of the investors have an enormous amount of patience.

Marcus Johnson (08:59):

Right.

Grace Harmon (09:00):

Because it isn't really here yet to the point of ... At OpenAI, Sam Altman had said we're five years away, maybe at least, maybe more, from having AGI. So if the tech isn't there, just in terms of intelligence, I think that says a lot about what you can ask consumers to pay for it. So I do think it just comes down to how patient the market can be, and I don't think that it will be that patient.

Marcus Johnson (09:01):

Yeah.

Gadjo Sevilla (09:27):

Yeah. Wall Street is not very patient with these things.

Grace Harmon (09:27):

No.

Gadjo Sevilla (09:31):

I mean, you can see how there was the initial excitement and all these investments. But they want a quick turnaround, which this is not the case for this kind of technology, not at this scale at least.

Marcus Johnson (09:44):

Yeah.

Grace Harmon (09:45):

And a lot of companies don't have the option to wait a really long time. But if we're talking about the Big Tech companies, we're talking about Apple, Amazon, Google, they have the wallets to wait a long time. But a lot of startups are going down, and it's about 60%. The rate of startup closeups raised to I think it was 60% year-over-year in Q1 of this year. It's really expensive to build a model. It's really expensive to deploy. It's really expensive to maintain it and improve it because they aren't perfect. And it isn't about just the patience of the CEO of Google. If you are not that wealthy of a company, then you aren't going to have the option of having an imperfect financial model.

Marcus Johnson (10:28):

Yeah. Yeah. ,I mean it does seem quite exclusionary, doesn't it? You just have to have the capital to be able to continue to invest for as long as possible, basically, to outlast your competitors, your opponents here. I mean, the AI money seems to be flowing, at least for now, in the US AI startups, raising $23 billion in capital last year. So there's money there. But I mean, Gadjo, to your point, it seems like Wall Street shareholders, less and less patience. We've seen that in Big Tech earnings with companies making X number of billions of dollars per quarter and it is not good enough. There's a certain expectation.

Gadjo Sevilla (10:59):

Yeah, it's never good enough.

Marcus Johnson (11:01):

Yeah. Well, so speaking of never good enough, let's talk about how AI might be overrated. This was an article by Greg Rosalsky of NPR. He does explicitly say, "I was assigned to write this." So he's not saying that it necessarily is overrated, but one of his assignments was to write an article outlining 10 reasons why AI might be overrated. He notes that investors have poured billions and billions of dollars into tech, as we've mentioned, and there's a growing chorus though of naysayers that are expressing doubts over it. One of the most prominent voices is MIT economist Daron Acemoglu who, when asked if AI would bring revolutionary changes to the economy within 10 years, replied, "No, no. Definitely not. I mean, unless you count a lot of companies over-investing in GenAI and then regressing it a revolutionary change."

(11:49):

Mr. Rosalsky's list of why AI might be overrated, the 10 things, a few of them are the fact that GenAI lies or hallucinates a lot, and another one is AI is bad for the environment because it consumes more energy than some small countries. There's obviously a bunch of others in here as well. Grace, I'll start with you. Which of these 10 reasons in the piece do you think is the main way that AI might be overrated?

Grace Harmon (12:12):

I think, in my head, the climate effects are one of the scariest because it's enormous the amount of water, the amount of heat, the amount of power that you need to keep these systems running, keep them running correctly. But I would say that AI's tendency to lie, to hallucinate, and just to not work super well is probably one of the main reasons. I mean, we saw at the Made by Google demo last week that when they brought forward one of the GenAI models to do some photo analysis, it didn't work for a minute. And the presenter handled it well, but I think it took three tries and a device change for it to work. And there's definitely pressure for companies to put out their products even if they're not perfect yet, but we're really far away from perfection. So I do think part of it's just the reality of the technology is that the tech that we have isn't perfect because we can't get it there yet, and consumers/investors aren't going to wait five years for us to have the OpenAI that doesn't say scary or inaccurate things sometimes.

Marcus Johnson (13:14):

Yeah. Gadjo, let's stay on that for a second because I'm wondering how are consumers supposed to know how good these technologies are? Because to Grace's point, they don't seem like they're as good as they're supposed to be. You do see these releases coming out and they say, "Oh, it was good enough to pass the bar exam in the bottom 10% of the class." New version, "Okay, now it's good enough to pass the bar exam, the law exam, in the top 10% of the class." But there are some instances where, for example, writing code, there's some research to suggest that much of the code that's written is not very good, producing more low-quality code. There's a study from Stanford University which found that coders who used AI assistants wrote significantly less-secure code. And there's researchers at Bilkent University in Turkey, they found over 30% of AI-generated code was incorrect and another 23% on top of that was partially incorrect. So that's over half of AI-generated code that's incorrect or partially so. How do folks know how to gauge how good or bad this stuff is yet?

Gadjo Sevilla (14:12):

See, that's a key problem, just bouncing back to what Grace has said about the struggle to productize AI. By the way, I chose number six ...

Marcus Johnson (14:21):

Okay.

Gadjo Sevilla (14:21):

Meaning we have yet to find an AI scaler app, and there's a lot of evidence of that. In the past year alone, we've seen products that whose key feature is AI. The Humane Pin. I can go on and on. There are all these products that they're just basically vessels for AI and as products, they've failed because of so many reasons. AI is inaccurate. It requires constant connection to the cloud, which is difficult when you're thinking about wearables. So there's still nothing that can really capture consumers in the sense that, "Oh, yeah. I do need this in my life and I'm willing to pay for it."

(15:02):

Right now, it feels it's a lot of small experiments. But as it says, no killer app yet. We might see that in enterprise when we have highly-specialized AI applications and we're seeing some companies come up with these, but these aren't off the shelf. You have to develop them. You have to get your staff to work on them. They have to be secure. There's all these steps that you need to kind of cover before you can say that this AI is a killer app. And it's not going to be a forever thing. It's going to be a killer app for your needs right now. Once those needs evolve, the AI has to evolve as well.

Marcus Johnson (15:38):

Yeah.

Grace Harmon (15:39):

It's interesting. One of the examples you had there was the Humane AI Pin because that one is doing so poorly, and I think one of the biggest issues that consumers have had with it is that it's overheating.

Marcus Johnson (15:49):

Oh, interesting.

Grace Harmon (15:49):

And so even if you get the AI that good with wearables, I do think that's just another challenge of you have to get the killer app and you have to find the killer way to have it work.

Marcus Johnson (15:58):

Yeah. Yeah, and there was a line in Bryan Walsh's article at Vox saying, "chatbots aren't yet a true product and it's not clear yet how big the market is for them," so that's something people are struggling with. Now, Kelsey Piper, who also writes for Vox, was saying, "Look. Electricity was invented, and then later on we figured out how to use it," all the different use cases. So again, maybe it's unfair to AI to say, "Hey. What are we supposed to be using you for?" and it's like, "Wait a second. I'm only two years old." But there are indicators to suggest that it is overrated, that maybe there is a bit of a bubble. Two things to that point.

(16:33):

One, AI isn't being used with much or as much regularity as we might think. There's a lot of surveys showing how many people use it. We have some numbers on ChatGPT users ourselves and GenAI users. But that's once a month, so it's not like it's part of their daily routine. A recent US census study found only 5% of businesses had used AI in the previous couple of weeks. Our numbers, I think, is close to 100 million people we're expecting over this year or next year to be using GenAI. But again, that's once a month. And then also, the rate of progress slowing down. They're trained on lots of data, but we've already vacuumed up the whole internet. And so there's a lot of copyright questions as well about some of the other data, but that also, I guess, is an issue, right, folks? You need more and more data to train them, but what happens when you've used it all?

Grace Harmon (17:20):

The model training part is tricky, I mean, especially depending on what medium you're trying to develop. If you're talking about music generation models, the issue is that there just isn't an enormous database of licensed content. So a lot of these companies are happening to go bit by bit, whether for a text generation, for a chatbot, for music generation to find a big enough database to train a model. And a lot of companies aren't being able to find that and they're taking routes they shouldn't and getting their hands slapped legally. I think that's one of the tough parts here. I think that could be a place outside of AI that we could see more products coming out is more companies finding legal ways just to offer training resources, even if they're not making their own models.

Marcus Johnson (17:20):

Yeah.

Gadjo Sevilla (18:05):

That's another expense, paying for quality training data, which it's a limited resource, as far as I know. And so for a lot of companies, they'll be exposed to vast stores of data which maybe aren't vetted. And the old saying, "Garbage in, garbage out," that eventually shows up when you're productizing this stuff.

Marcus Johnson (18:30):

Yeah. Yeah. I'll end with this. Greg Rosalsky of NPR, who wrote this 10 ways or reasons why AI might be overrated, he spoke to, as I mentioned, MIT economist Daron Acemoglu, who said that his number one reason that AI is overrated in his opinion was because humans are underrated. He said, "A lot of people in the industry don't recognize how versatile, talented, and multifaceted human skills and capabilities are. And once you do that, you tend to overrate machines ahead of humans and underrate humans." Gadjo and Grace's full report is called, "Is AI Driving Big Tech Earnings or Fading Fast?" Pro/Plus users, you can head to emarketer.com, of course, or there's a link in the show notes. That's all we've got time for for today's episode though. Thank you so, so much to my guests today. Thank you to Grace.

Grace Harmon (19:17):

Yeah, thank you. Nice talking with you guys.

Marcus Johnson (19:19):

Yes, indeed. Thank you for being here. Thank you to Gadjo.

Gadjo Sevilla (19:21):

Thank you. This was a great conversation. Let's do it again.

Marcus Johnson (19:24):

Yes, indeed. Yes, indeed. Thank you to Victoria. She edits the show. Stuart runs the team. Sophie does our social media. And thanks to everyone for listening in, of course. We hope to see you tomorrow for the Behind the Numbers Weekly Listen. That is an EMARKETER video podcast. You can watch along on YouTube or you can listen to it the usual way.

MUSIC (19:42):

(Instrumental music)